recipes.py ignores --no-fetch and --deps-path

[erock2112]

eg. in the Skia repo:

$ mkdir /tmp/recipe_deps
$ infra/bots/recipes.py --deps-path=/tmp/recipe_deps fetch
...
$ infra/bots/recipes.py --deps-path=/tmp/recipe_deps --no-fetch simulation_test train
(clones recipe_engine into infra/bots/.recipe_deps without looking for it in /tmp/recipe_deps)

The clone is taking a long time on our raspberry pi bots, and we'd like to package the recipe_deps as a CIPD package, and then run recipes.py in our swarming task with --no-fetch. CIPD doesn't like to install to skia/infra/bots/.recipe_deps, so we have to install it elsewhere and use --deps-path to point recipes.py to the right place, but because of this bug we clone the recipe engine unnecessarily even when the user requested us not to.

[martiniss]

The recipe engine wrapper you have in your repo (copy in this repo is at doc/recipes.py) automatically tries to fetch the recipe engine to .recipe_deps. It doesn't try to detect the --deps-path argument, which is a bug. Will fix

[vadimsht]

"CIPD doesn't like to install to skia/infra/bots/.recipe_deps"

what do you mean by that?

[erock2112]

Now that I look again, it seems to be isolate failing to write files after CIPD has written into skia/infra/bots/.recipe_deps: https://chromium-swarm.appspot.com/user/task/31a4a3c1b2bbc310. Anyway, the nesting causes problems.

[vadimsht]

Yeah, isolate probably expects clean directory :-/

[riannucci]

FYI, I'm adding a 'bundle' command to recipes which will generate a stand-alone hermetic version of the recipes in a given repo. It is designed with these sorts of uses in mind (fetching in a single location, totally offline operation, compatibility with isolate). I have a preliminary CL up here: https://chromiumcodereview.appspot.com/2352163003/

[erock2112]

Sounds great!

[erock2112]

Friendly ping. This is causing us a lot of pain and we'd like to get rid of the transparent fetching.

[martiniss]

I think the CL (https://chromiumcodereview.appspot.com/2352163003/) is good to go. Pinged it

[riannucci]

unfortunately that CL is not good to go yet, and I stalled on it. it's still on my radar though and I'll be actually fixing it this quarter.

[riannucci]

This is actually fixed now (forgot to reference this bug!):

• 5e45bcc
• fe33522

[erock2112]

As of this morning, recipes.py is still ignoring --no-fetch:

$ infra/bots/recipes.py --deps-path=/tmp/recipe_deps --no-fetch simulation_test train
Cloning into '/usr/local/google/home/borenet/go/src/go.skia.org/infra/infra/bots/.recipe_deps/recipe_engine'...
remote: Total 6188 (delta 3847), reused 6188 (delta 3847)
Receiving objects: 100% (6188/6188), 6.78 MiB | 0 bytes/s, done.
Resolving deltas: 100% (3847/3847), done.
WARNING:root:(Not Bad) Using custom deps path: /tmp/recipe_deps
......
----------------------------------------------------------------------
Trained 6 tests in 0.237s (ran 6 tests)