[英]Is it safe to run conda-build in parallel?
First question, please yell at me if I'm not following any established norms here:)第一个问题,如果我不遵守这里的任何既定规范,请对我大喊大叫:)
I'm setting up a full conda-build pipeline for the first time and I've got a Makefile that looks something like this:我第一次设置了一个完整的 conda 构建管道,我有一个看起来像这样的 Makefile:
build:
echo "Building packages."
conda-build my_first_package/conda-recipe --output-folder /path/to/dev/channel
conda-build my_second_package/conda-recipe --output-folder /path/to/dev/channel
conda-build my_third_package/conda-recipe --output-folder /path/to/dev/channel
conda index /path/to/dev/channel
The three packages in question are pretty tightly connected to each other and are being stored in the same repo so it would be nice to build each one in the same pipeline.有问题的三个包彼此紧密相连,并且存储在同一个 repo 中,因此最好在同一个管道中构建每个包。 What I'd like to do is spin off a separate process for each of those and run each conda-build command in parallel.
我想做的是为每个进程分拆一个单独的进程并并行运行每个 conda-build 命令。 Not sure if this is a safe action to take though as I don't really understand what conda-build is doing to "build" the package.
不确定这是否是一个安全的操作,因为我真的不明白 conda-build 正在做什么来“构建”package。
I'll be testing this idea later today and will post an update but what I'm really worried about is that it will work with some unintended side effects and I'm not sure how to go about testing for those.我将在今天晚些时候测试这个想法并将发布更新,但我真正担心的是它会产生一些意想不到的副作用,而且我不确定如何测试 go。 So I think my question is: Can conda-build build multiple packages in parallel into the same conda channel?
所以我认为我的问题是: conda-build 可以将多个包并行构建到同一个 conda 通道中吗?
Alternatively, for very tightly coupled packages one may want to look into single builds with multiple outputs.或者,对于非常紧密耦合的包,可能需要查看具有多个输出的单个构建。 For reference, see the Outputs Section documentation of
conda-build
.有关参考,请参阅
conda-build
的输出部分文档。 It may also be worth checking out a full-blown example, like Conda Forge's matplotlib-feedstock
, which builds matplotlib
, matplotlib-base
, and mpl-sample-data
all from a single recipe.还可能值得查看一个完整的示例,例如Conda Forge 的
matplotlib-feedstock
feedstock ,它从单个配方构建matplotlib
、 matplotlib-base
和mpl-sample-data
。
In this particular case, I'd imagine something like在这种特殊情况下,我想像
meta.yaml元.yaml
...
outputs:
- name: my_first_package
...
- name: my_second_package
...
- name: my_third_package
...
...
then run a single conda build
command.然后运行一个
conda build
命令。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.