[英]run a crontab job using an anaconda env
I want to have a cron job execute a python script using an already existing anaconda python environment called my_env.我想让一个 cron 作业使用已经存在的名为 my_env 的 anaconda python 环境执行 python 脚本。 The only thing I can think to do is have the cron job run a script called my_script.bash
which in turn activates the env and then runs the python script.我唯一能想到的就是让 cron 作业运行一个名为my_script.bash
的脚本,该脚本反过来激活 env,然后运行 python 脚本。
#!/bin/bash
source activate my_env
python ~/my_project/main.py
Trying to execute this script from the command lines doesn't work:尝试从命令行执行此脚本不起作用:
$ sh scripts/my_script.bash
scripts/my_script.bash: 9: scripts/my_script.bash: source: not found
What do I need to do to make sure the proper environment is activated.我需要做什么来确保激活正确的环境。 Its ok to explain it to me like I'm 5.可以像我 5 岁一样向我解释。
I recently switched from canopy to Anaconda precisely to get away from having to activate an env in cron jobs.我最近从canopy切换到 Anaconda 正是为了避免在 cron 作业中激活环境。 Anaconda makes this very simple, based on the PATH enviornment variable. Anaconda 基于 PATH 环境变量使这变得非常简单。 (I'm using miniconda not the full Anaconds install but I believe anaconda should work the same way) (我使用的是miniconda而不是完整的 Anaconda 安装,但我相信 anaconda 应该以同样的方式工作)
There are two different approaches, I've tested;我测试过两种不同的方法;
Add a shebang in your python script, main.py在你的 python 脚本 main.py 中添加一个 shebang
#!/home/users/user_name/miniconda2/envs/my_env/bin/python
Add PATH to the top of your crontab将 PATH 添加到 crontab 的顶部
PATH=/home/users/user_name/miniconda2/envs/my_env/bin
Update:更新:
Jérôme's answer and cbarrick's comments are correct. Jérôme 的回答和 cbarrick 的评论是正确的。 I just got burned using the above approach in a Conda env which needed pynco,
which needs the full conda environment to find proper the nco
commands, such as ncks, ncrcat.
我刚刚在需要pynco,
它需要完整的 conda 环境来找到正确的nco
命令,例如ncks, ncrcat.
Solved by running a bash script from cron which calls conda activate first.通过从 cron 运行 bash 脚本来解决,该脚本首先调用 conda activate。
After MUCH fiddling I got crontab
to activate my conda environment with conda activate my_env
and run the Python interpreter within that environment.经过大量摆弄后,我让crontab
使用conda activate my_env
激活我的 conda 环境,并在该环境中运行 Python 解释器。
Note I'm using Ubuntu 18.04.注意我使用的是 Ubuntu 18.04。
Background背景
When the Anaconda installer initializes conda, it appends a snippet at the end of the ~/.bashrc
file.当 Anaconda 安装程序初始化 conda 时,它会在~/.bashrc
文件的末尾附加一个片段。 This file is executed each time the user opens bash
interactively.每次用户以交互方式打开bash
时都会执行此文件。 The snippet allows the user to run conda
commands (ie conda activate my_env
) from bash
.该片段允许用户从bash
运行conda
命令(即conda activate my_env
)。
Anaconda installer v2020.02 appended the following conda
snippet in ~/.bashrc
: Anaconda 安装程序 v2020.02在~/.bashrc
中附加了以下conda
代码段:
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/opt/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/opt/anaconda3/etc/profile.d/conda.sh" ]; then
. "/opt/anaconda3/etc/profile.d/conda.sh"
else
export PATH="/opt/anaconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<
/opt/anaconda3/
to be replaced with the correct reference: usually /home/USERNAME/anaconda3/
.将路径/opt/anaconda3/
替换为正确的引用:通常是/home/USERNAME/anaconda3/
。The problem问题
Sourcing ~/.bashrc
in crontab -e
won't work (at least not on Ubuntu).在crontab -e
中采购~/.bashrc
将不起作用(至少在 Ubuntu 上不起作用)。
Explanation:解释:
~/.bashrc
has the following (or similar) line at the beginning of the file:在 Ubuntu 上, ~/.bashrc
在文件的开头有以下(或类似的)行:# If not running interactively, don't do anything
[ -z "$PS1" ] && return
~/.bashrc
file in crontab
, the rest of the .bashrc
file will not execute because crontab
is not running interactively (see another post on this topic ).这意味着如果我们尝试在crontab
中获取~/.bashrc
文件,则.bashrc
文件的其余部分将不会执行,因为crontab
没有以交互方式运行(请参阅有关此主题的另一篇文章)。 Which means that the conda
snippet mentioned above will never get executed by crontab
even if we source ~/.bashrc
.这意味着即使我们 source ~/.bashrc
,上面提到的conda
片段也永远不会被crontab
执行。 The solution I have found is to copy the conda
snippet to a separate file.我找到的解决方案是将conda
片段复制到单独的文件中。
conda
snippet from ~/.bashrc
to ~/.bashrc_conda
1. 将conda
片段从~/.bashrc
复制到~/.bashrc_conda
Copy the snippet mentioned above to another file, for example ~/.bashrc_conda
.将上述代码段复制到另一个文件,例如~/.bashrc_conda
。
Ensure that:确保这件事:
crontab -e
insert 2 lines to run bash
instead of sh
and to source ~/.bashrc_conda
2. 在crontab -e
中插入 2 行来运行bash
而不是sh
和 source ~/.bashrc_conda
Run crontab -e
and add the following 2 lines before the cronjob :运行crontab -e
并在 cronjob 之前添加以下 2 行:
SHELL=/bin/bash
BASH_ENV=~/.bashrc_conda
Explanation:解释:
SHELL=/bin/bash
means that crontab
will run the cronjobs via bash
instead of sh
(default). SHELL=/bin/bash
意味着crontab
将通过bash
而不是sh
运行 cronjobs(默认)。 See post .见帖子。
BASH_ENV=~/.bashrc_conda
sources the conda
snippet to bash
run by chrontab
. BASH_ENV=~/.bashrc_conda
将conda
片段提供给chrontab
运行的bash
。 See post and post .请参阅post和post 。
crontab -e
insert in the cronjob line conda activate my_env;
3. 在crontab -e
中插入 cronjob 行conda activate my_env;
before the desired .py
script execution在所需的.py
脚本执行之前Example of entry for a script that would execute at noon 12:30 each day within the desired conda environment:在所需的 conda 环境中每天中午 12:30 执行的脚本的条目示例:
30 12 * * * conda activate my_env; python /path/to/script.py
Notice conda activate my_env;
注意conda activate my_env;
before the command to run the Python interpreter.在运行 Python 解释器的命令之前。
If the conda
snippet in .bashrc
gets updated by a conda
update, it will of course not be reflected in the separate .bashrc_conda
file.如果.bashrc
中的conda
片段被conda
更新更新,它当然不会反映在单独的.bashrc_conda
文件中。 One may need to check for updates from time to time.可能需要不时检查更新。
One could also to append ; conda deactivate
也可以附加; conda deactivate
; conda deactivate
at the end of that cronjob, but this may be redundant. ; conda deactivate
在该 cronjob结束时停用,但这可能是多余的。
Don't call sh
but bash
.不要叫sh
而叫bash
。 source
is a bash command. source
是一个 bash 命令。
- sh scripts/my_script.bash
+ bash scripts/my_script.bash
Or just要不就
chmod +x scripts/my_script.bash
./scripts/my_script.bash
since you added the bash shebang.因为你添加了 bash shebang。
In my case, I got this error when I ran this line of shell script: source activate my_env
就我而言,当我运行这行 shell 脚本时出现此错误: source activate my_env
activate: No such file or directory
So I changed source activate my_env
to source /path/to/conda/bin/activate my_env
.所以我将source activate my_env
更改为source /path/to/conda/bin/activate my_env
。 Then it starts working.然后它开始工作。
As of May 2022, I just use a .bat
in Windows 10 to activate myenv
and then start my localhost
or whatever script you need:截至 2022 年 5 月,我只需在 Windows 10 中使用.bat
来激活myenv
,然后启动我的localhost
或您需要的任何脚本:
@echo off
set CONDAPATH=C:\Users\MyName\anaconda3
set ENVNAME=myenv
if %ENVNAME%==base (set ENVPATH=%CONDAPATH%) else (set ENVPATH=%CONDAPATH%\envs\%ENVNAME%)
call %CONDAPATH%\Scripts\activate.bat %ENVPATH%
call cd /d d:/mysite
python manage.py runserver 0.0.0.0:8000
A simple solution that worked for me was to specify a full path to conda in the crontab entry, and to use the conda run -n <env>
option to execute the command in the required environment.一个对我有用的简单解决方案是在 crontab 条目中指定 conda 的完整路径,并使用conda run -n <env>
选项在所需环境中执行命令。 In my case I wanted to launch a command on start up, so my crontab entry looked like this:就我而言,我想在启动时启动一个命令,所以我的 crontab 条目如下所示:
@reboot ~/miniconda3/bin/conda run -n <env> <command>
where <env>
and <command>
are substituted according to your own requirements.其中<env>
和<command>
根据您自己的要求替换。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.