How to execute a script in a virtual environment from a bash script on server?

Question

I am trying to run a python script in a virtual environment in a server using oarsub :

So firstly I run this command in a server name "a" :

oarsub -l /host=1/gpu=1,walltime=2:00:00 './training_corpus1.sh'

training_corpus1.sh looks like this at the beguinning :

#!/bin/bash

cd /home/ge/ke/anaconda3/envs

source activate env

cd ~/eXP/bert

CUDA_VISIBLE_DEVICES=0,1 python training.py \
    --exp_name bert \ ...

At the beguinning, I am suppose to open my virtual environment and then run the script but I am always getting this error in the OAR.18651.stderr file :

./training_corpus1.sh: line 5: activate: No such file or directory
Traceback (most recent call last):
  File "training.py", line 18, in <module>
    from xlm.slurm import init_signal_handler, init_distributed_mode
  File "/home/ge/ke/eXP/bert/xlm/slurm.py", line 11, in <module>
    import torch
ImportError: No module named torch

Torch is located in my virtual environment , it seems that It did not open.Whenusing "conda " a thte place of "source" I get:

./training_corpus1.sh: line 5: conda: command not found

Answer 1

For a virtual environment and bash the activation command is

source env/bin/activate

where env is the directory of the virtual environment to activate.

PS. Let me advice you to start any script with set -e to allow fast failing on any error:

#!/usr/bin/env bash
set -e
…