是否可以在 HTCondor 中检索 python 脚本的输出？

Question

I would like to run some python scripts that return some dictionaries on a cluster managed by HTCondor. I run the same on another cluster running MPI/Slurm and mpi4py and I was able to collect the output without saving files and transfer them to the main node. I cannot understand how to do it in HTCondor (even using python wrappers).

Is it possible to retrieve the output of python scripts in HTCondor? Do i need to use the schedd.retrieve("ClusterId == %d" % cluster) command? if so an example will be really appreciated.

Thanks!

Answer 1

You may specify a file to which you want to save your stdout in your submit file (see also Condor output file updating ).

You can watch the output using htcondor_tail (not sure what the equivalent is in the python api).

Slightly off-topic: If you run python you can also use eg Dask to do your remote python stuff (see also Dask with HTCondor scheduler )