How to redirect stdout stderr of os.execlpe() into a file

Question

I am hitting my head against the wall, running a jpyter nbconvert from python the following way, as it allows me to pass arguments to the jupyter notebook:

env['IPYTHONARGV'] = json.dumps({'timeperiod':timeperiod,'infile':infile})
os.execlpe('jupyter', 'jupyter', 'nbconvert', '--execute','notebook.ipynb', 
           '--to', 'html', '--output', output_html, '2>&1', '1>log.out',  env)

When leaving out the '2>&1', '1>log.out', part, the command works just fine. But with the bash redirect, the command complains the following:

[NbConvertApp] WARNING | pattern '2>&1' matched no files
[NbConvertApp] WARNING | pattern '1>log.out' matched no files

Does anybody know how to solve that problem?

Answer 1

The redirections 2>&1 ad 1>log.out are interpreted by the shell, but you are providing them to the command as arguments. That's why Jupyter complains about not being able to find them as files.

You can use subprocess with shell=True :

import subprocess as sp
env['IPYTHONARGV'] = json.dumps({'timeperiod':timeperiod,'infile':infile})
sp.check_call('jupyter nbconvert --execute notebook.ipynb --to html --output output_html 2>&1 1>log.out', shell=True, env=env)

You can use sp.check_output() and remove the redirect if you need to process the output in Python.