通过命令行运行 jupyter notebook 并将输出记录到文件中

Question

I am running a notebook on command line in AWS SageMaker but I am not able to log outputs to a file. I added print , logging.info and sys.stdout.write statements in hope of capturing something but to no avail.

Below is the code I am using

NOTEBOOK_FILE="/home/ec2-user/SageMaker/mynb.ipynb"
LOG_FILE="/home/ec2-user/SageMaker/logs.txt"

nohup jupyter nbconvert  --to notebook --inplace --ExecutePreprocessor.timeout=-1 --ExecutePreprocessor.kernel_name=python3 --execute "$NOTEBOOK_FILE" > "$LOG_FILE" &

On doing cat "$LOG_FILE" I get just a couple lines of logs and nothing after that. Output is below -

[NbConvertApp] WARNING | Config option `kernel_spec_manager_class` not recognized by `NbConvertApp`.
[NbConvertApp] Converting notebook /home/ec2-user/SageMaker/tata1mg.ipynb to notebook
[NbConvertApp] Executing notebook with kernel: python3

PS:

Answer 1

You can convert the notebook to a script: jupyter nbconvert --to script "$NOTEBOOK_FILE"

and then run the Python Script: python3 mynb.py > $LOG_FILE