jupyter notebook - run notebook (and start running cells) from command line and access it from browser

Question

I want to execute jupyter notebook - but with the ability to connect into the browser in order to monitor the progress ( all from command line ).

Why? I have multiple long trainings that I want to load as soon as my machine in AWS loads - and I don't want to go each time to the browser and press "start and run all cells".

I saw that I can convert into .py file (But then I'll have no ability to monitor the training progress) - So that's not good. I need the inter-activeness of the browser after I run it from the command line.

Best, Harel

Answer 1

I am actually looking for the same answer. Currently I am using a workaround which adjusts the custom.js file in the .jupyter folder. I thought I might share my solution so other users might be able to improve it.

The answer is based on the observation that all cells are recalculated if you execute the javascript "Jupyter.notebook.execute_all_cells()" in the console of your browser hosting the .ipynb (with inspect-element)

Unfortunately, I've not been able to do this directly. Selenium might provide some users with an answer, but due to company policy I can't install the selenium driver for Edge (which I have to use). Hence, I designed this workaround:

#this js code enforces all cells to be executed once the notebook is loaded.
js='''
if (Jupyter.notebook.kernel) {
   Jupyter.notebook.execute_all_cells();
} else {
  Jupyter.notebook.events.one('kernel_ready.Kernel', (e) => {
    Jupyter.notebook.execute_all_cells();
  });
}
'''
#now temprorarily append this piece of code to your custom.js file
custom_js_path = os.path.join(jupyter_dir, 'custom', 'custom.js')

if os.path.isfile(custom_js_path):
    f = open(custom_js_path,"r+")
    original = f.read()
    f.write(js)
    f.close()
else:
    f = open(custom_js_path,"x")
    f.write(js)
    f.close()

# the command to start Jupyter notebooks and the notebook itself, which needs to be executed in a seperate thread since it is blocking calculations
def f():
    subprocess.check_call(["jupyter", "notebook", "C:\\Users\\Untitled.ipynb"], shell=True)
    return 
pool = ThreadPool(processes=1)
async_result = pool.apply_async(f)  

#... Wait till notebook is started
time.sleep(25)

# now revert the custom.js to its original state
with open(custom_js_path, "w") as f:
    f.write(original)
    f.close()