How to edit and debug a library in python?

Question

I have created a my own library(package) and installed as development using pip install -e Now, I would like to edit this library(.py) files and see the update in jupyter notebook. Every time, I edit a library(.py) files I am closing and reopening ipython notebook to see the update. Is there any easy way to edit and debug .py package files ?

Answer 1

Put this as first cell of your notebooks:

%load_ext autoreload
%autoreload 2

More info in the doc .

Answer 2

When you load jupyter, you are initializing a python kernel. This will lock your python to the environment it was at when you loaded the kernel.

In this case, your kernel contains your local egg installed package at the point where it was when you loaded jupyter. Unfortunately, you will need to reload jupyter every time you update your local package.

@BlackBear has a great solution of using autoreload in your first cell:

%load_ext autoreload
%autoreload 2

A follow up solution assumes you do not need to make changes to your notebooks, but just want the updated outputs given changes to your package. One way I have gotten around this is to use automated notebook generation processes using jupyter nbconvert and shell scripting. You essentially create some jupyter templates stored in a templates folder that you will auto execute every time you update your package.

An example script:

rm ./templates/*.nbconvert.ipynb
rm ./*.nbconvert.ipynb

for file in "templates"/*.ipynb
do
  echo $file
  jupyter nbconvert --to notebook --execute $file
done

mv ./templates/*.nbconvert.ipynb .

Assuming you want to actively debug your package, I would recommend writing test scripts that load a fresh kernel every time. EG:

#mytest.py
from mypackage import myfunction

expected_outputs={'some':'expected','outputs':'here'}
if myfunction(inputs)==expected_outputs:
    print('Success')
else:
    print('Fail')

python3 mytest.py