Share data between IPython Notebooks

Question

If I have several IPython notebooks running on the same server. Is there any way to share data between them? For example, importing a variable from another notebook? Thanks!

Answer 1

This works for me :

The %store command lets you pass variables between two different notebooks.

data = 'this is the string I want to pass to different notebook' %store data

Now, in a new notebook… %store -r data print(data) this is the string I want to pass to different notebook

I've successfully tested with sklearn dataset :

from sklearn import datasets

dataset = datasets.load_iris()

%store dataset

in notebook to read data :

%store -r dataset

src : https://www.dataquest.io/blog/jupyter-notebook-tips-tricks-shortcuts/

Answer 2

Notebooks in Jupyter Lab can share the same kernel. In your notebook you can choose the kernel of another notebook and variables from the other notebook will be available in both notebooks.

1. Click on the button that describes your current kernel.
2. Choose the kernel of the other notebook, whose variables you want to access.

Answer 3

IPython supports the %store magic ( here is the documentation ). It seems to have the same constraints of pickle : if the file can be pickled it will also be storable.

Anyway, it will work for sure with common Python types. Here's a basic example:

var_1 = [1,2,3,4] #list
var_2 = {'a':1,'b':2,'c':3} #dict
var_3 = (6,7,8) #tuple
var_4 = {'d','e','f'} #set
%store var_1
%store var_2
%store var_3
%store var_4

 Stored 'var_1' (list)
 Stored 'var_2' (dict)
 Stored 'var_3' (tuple)
 Stored 'var_4' (set)

Then on a different IPython notebook it will be sufficient to type:

%store -r var_1 
%store -r var_2 
%store -r var_3 
%store -r var_4

Answer 4

If your data is in a single variable then have a try at saving it to a file using the %save magic in one notebook and then reading it back in another.

The one difficulty is that the text file will contain the data but no variable definition so I usually contatenate it with a variable definition and then exec the result.

Answer 5

I believe that theoretically you should be able to do so with messaging , though I would have to dig a lot deeper to figure it out.

Why would you need this capability though?

Answer 6

Thanks a lot for these answers. It is really needed in various scenarios. In my case I have a dataframe of around 5 lakhs rows. I am applying a Bert vector operations on all the columns which contains text. As the process starts you won't be able to look what actual values have been put inside the dataframe. Hence using this %save magic function we can keep on looking at the progress from another notebook. Thanks a lot guys. Thanks @Kyle for asking the question