Importing a Dataframe from one Jupyter Notebook into another Jupyter Notebook

Question

I wrote a python script to get data from my Gmail account which I imported as a pandas dataframe into a Jupyter notebook. The notebook is called "Automation via Gmail API" and the dataframe is simply called "df". Now I want to use this df to update a Google Sheet via the Google Sheets API. To this end I created another notebook - "Automation via Sheets API". But how can I access df in the "Automation via Sheets API" notebook? Apparently, Jupyter provides some functionality to load a notebook into another notebook . I simply copy and pasted the code of the "Notebook Loader" into my Sheets-notebook and only changed "path" and "fullname", but it doesn't work and I don't have a clue why:

#Load df from the "Automation via Gmail API" notebook.

fullname = "Automation via Gmail API.ipynb"

class NotebookLoader(object):
    """Module Loader for Jupyter Notebooks"""
    def __init__(self, path="C:\\Users\\Moritz Wolff\\Desktop\\gmail automatisierung\\Gmail API"):
        self.shell = InteractiveShell.instance()
        self.path = path

    def load_module(self, fullname="Automation via Gmail API.ipynb"):
        """import a notebook as a module"""
        path = find_notebook(fullname, self.path)

[...]

There is no error-message. Is my strategy flawed from the start or do I simply miss a little detail? Any help is appreciated.

Answer 1

A direct option is to save the dataframe as a text table in the original notebook and read it into the other. Instead of plain text you can also save the dataframe itself as serialized Python for a little more efficiency/convenience.

Options from source notebook:

df.to_csv('example.tsv', sep='\t') # add `, index = False` to leave off index
# -OR-
df.to_pickle("file_name.pkl")

Options in reading notebook:

import pandas as pd
df = pd.read_csv('example.tsv', sep='\t')
#-OR-
df = pd.read_pickle("file_name.pkl")

I used tab delimited tabular text structure, but you are welcome to use comma-separated.

Answer 2

I would avoid loading your notebook from another notebook unless you are sure that is how you want to approach your problem.

You can always export your dataframe to a csv using pandas.DataFrame.to_csv() , then load it in your other notebook with pandas.read_csv()

import pandas as pd

df = ['test','data']
df.to_csv('data1.csv')

Then in your other notebook:

df = pd.read_csv('data1.csv', index_col = 0)

Alternatively you can try using the %store magic function:

df = ['test','data']

%store df

Then to recall it in another notebook to retrieve it:

%store -r df

One constraint about this method is that you have to %store your data each time the variable is updated.

Documentation: https://ipython.readthedocs.io/en/stable/config/extensions/storemagic.html