Python Performance concern - Multiple methods reading same Pandas Data Frame from a particular Pickle file

Question

We have multiple (4) methods reading the same Pandas Data Frame from a particular / the same - Pickle file stored on the local directory. Code creating the only pickle file as below :-

df_for_bokeh = pd.read_sql(sql_command,engine)
df_for_bokeh.to_pickle("./df_holoviewPlots.pkl")

Code snippets from the methods reading the pickle file as below :-

df_for_bokeh = pd.read_pickle("./df_holoviewPlots.pkl")
df_for_bokeh1 = pd.read_pickle("./df_holoviewPlots.pkl")
df_for_bokeh2 = pd.read_pickle("./df_holoviewPlots.pkl")
df_for_bokeh3 = pd.read_pickle("./df_holoviewPlots.pkl")

As seen above My concern with performance is - would this be better or should we pickle the DF into 4 separate in place of the same Pickle file.

We can not have the pickle file - unpickled to give only 1 DataFrame . We shall need atleast - 4 different data frames to be read from the one Pickle file , and that too almost at the same time.

Answer 1

Is there a reason you can't do the following:

1. Set a loading lock state
2. Load the date once
3. Deep copy to as many objects as you need https://docs.python.org/3.7/library/copy.html
4. Release the loading lock state

This would give you the same data in 4 different independent data frames.