Pandas dataframe, empty or with 3 column to pickle

Question

I'm not used to pandas at all, thus the several question on my problem.

I have a function computing computing a list called solutions . This list can either be made of tuples of 3 values (a, b, c) or empty.

solutions = [(a,b,c), (d,e,f), (g,h,i)]

To save it, I first turn it into a numpy array, and then I save it with pandas after naming the columns.

solutions = np.asarray(solutions)
df = pd.DataFrame(solutions)
df.columns = ["Name1", "Name2", "Name3"]
df.to_pickle(path)

My issue is that I sometimes have a empty solutions list: solutions = [] . Thus, the line df.columns raises an error. To bypass it, I currently check the size of solutions, and if it is empty, I do:

pickle.dump([], path, "wb")

I would like to be a more consistent between my data type, and to save the SAME format between both scenario. => If the list is empty, I would like to save the 3 columns name with an empty data frame. Ultimate goal, is to reopen the file with pd.read_pickle() and to access easily the data in it.

Second issue, I would like to reopen the files pickled, and to add a column. Could you show me the right way to do so?

And third question, how can I select a part of the dataframe. For instance, I want all lines in which the column Name1 value % 0.25 == 0 .

Thanks

Answer 1

Create your dataframe using:

df = pandas.DataFrame(data=solutions, columns=['name1', 'name2', 'name3'])

If solutions is empty, it will nevertheless create a dataframe with 3 columns and 0 row.

In [2]: pd.DataFrame(data=[(1,2,3), (4,5,6)], columns=['a','b','c'])
Out[2]: 
   a  b  c
0  1  2  3
1  4  5  6
In [3]: pd.DataFrame(data=[], columns=['a','b','c'])
Out[3]: 
Empty DataFrame
Columns: [a, b, c]
Index: []

Answer 2

For your third question:

df["Name1"] % 0.25 == 0

computes a series of booleans which are true where the value in the first column can be divided by 0.25. You can use it to select the rows of your dataframe:

df[ df["Name1"] % 0.25 == 0 ]