pandas - Subtract 2 similar pivot table of data frame

Question

I have a 36 rows x 36 columns dataframe of pivot table which I transform using code below:

df_pivoted = pd.pivot_table(df,index='From',columns='To',values='count')
df_pivoted.fillna(0,inplace=True)

I transpose the same dataframe using this code:

df_trans = df_pivoted.transpose()

and want to substract those two dataframes with this code:

new_pivoted = df_pivoted - df_trans

It gives me 72 rows x 72 columns dataframe with NaN value in all cell.

Then I try to use other code:

delta = df_pivoted.subtract(df_trans, fill_value=0)

However, it yields 72 rows x 72 columns with dataframe that looks like this:

Please help me to find the difference between the original dataframe with the transpose dataframe.

Answer 1

After transforming of you DataFrame (pivot table) you have new DataFrame where columns become Indices and vise versa. Now when you subtract on df from another Pandas use columns and Indices and fill NaN in the rest.

if you need to subtract values no matter of index and columns use:

delta = df_pivoted.values - df_trans.values

If you want to keep Columns and Index of df_trans in df_pivoted:

df_trans = pd.DataFrame(data=df_pivoted.transpose().values, 
                        index=df_pivoted.index,
                        columns = df_pivoted.columns)

delta = df_pivoted - df_trans

Now simple subtraction works.

Hope that helps!