Excel/谷歌表格 pivot function VS python pivot function

Question

I am trying to pivot a dataframe in python. I used the pivot_table function in python. The trouble is that when I tried to verify the result with the pivot table created in Google Sheets, both the results were different . Also the number of rows after pivoting is different in both the cases . I am confused now as to what is going on here? Why are the results of a same function different? Aren't they supposed to do the same thing??!

Here is the link to the data and the pivot table done in Google Sheets. The Order item ID 2 should be the index as it is the unique ID and the Settlement Value should be the values where it should be aggregated.

Below is the code I use to create pivot table in python with the same data:

payout = pd.read_excel('combined aug-dec payout.xlsx')
payout['Order item ID'] = payout['Order item ID'].apply(str)
payout['Order item ID 2'] = 'OI:' + payout['Order item ID']
pivot_settle = payout.pivot_table(index = ['Order item ID 2'], values = ['Settlement Value'], aggfunc = 'sum')

Any help is appreciated!

Answer 1

The problem comes from the fact that Order item ID 2 contains Google Sheet formulas, not raw data, that are not parsed by Pandas as intended: OI:121314525072009 01 becomes OI:121314525072009 00 when the Excel file is imported. And thus, when you aggregate on this column, you end up whith less rows than in Google Sheet pivot table.

So, first, copy/paste values to overwrite formulas in this column, save the file, and then, pivoting your data with Pandas:

import pandas as pd

df = pd.read_excel("combined aug-dec payout.xlsx")

print(
    pd.pivot_table(
        df,
        index="Order item ID 2",
        values="Settlement Value",
        aggfunc=sum,
    ).shape[0]
)
# Ouput: 17,754 rows

And with Google Sheets: 17,754 rows .