熊猫：如何分组/枢轴保留 NaN？ 将 float 转换为 str 然后再转换回 float 有效但似乎令人费解

Question

I am tracking in which month a certain event has taken place. If it hasn't, the "month" field is a NaN. The starting table looks like this:

+-------+----------+---------+
| Month | Category | Balance |
+-------+----------+---------+
| 1     | a        |     100 |
| nan   | a        |     300 |
| 2     | a        |     200 |
+-------+----------+---------+

I am trying to build a crosstab like this:

+-------+----------------------------------+
| Month | Category a - cumulative % amount |
+-------+----------------------------------+
|     1 |                             0.16 |
|     2 |                             0.50 |
+-------+----------------------------------+

In month 1, the event has happened for 100/600, ie for 16% In month 2, the event has happened, cumulatively, for (100 + 200) / 600 = 50%, where 100 is in month 1 and 200 in month 2.

My issue is with NaNs. Pandas automatically removes NaNs from any groupby / pivot / crosstab. I could convert the month field to string, so that grouping it won't remove the NaNs, but then pandas sorts by the month as if it were a string, ie it would sort: 10, 48, 5, 6.

Any suggestions?

The bit below works but seems extremely convoluted:

• I convert 'month' to string
• Do a crosstab
• Convert month back to float (can I do it without moving the index to a column and then the column back to the index?)
• Sort again
• Do the cumsum

Code:

import numpy as np
import pandas as pd

df = pd.DataFrame()
mylen = int(10e3)
df['ix'] = np.arange(0,mylen)
df['amount'] = np.random.uniform(10e3,20e3,mylen)
df['category'] = np.where( df['ix'] <=4000, 'a','b' )
df['month'] = np.random.uniform(3,48,mylen)
df['month'] = np.where( df['ix'] <=1000, np.nan, df['month'] )
df['month rounded'] = np.ceil(df['month'])

ct = pd.crosstab(df['month rounded'].astype(str) , df['category'], \
                 values = df['amount'] ,aggfunc = 'sum', margins = True ,\
                     normalize = 'columns', dropna = False)

# the index is 'month rounded'
ct = ct.reset_index()
ct['month rounded'] = ct['month rounded'].astype('float32')
ct = ct.sort_values('month rounded')
ct = ct.set_index('month rounded')
ct2 = ct.cumsum (axis = 0)

Answer 1

Use:

new_df = df.assign(cumulative=df['Balance'].mask(df['Month'].isna())
                                           .groupby(df['Category'])
                                           .cumsum()
                                           .div(df.groupby('Category')['Balance']
                                                  .transform('sum'))).dropna()
print(new_df)
   Month Category  Balance  cumulative
0    1.0        a      100    0.166667
2    2.0        a      200    0.500000

---

If you want create a DataFrame for each Category you could create a dict:

df_category = {i:group for i,group in new_df.groupby('Category')}

Answer 2

df['Category a - cumulative % amount'] = (
    df.groupby(by=df.Month.fillna(np.inf))
    .apply(lambda x: x.Balance.cumsum().div(df.Balance.sum()))
    .reset_index(level=0, drop=True)
)

df.dropna()

    Month   Category    Balance Category a - cumulative % amount
0   1       a           100     0.166667
2   2       a           200     0.333333