向量化对一个 Pandas dataframe 的不同列的聚合操作

Question

I have a Pandas dataframe, mostly containing boolean columns. A small example is:

import pandas as pd

df = pd.DataFrame({"A": [1, 2, 3, 1, 2, 3],
                   "B": ['a', 'b', 'c', 'a', 'b', 'c'],
                   "f1": [True, True, True, True, True, False],
                   "f2": [True, True, True, True, False, True],
                   "f3": [True, True, True, False, True, True],
                   "f4": [True, True, False, True, True, True],
                   "f5": [True, False, True, True, True, True],
                   "target1": [True, False, True, True, False, True],
                   "target2": [False, True, True, False, True, False]})

df

Outout:

    A   B   f1      f2      f3      f4      f5    target1  target2
0   1   a   True    True    True    True    True    True    False
1   2   b   True    True    True    True    False   False   True
2   3   c   True    True    True    False   True    True    True
3   1   a   True    True    False   True    True    True    False
4   2   b   True    False   True    True    True    False   True
5   3   c   False   True    True    True    True    True    False

for each True and False class of each f columns and for all groups in ("A", "B") columns, I want to do a sum over target1 and target2 columns. Using a loop over f columns, we have:

for col in ["f1", "f2", "f3", "f4", "f5"]:
    print(col, "\n", 
          df[df[col]].groupby(["A", "B"]).agg({"target1": "sum", "target2": "sum"}), "\n",
          df[~df[col]].groupby(["A", "B"]).agg({"target1": "sum", "target2": "sum"}))

Now, I need to do it without the for loop; I mean a vecotization over f columns to reduce the computation time (computation time should be almost equal to time needed for doing it for one f column).

Answer 1

Use DataFrame.melt , so possible aggreagte by columns names f and value for True/False s:

df = df.melt(['A','B','target1','target2'])

df1 = df.groupby(["A", "B","variable","value"]).agg({"target1": "sum", "target2": "sum"})
print (df1)
                    target1  target2
A B variable value                  
1 a f1       True         2        0
    f2       True         2        0
    f3       False        1        0
             True         1        0
    f4       True         2        0
    f5       True         2        0
2 b f1       True         0        2
    f2       False        0        1
             True         0        1
    f3       True         0        2
    f4       True         0        2
    f5       False        0        1
             True         0        1
3 c f1       False        1        0
             True         1        1
    f2       True         2        1
    f3       True         2        1
    f4       False        1        1
             True         1        0
    f5       True         2        1

Then selecting is possible by:

print (df1.query("variable=='f1' and value==True").droplevel([-1,-2]))
     target1  target2
A B                  
1 a        2        0
2 b        0        2
3 c        1        1

Or:

idx = pd.IndexSlice
print (df1.loc[idx[:, :, 'f1', True],:].droplevel([-1,-2]))
     target1  target2
A B                  
1 a        2        0
2 b        0        2
3 c        1        1