计算 Pandas DataFrame 中一组列的平均值的最有效方法

Question

I have a DataFrame with columns like this:

["A_1", "A_2", "A_3", "B_1", "B_2", "B_3"]

I'd like to "collapse" the various A and B columns in a single column each and calculate their mean value. In short, at the end of the operation I'd get:

["A", "B"]

where "A" is the column-wise mean of all "A" columns and "B" the mean of all "B" columns.

As far as I understood, groupby is not suited for this task, or perhaps I'm using it incorrectly:

grouped = data.groupby([item for item in data if "A" not in item])

If I use axis=1 , all I get is an empty DataFrame when calling mean(), and if not I'm not getting the desired effect. I would like to avoid building a separate DataFrame to be fillled with the means via iteration (eg by calculating means separately then adding them like new_df["A"] = mean_a ). Is there an efficient solution for this?

Answer 1

You want to make use of the built-in mean() function that accepts an axis argument to specify row-wise means. Since you know your specific column name convention for the different means that you want, you can use the example code below to do it very efficiently. Here I chose to just make two additional columns rather than to actually destroy the existing data. I could have also put these new columns into a new data frame; it just depends on what your needs are and what's convenient for you. The same basic idea will work in either case.

In [1]: import pandas

In [2]: dfrm = pandas.DataFrame([[1,2,3,4,5,6],[7,8,9,10,11,12],[13,14,15,16,17,18]], columns = ['A_1', 'A_2', 'A_3', 'B_1', 'B_2', 'B_3'])

In [3]: dfrm
Out[3]: 
   A_1  A_2  A_3  B_1  B_2  B_3
0    1    2    3    4    5    6
1    7    8    9   10   11   12
2   13   14   15   16   17   18

In [4]: dfrm["A_mean"] = dfrm[[elem for elem in dfrm.columns if elem[0]=='A']].mean(axis=1)

In [5]: dfrm
Out[5]: 
   A_1  A_2  A_3  B_1  B_2  B_3  A_mean
0    1    2    3    4    5    6       2
1    7    8    9   10   11   12       8
2   13   14   15   16   17   18      14

In [6]: dfrm["B_mean"] = dfrm[[elem for elem in dfrm.columns if elem[0]=='B']].mean(axis=1)

In [7]: dfrm
Out[7]: 
   A_1  A_2  A_3  B_1  B_2  B_3  A_mean  B_mean
0    1    2    3    4    5    6       2       5
1    7    8    9   10   11   12       8      11
2   13   14   15   16   17   18      14      17

Answer 2

I don't know about efficient, but I might do something like this:

~/coding$ cat colgroup.dat
A_1,A_2,A_3,B_1,B_2,B_3
1,2,3,4,5,6
7,8,9,10,11,12
13,14,15,16,17,18
~/coding$ python
Python 2.7.3 (default, Apr 20 2012, 22:44:07) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
>>> df = pandas.read_csv("colgroup.dat")
>>> df
   A_1  A_2  A_3  B_1  B_2  B_3
0    1    2    3    4    5    6
1    7    8    9   10   11   12
2   13   14   15   16   17   18
>>> grouped = df.groupby(lambda x: x[0], axis=1)
>>> for i, group in grouped:
...     print i, group
... 
A    A_1  A_2  A_3
0    1    2    3
1    7    8    9
2   13   14   15
B    B_1  B_2  B_3
0    4    5    6
1   10   11   12
2   16   17   18
>>> grouped.mean()
key_0   A   B
0       2   5
1       8  11
2      14  17

I suppose lambda x: x.split('_')[0] would be a little more robust.