在numpy中按变量分组的行中取平均值

Question

I have an numpy array like below.

array([[ 0.23810484,  0.00020161,  0.41350806,  0.2421371 ,  0.02237903,
         0.08084677,  0.00020161,  0.00221774,  0.00020161,  0.00020161],
       [ 0.04279661,  0.05974576,  0.02584746,  0.00042373,  0.00042373,
         0.00042373,  0.00042373,  0.73771186,  0.00889831,  0.12330508]])

It is 5000X10.

I also have a Pandas Series object which is again 5000 length. Its values are like this>

5061             Terminated
17410    Completed Negative

There are total three distinct cateogries. Each series value is a category for the corresponding row in the first numpy array.

What I want to get is to take an average of each variable in first array grouped by the categories in Series. So in the end I would have a numpy array with 3 rows for each category of series and ten columns whose value will be average across all 5000 rows.

Please advise

Answer 1

You can add each column from the numpy array to a separate column in the pandas DataFrame, and then use DataFrame.groupby() to group based on your required column and then take mean() . Example (Assuming your series is called series , and numpy array is called narray ) -

df = pd.DataFrame(series)
for i in range(10):
    df[i] = narray[:,i]

df.groupby('required_column').mean()

Demo -

In [77]: df = pd.DataFrame([[5061,'Terminated'],[17410,'Completed Negative']],columns=['index','groupcol']).set_index('index')

In [78]: df
Out[78]:
                 groupcol
index
5061           Terminated
17410  Completed Negative

In [79]: x
Out[79]:
array([[  2.38104840e-01,   2.01610000e-04,   4.13508060e-01,
          2.42137100e-01,   2.23790300e-02,   8.08467700e-02,
          2.01610000e-04,   2.21774000e-03,   2.01610000e-04,
          2.01610000e-04],
       [  4.27966100e-02,   5.97457600e-02,   2.58474600e-02,
          4.23730000e-04,   4.23730000e-04,   4.23730000e-04,
          4.23730000e-04,   7.37711860e-01,   8.89831000e-03,
          1.23305080e-01]])

In [80]: for i in range(10):
   ....:     df[i] = x[:,i]
   ....:

In [81]: df
Out[81]:
                 groupcol         0         1         2         3         4  \
index
5061           Terminated  0.238105  0.000202  0.413508  0.242137  0.022379
17410  Completed Negative  0.042797  0.059746  0.025847  0.000424  0.000424

              5         6         7         8         9
index
5061   0.080847  0.000202  0.002218  0.000202  0.000202
17410  0.000424  0.000424  0.737712  0.008898  0.123305

In [82]: df.groupby('groupcol').mean()
Out[82]:
                           0         1         2         3         4  \
groupcol
Completed Negative  0.042797  0.059746  0.025847  0.000424  0.000424
Terminated          0.238105  0.000202  0.413508  0.242137  0.022379

                           5         6         7         8         9
groupcol
Completed Negative  0.000424  0.000424  0.737712  0.008898  0.123305
Terminated          0.080847  0.000202  0.002218  0.000202  0.000202

---

If you want the result as a list , you can do -

df.groupby('required_column').mean().values.tolist()

Demo -

In [83]: df.groupby('groupcol').mean().values.tolist()
Out[83]:
[[0.04279661,
  0.05974576,
  0.02584746,
  0.00042373,
  0.00042373,
  0.00042373,
  0.00042373,
  0.73771186,
  0.00889831,
  0.12330508],
 [0.23810484,
  0.00020161,
  0.41350806,
  0.2421371,
  0.02237903,
  0.08084677,
  0.00020161,
  0.00221774,
  0.00020161,
  0.00020161]]