重采样多级索引，或沿矩阵/数组的第三维求平均值

Question

I have gridded satellite data stored in a dataframe. Normally, this dataframe gets sliced to make imshow plots on a day-by-day basis, which is trivial. However, I would like to plots annual means of the data, which is where I am currently stuck. The dataframe has a multi-level index (datetime, latitude coordinate) with columns making up the longitude coordinates.

import pandas as pd, numpy as np

dates  = pd.date_range('20140101',periods=10,freq='1D')
others = np.arange(0,5)
index  = [(d,o) for o in others for d in dates]
index  = pd.MultiIndex.from_tuples(index, names=['DATES','LAT'])
data   = np.random.randint(0,20,(50,10))

df = pd.DataFrame(data=data,index=index,columns=np.arange(0,10))
df.columns.names = ['LON']

If I were using arrays I would normally stack them along the third dimension and then average on the third dimension. eg

mat = np.ones( (5,10,1) )

# stack on day-by-day basis so lat/lon pairs sit on top of each other 
# on the third dimension
for heute in df.index.get_level_values(0).unique():
    tmp = df.xs(heute, level=0)

    mat = np.dstack( (mat,tmp.as_matrix()) )

ave = mat[:,:,1:].mean(axis=2)

While this would work, I suspect there is a method of doing this within Pandas. However, for this I do not know where to start. I have played around with groupby and resample, but I have been unable to make those work. Any help would be appreciated.

Answer 1

Here we go:

import pandas as pd, numpy as np
pd.set_option('display.float_format',lambda x: '{:,.1f}'.format(x))
np.random.seed(1)

dates  = pd.date_range('20140101',periods=10,freq='1D')
others = np.arange(0,5)
index  = [(d,o) for o in others for d in dates]
index  = pd.MultiIndex.from_tuples(index, names=['DATES','LAT'])
data   = np.random.randint(0,20,(50,10))

df = pd.DataFrame(data=data,index=index,columns=np.arange(0,10))
df.columns.names = ['LON']

# answer 
df = df.stack()
df= df.groupby(level=['LAT','LON']).mean()
print df.unstack(level=['LON'])

which yields:

LON    0    1    2    3    4    5    6    7    8    9
LAT                                                  
0    8.8  8.5 10.8  9.2  9.0 10.8  9.3  9.3  7.6  9.1
1   10.6  8.5 10.6 12.2  8.0  8.8  9.5 11.3 10.8  9.5
2   11.0 10.3  8.2 11.2  9.9  8.4 13.5  9.7  7.8  9.0
3    8.1  6.2  8.8 12.6 10.6  7.1  8.8  9.3 11.7 10.2
4    9.1 10.1  7.8  8.7  7.4  7.3 10.2 11.9  8.3 11.9

Whilst your array approach yields:

[[  8.8   8.5  10.8   9.2   9.   10.8   9.3   9.3   7.6   9.1]
 [ 10.6   8.5  10.6  12.2   8.    8.8   9.5  11.3  10.8   9.5]
 [ 11.   10.3   8.2  11.2   9.9   8.4  13.5   9.7   7.8   9. ]
 [  8.1   6.2   8.8  12.6  10.6   7.1   8.8   9.3  11.7  10.2]
 [  9.1  10.1   7.8   8.7   7.4   7.3  10.2  11.9   8.3  11.9]]