按组减少熊猫数据框

Question

I've been searching extensively but can't get my head around this issue:

I have a dataframe in pandas that looks like this:

date    ticker Name NoShares SharePrice Volume Relation
2/1/10  aaa    zzz  1        1          1      d 
2/1/10  aaa    yyy  1        2          5      o
2/1/10  aaa    zzz  2        5          2      d  
2/5/10  bbb    xxx  5        5          1      do
2/5/10  ccc    www  5        5          1      d
2/5/10  ccc    www  5        5          1      d
2/5/10  ddd    vvv  5        5          1      o
2/6/10  aaa    zzz  1        1          3      d

Requirements

1. I want to group by date and Name and:
2. have the number of shares summed up
3. have a weighted mean column for the share price (the weights are the NoShares)
4. average the volume and have it as a column
5. Leave relation as it is

so my output would look like this:

date    ticker Name NoShares SharePrice Volume Relation
2/1/10  aaa    zzz  3        3.6        1      d 
2/1/10  aaa    yyy  1        2          5      o
2/5/10  bbb    xxx  5        5          1      do
2/5/10  ccc    www  10       5          1      d
2/5/10  ddd    vvv  5        5          1      o
2/6/10  aaa    zzz  1        1          3      d

I tried the documentation and other answers on stack overflow but don't seem to be able to get it right. Appreciate the help. Cheers.

Answer 1

here's my solution:

grpby = df.groupby(['date','Name'])
a = grpby.apply(lambda x: np.average(a = x['SharePrice'],weights=x['NoShares'])).to_frame(name='SharePrice')
b = grpby.agg({'NoShares':'sum','Volume':'mean','Relation':'max'})
print b.join(a)

             Volume Relation  NoShares  SharePrice
date   Name                                       
2/1/10 yyy   5.0000        o         1      2.0000
       zzz   1.5000        d         3      3.6667
2/5/10 vvv   1.0000        o         5      5.0000
       www   1.0000        d        10      5.0000
       xxx   1.0000       do         5      5.0000
2/6/10 zzz   3.0000        d         1      1.0000

just reset_index() afterwards.

Answer 2

I made an assumption here. When you said group by date and Name and to keep relation - I am assuming that ticker and relation will also be unique to those groups. So for simplicity I am grouping by all 4.

df = pd.DataFrame([
                ['2/1/10', 'aaa', 'zzz', 1, 1, 1, 'd'], 
                ['2/1/10', 'aaa', 'yyy', 1, 2, 5, 'o'],
                ['2/1/10', 'aaa', 'zzz', 2, 5, 2, 'd'],  
                ['2/5/10', 'bbb', 'xxx', 5, 5, 1, 'do'],
                ['2/5/10', 'ccc', 'www', 5, 5, 1, 'd'],
                ['2/5/10', 'ccc', 'www', 5, 5, 1, 'd'],
                ['2/5/10', 'ddd', 'vvv', 5, 5, 1, 'o'],
                ['2/6/10', 'aaa', 'zzz', 1, 1, 3, 'd']],
             columns = ['date', 'ticker', 'Name', 'NoShares',
                        'SharePrice', 'Volume', 'Relation'])

def process_date(dg):
    return pd.DataFrame([[
                        dg['NoShares'].sum(),
                        (dg['NoShares'] * dg['SharePrice']).sum() / dg['NoShares'].sum(),
                        dg['Volume'].mean(),
                        ]], columns=['NoShares', 'SharePrice', 'Volume'])

df.groupby(['date', 'ticker', 'Name', 'Relation']).apply(process_date).reset_index(4, drop=True).reset_index(drop=False)

Results:

     date ticker Name Relation  NoShares  SharePrice  Volume
0  2/1/10    aaa  yyy        o         1    2.000000     5.0
1  2/1/10    aaa  zzz        d         3    3.666667     1.5
2  2/5/10    bbb  xxx       do         5    5.000000     1.0
3  2/5/10    ccc  www        d        10    5.000000     1.0
4  2/5/10    ddd  vvv        o         5    5.000000     1.0
5  2/6/10    aaa  zzz        d         1    1.000000     3.0

Answer 3

Both Dickster's and Leo's answers work well but just be aware that .groupby has dropna=True set by default. So if you have a dataset and perform groupby on multiple columns where some of those columns might contain NaN's Pandas will drop these groups. The final DataFrame will have less rows then.

The same SQL query on a SQL Server don't drop rows with NULL values in columns that are in a group by clause. I don't know if that's true for other RDBMS but just bear in mind that Pandas by default treat group by in a different way.