Python pandas groupby与按列过滤的其他行的区别

Question

I am struggling with Python Pandas with groupby. How should I accomplish the following? For every fruit I would like to find the difference with the 'step 0' value of that fruit.

df = pd.DataFrame({'Fruit' : ['Apple', 'Apple', 'Apple', 'Banana', 'Banana', 'Banana'], 'Step' : [0, 1, 2, 0, 1, 2], 'Value' : [100, 102, 105, 200, 210, 195] })

    Fruit  Step  Value     to-be
0   Apple     0    100  -->  0
1   Apple     1    102  -->  2
2   Apple     2    105  -->  5
3  Banana     0    200  -->  0
4  Banana     1    210  --> 10
5  Banana     2    195  --> -5

Thank you!

Answer 1

This should do it:

df.groupby('Fruit').apply(lambda g: g.Value - g[g.Step == 0].Value.values[0])

First, we're grouping by the column you care about (Fruit). Then we're applying a function to each group (using a lambda which lets us specify a function in-line). For each group, we find the row(s) where g.Step == 0 , then get the Value entry from that row, and use values[0] to get the first Value (in case there were multiple places where g.Step == 0 ). Then we just subtract that one value from all the rows in the group, and return it.

If you want to add it as a column to the dataframe, you can drop the index:

res = df.groupby('Fruit').apply(lambda g: g.Value - g[g.Step == 0].Value.values[0])
df['Result'] = res.reset_index(drop=True)

Answer 2

Think this does the trick. It simply loops through the rows and applies a new 'first' value each time the step is equal to 0. Then calculates the difference from that first value.

rows = range(df.shape[0])
df['count'] = 0
for r in rows:
    step = df.iloc[r,1]
    value = df.iloc[r,2]
    if step == 0:
        first = value
    df.iloc[r,3] = value - first

Answer 3

I am a newbie to pandas, but at least the following code works. The end of result,

    Fruit  Step  Value  to-be
0   Apple     0    100      0
1   Apple     1    102      2
2   Apple     2    105      5
3  Banana     0    200      0
4  Banana     1    210     10
5  Banana     2    195     -5

[6 rows x 4 columns]

---

The source code is as follows.

import pandas as pd

df = pd.DataFrame({'Fruit' : ['Apple', 'Apple', 'Apple', 'Banana', 'Banana', 'Banana'], 
                    'Step' : [0, 1, 2, 0, 1, 2], 
                    'Value' : [100, 102, 105, 200, 210, 195] })

list_groups = list()

# loop over dataframe groupby `Fruit`
for name, group in df.groupby('Fruit'):
    group.sort('Step', ascending=True) # sorted by `Step`

    row_iterator = group.iterrows()

    # get the base value
    idx, first_row = row_iterator.next()
    base_value = first_row['Value']

    to_be = [0] # store the values of the column `to-be`
    for idx, row in row_iterator:
        to_be.append(row['Value'] - base_value)

    # add a column to group
    group['to-be'] = pd.Series(to_be, index=group.index)

    list_groups.append(group)


# Concatenate dataframes
result = pd.concat(list_groups)

print(result)

---

@ASGM, I run your code,

res = df.groupby('Fruit').apply(lambda g: g.Value - g[g.Step == 0].Value.values[0])
df['Result'] = res.reset_index(drop=True)

but encounter the issue,

Traceback (most recent call last):
  File "***.py", line 9, in <module>
    df['Result'] = res.reset_index(drop=True)
  File "/usr/lib/python2.7/dist-packages/pandas/core/frame.py", line 1887, in __setitem__
    self._set_item(key, value)
  File "/usr/lib/python2.7/dist-packages/pandas/core/frame.py", line 1968, in _set_item
    NDFrame._set_item(self, key, value)
  File "/usr/lib/python2.7/dist-packages/pandas/core/generic.py", line 1068, in _set_item
    self._data.set(key, value)
  File "/usr/lib/python2.7/dist-packages/pandas/core/internals.py", line 3024, in set
    self.insert(len(self.items), item, value)
  File "/usr/lib/python2.7/dist-packages/pandas/core/internals.py", line 3039, in insert
    self._add_new_block(item, value, loc=loc)
  File "/usr/lib/python2.7/dist-packages/pandas/core/internals.py", line 3162, in _add_new_block
    self.items, fastpath=True)
  File "/usr/lib/python2.7/dist-packages/pandas/core/internals.py", line 1993, in make_block
    placement=placement)
  File "/usr/lib/python2.7/dist-packages/pandas/core/internals.py", line 64, in __init__
    '%d' % (len(items), len(values)))
ValueError: Wrong number of items passed 1, indices imply 3
[Finished in 0.4s with exit code 1]