Creating two shifted columns in grouped pandas data-frame

Question

I have looked all over and I still can't find an example of how to create two shifted columns in a Pandas Dataframe within its groups.

I have done it with one column as follows:

data_frame['previous_category'] = data_frame.groupby('id')['category'].shift()

But I have to do it with 2 columns, shifting one upwards and the other downwards.

Any ideas?

Answer 1

It is possible by custom function with GroupBy.apply , because one column need shift down and second shift up:

df = pd.DataFrame({
         'B':[4,5,4,5,5,4],
         'C':[7,8,9,4,2,3],
         'F':list('aaabbb')
})

def f(x):
    x['B'] = x['B'].shift()
    x['C'] = x['C'].shift(-1)
    return x

df = df.groupby('F').apply(f)
print (df)
     B    C  F
0  NaN  8.0  a
1  4.0  9.0  a
2  5.0  NaN  a
3  NaN  2.0  b
4  5.0  3.0  b
5  5.0  NaN  b

If want shift same way only specify all columns in lists:

df[['B','C']] = df.groupby('F')['B','C'].shift()
print (df)
     B    C  F
0  NaN  NaN  a
1  4.0  7.0  a
2  5.0  8.0  a
3  NaN  NaN  b
4  5.0  4.0  b
5  5.0  2.0  b