Python Pandas：如何在 Z6A8064B5DF4794555500553C47C55057DZ 的特定列中的两个非连续行中相减

Question

I am trying to populate the values in a new column in a Pandas df by subtracting the value of two non-consecutive rows in a different column within the same df. I can do it, so long as the df does not have a column with dates in it. But if it does have a column with dates then pandas throws an error.

Assume the following dataframe.

import pandas as pd
import numpy as np

df = pd.DataFrame(np.array([[1, 2, 3], [4, 5, 6], [7, 55, 9], [10, 99, 19], [27, 38, 29], [39, 10, 72]]),
                   columns=['a', 'b', 'c'])
df['Date'] = ['2020-01-02', '2020-01-05', '2020-06-10', '2020-08-05', '2020-09-01', '2020-10-29']
df['Date'] = pd.to_datetime(df['Date'])

df['d'] = ''
df = df[['Date', 'a', 'b', 'c', 'd']]

This gives me a df that looks like this:

    Date        a   b   c   d
0   2020-01-02  1   2   3   
1   2020-01-05  4   5   6   
2   2020-06-10  7   55  9   
3   2020-08-05  10  99  19  
4   2020-09-01  27  38  29  
5   2020-10-29  39  10  72  

I am trying to create a new column 'd' that, for each row, subtracts the value in column 'b' two rows below from the row in question. For instance, the value in row [0], column ['d'] would be calculated as df.loc[2]['b'] - df.loc[0]['b'].

What I'm trying (which doesn't work) is:

for i in range(len(df)-2):
    df.loc[i]['d'] = df.loc[i+2]['b'] - df.loc[i]['b']

I can get this to work if I have no date in the df. But when I add a column with dates, it throws an error message saying

A value is trying to be set on a copy of a slice from a DataFrame

I can't figure out why a date column causes the df to be unable to do math on columns with only int64 data. I've tried searching this site and just can't seem to solve the problem. Any help would be greatly appreciated.

Answer 1

You can do it in vectorized form using shift (which is considerably faster than using loops):

df['d'] = df['b'].shift(-2) - df['b']
df

Output:

        Date   a   b   c     d
0 2020-01-02   1   2   3  53.0
1 2020-01-05   4   5   6  94.0
2 2020-06-10   7  55   9 -17.0
3 2020-08-05  10  99  19 -89.0
4 2020-09-01  27  38  29   NaN
5 2020-10-29  39  10  72   NaN