有条件地向前填写大熊猫

Question

我有一个数据帧：

>>> k
Out[87]: 
      Date        S       E           cp  Last         Q            code
30 2017-11-10   22500 2017-11-17       P 170.00     828.47  11/17/2017P22500
32 2017-11-10   22625 2017-11-17       P 180.00     646.91  11/17/2017P22625
35 2017-11-10   22750 2017-11-17       C 145.00     651.84  11/17/2017C22750
36 2017-11-13   22500 2017-11-17       P 245.00        nan  11/17/2017P22500
38 2017-11-13   22625 2017-11-17       P 315.00        nan  11/17/2017P22625
41 2017-11-13   22750 2017-11-17       C  35.00        nan  11/17/2017C22750
42 2017-11-14   22500 2017-11-17       P 215.00        nan  11/17/2017P22500
44 2017-11-14   22625 2017-11-17       P 305.00        nan  11/17/2017P22625
47 2017-11-14   22750 2017-11-17       C  26.00        nan  11/17/2017C22750
48 2017-11-15   22500 2017-11-17       P 490.00        nan  11/17/2017P22500
50 2017-11-15   22625 2017-11-17       P 605.00        nan  11/17/2017P22625
53 2017-11-15   22750 2017-11-17       C   4.00        nan  11/17/2017C22750
54 2017-11-16   22500 2017-11-17       P 140.00        nan  11/17/2017P22500
56 2017-11-16   22625 2017-11-17       P 295.00        nan  11/17/2017P22625
59 2017-11-16   22750 2017-11-17       C   4.00        nan  11/17/2017C22750
60 2017-11-17   22250 2017-11-24       P 165.00     707.57  11/24/2017P22250
61 2017-11-17   22375 2017-11-24       P 195.00     607.16  11/24/2017P22375
65 2017-11-17   22500 2017-11-24       C 175.00     666.56  11/24/2017C22500
66 2017-11-20   22250 2017-11-24       P 175.00        nan  11/24/2017P22250
67 2017-11-20   22375 2017-11-24       P 225.00        nan  11/24/2017P22375
71 2017-11-20   22500 2017-11-24       C  75.00        nan  11/24/2017C22500
72 2017-11-21   22250 2017-11-24       P  70.00        nan  11/24/2017P22250
73 2017-11-21   22375 2017-11-24       P 120.00        nan  11/24/2017P22375
77 2017-11-21   22500 2017-11-24       C  95.00        nan  11/24/2017C22500
78 2017-11-22   22250 2017-11-24       P  15.00        nan  11/24/2017P22250
79 2017-11-22   22375 2017-11-24       P  35.00        nan  11/24/2017P22375
83 2017-11-22   22500 2017-11-24       C 125.00        nan  11/24/2017C22500
84 2017-11-24   22375 2017-12-01       P 140.00     834.13  12/01/2017P22375
85 2017-11-24   22500 2017-12-01       P 185.00     763.76  12/01/2017P22500
89 2017-11-24   22625 2017-12-01       C 165.00     750.45  12/01/2017C22625

我想根据代码列填充Q列中的nans。 例如，索引行30中的代码与行36中的代码相同，所以我想在那里放置相同的Q.

我目前正在以下列方式做，有更好的方法吗？

k1= k.drop(['Date','S','E','cp','Last'],axis=1).dropna()
k1.columns =['Q_new', 'code']
k2 = k.merge(k1, on = 'code')
k2= k2.drop(['Q'],axis=1)
k2 = k2.sort('Date')

Answer 1

groupby + ffill和bfill

df.Q=df.groupby('code').Q.apply(lambda x : x.ffill().bfill())
df
Out[755]: 
          Date      S           E cp   Last       Q              code
30  2017-11-10  22500  2017-11-17  P  170.0  828.47  11/17/2017P22500
32  2017-11-10  22625  2017-11-17  P  180.0  646.91  11/17/2017P22625
35  2017-11-10  22750  2017-11-17  C  145.0  651.84  11/17/2017C22750
36  2017-11-13  22500  2017-11-17  P  245.0  828.47  11/17/2017P22500
38  2017-11-13  22625  2017-11-17  P  315.0  646.91  11/17/2017P22625
41  2017-11-13  22750  2017-11-17  C   35.0  651.84  11/17/2017C22750
42  2017-11-14  22500  2017-11-17  P  215.0  828.47  11/17/2017P22500
44  2017-11-14  22625  2017-11-17  P  305.0  646.91  11/17/2017P22625
47  2017-11-14  22750  2017-11-17  C   26.0  651.84  11/17/2017C22750
48  2017-11-15  22500  2017-11-17  P  490.0  828.47  11/17/2017P22500
50  2017-11-15  22625  2017-11-17  P  605.0  646.91  11/17/2017P22625
53  2017-11-15  22750  2017-11-17  C    4.0  651.84  11/17/2017C22750
54  2017-11-16  22500  2017-11-17  P  140.0  828.47  11/17/2017P22500
56  2017-11-16  22625  2017-11-17  P  295.0  646.91  11/17/2017P22625
59  2017-11-16  22750  2017-11-17  C    4.0  651.84  11/17/2017C22750
60  2017-11-17  22250  2017-11-24  P  165.0  707.57  11/24/2017P22250
61  2017-11-17  22375  2017-11-24  P  195.0  607.16  11/24/2017P22375
65  2017-11-17  22500  2017-11-24  C  175.0  666.56  11/24/2017C22500
66  2017-11-20  22250  2017-11-24  P  175.0  707.57  11/24/2017P22250
67  2017-11-20  22375  2017-11-24  P  225.0  607.16  11/24/2017P22375
71  2017-11-20  22500  2017-11-24  C   75.0  666.56  11/24/2017C22500
72  2017-11-21  22250  2017-11-24  P   70.0  707.57  11/24/2017P22250
73  2017-11-21  22375  2017-11-24  P  120.0  607.16  11/24/2017P22375
77  2017-11-21  22500  2017-11-24  C   95.0  666.56  11/24/2017C22500
78  2017-11-22  22250  2017-11-24  P   15.0  707.57  11/24/2017P22250
79  2017-11-22  22375  2017-11-24  P   35.0  607.16  11/24/2017P22375
83  2017-11-22  22500  2017-11-24  C  125.0  666.56  11/24/2017C22500
84  2017-11-24  22375  2017-12-01  P  140.0  834.13  12/01/2017P22375
85  2017-11-24  22500  2017-12-01  P  185.0  763.76  12/01/2017P22500
89  2017-11-24  22625  2017-12-01  C  165.0  750.45  12/01/2017C22625

Answer 2

您可以在groupby对象上使用transform 。

df.loc[:, 'Q'] = df.groupby('code')['Q'].transform(lambda group: group.ffill())

计时

%timeit -n 1000 df.loc[:, 'Q'] = df.groupby('code')['Q'].transform(lambda group: group.ffill())
# 1000 loops, best of 3: 2.41 ms per loop

%timeit -n 1000 df.loc[:, 'Q'] = df.groupby('code')['Q'].ffill()
# 1000 loops, best of 3: 3.66 ms per loop