用多个组插补缺失值

Question

I am merging monthly data with quarterly financial data for different companies in Python. Each stock has monthly data for some columns, and only quarterly data for others. Below is a sample dataframe.

import numpy as np
import pandas as pd 
raw_data = {'gvkey': [1004, 1004, 1004, 1004, 1004, 1004, 1045, 1045, 1045, 1045, 1045, 1045,], 
        'date': ['2018-08-31', '2018-09-30', '2018-10-31', '2018-11-30', '2018-12-31', '2019-01-31', '2018-08-31', '2018-09-30', '2018-10-31', '2018-11-30', '2018-12-31', '2019-01-31'], 
        'trt1m': [-1.5609, 2.6141, -0.4907, -8.1757, -14.5342, 1.1114, -0.2488, -14.939, 5.6241, 8.5137, 2.3091, -7.335], 
        'epsfxq': [np.NaN, 0.52, np.NaN, np.NaN,  .54, np.NaN, np.NaN, -.28, np.NaN, np.NaN, -3.29, np.NaN],
        'roa': [0.079, 0.079, 0.079, 0.082, 0.082, 0.082, .104, .104, .104, .090, .090, .090]}

df = pd.DataFrame(raw_data, columns = ['gvkey', 'date', 'trt1m', 'epsfxq', 'roa'])
df.head(12)

I am trying to impute missing data for the NaN values I have in my data frame, however, when I groupby the date or the gvkey (read: StockID), I am able to do a forward fill (ffill) or backward fill (bfill) successfully to the missing values, however I lose the date and gvkey columns when I do this.

Does anyone have any advice on how to impute these missing values for multiple groups (grouped by date and gvkey, in this example? I would greatly appreciate any advice you can give.

Thank you

Answer 1

df.fillna(method='ffill') should do the trick, no need to group.

ADDITION To answer the OP concern:

ll=[]
for i, j in df.groupby(gvkey):
   ll.append(j.fillna(method='ffill')
newdf = pd.concat(ll)

Answer 2

This works:

fill_cols = ['epsfxq']
df[fill_cols] = df.groupby(['gvkey'])[fill_cols].ffill()
df[fill_cols] = df.groupby(['gvkey'])[fill_cols].bfill()
df.head(12)

Thanks for your help.