Column of original indices in groupby pandas

Question

I'm using the following function:

first_last = df.groupby(['stock', Grouper(freq='D')])['price'].agg(['first','last'])

, which gives me a dataframe with the first non-nan and the last non-nan prices per day per stock.

Could you please hel me, how can I add two columns to the created "first_last" df, so that they contain the original indices of dataframe "df", from which first&last values are taken?

The original df is of the following form:

    Index                  price              stock            
2016-10-21 17:00:00        150                 85
2016-10-21 17:30:00        100                 85
2016-10-21 17:00:00         50                 88

-- I need to have "Index" in front of each value of first and last price values in df "first_last".

Answer 1

You need create helper column from DatetimeIndex with same missing values like price columns and then aggregate both columns:

df['idx'] = df.index.where(df['price'].notnull(),  np.nan)

first_last = df.groupby(['stock', pd.Grouper(freq='D')])['price', 'idx'].agg(['first','last'])
first_last.columns = first_last.columns.map('_'.join)
print (first_last)
                  price_first  price_last           idx_first  \
stock Index                                                     
85    2016-10-21          150         100 2016-10-21 17:00:00   
88    2016-10-21           50          50 2016-10-21 17:00:00   

                            idx_last  
stock Index                           
85    2016-10-21 2016-10-21 17:30:00  
88    2016-10-21 2016-10-21 17:00:00