I have a dataframe holding trade data every 5 minutes, like
open close
datetime
2015-02-02 08:00:00 43.5 NaN
2015-02-02 08:10:00 43.3 0
2015-02-02 08:15:00 43.2 7
2015-02-02 08:20:00 NaN NaN
2015-02-02 08:25:00 43.1 9
2015-02-02 08:35:00 43.0 9
2015-02-02 08:40:00 43.0 11
2015-02-02 08:45:00 NaN NaN
2015-02-02 08:50:00 NaN NaN
2015-02-02 08:55:00 NaN NaN
2015-02-02 09:00:00 43.1 9
and I am looking to fill the missing rows like at the 08:30:00 timestamp, with just np.nan
and then forward fill. I've looked into using the pd.date_range
function to calculate the index per five minute interval from a start to an end date, and just naively assigning that to be my dataframe's index, but as I thought, that raises an error.
I also looked at this question which is very similar to what I'm asking, but the answer uses resample
. I don't know how that solved the OP's problem because you can't treat the resample object like a dataframe and query it in the same way, as far as I know.
EDIT: I ended up finding a way to get this done. I made a dataframe with the same columns with the whole date range I want using date_range
, and then updating this dataframe with the values I actually have from the trade data using update
to get something out of the resample object, you need to add a dispatching method (see the docs ), eg:
import numpy as np
import pandas as pd
df = pd.DataFrame({'open': [43.5,43.3,43.2,np.NaN,43.1,43.0,43.0,np.NaN,np.NaN,np.NaN,43.1],
'close': [np.NaN,0,7,np.NaN,9,9,11,np.NaN,np.NaN,np.NaN,9]},
index = pd.to_datetime(['2015-02-02 08:00:00','2015-02-02 08:10:00','2015-02-02 08:15:00',
'2015-02-02 08:20:00','2015-02-02 08:25:00','2015-02-02 08:35:00',
'2015-02-02 08:40:00','2015-02-02 08:45:00','2015-02-02 08:50:00',
'2015-02-02 08:55:00','2015-02-02 09:00:00']))
df1 = df.resample('5min').mean()
# df1
# open close
# 2015-02-02 08:00:00 43.5 NaN
# 2015-02-02 08:05:00 NaN NaN
# 2015-02-02 08:10:00 43.3 0.0
# 2015-02-02 08:15:00 43.2 7.0
# 2015-02-02 08:20:00 NaN NaN
# 2015-02-02 08:25:00 43.1 9.0
# 2015-02-02 08:30:00 NaN NaN
# 2015-02-02 08:35:00 43.0 9.0
# 2015-02-02 08:40:00 43.0 11.0
# 2015-02-02 08:45:00 NaN NaN
# 2015-02-02 08:50:00 NaN NaN
# 2015-02-02 08:55:00 NaN NaN
# 2015-02-02 09:00:00 43.1 9.0
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.