How to fill values from a pandas dataframe to another dataframe with different datetimeindex

Question

I have two dataframes. One has a 5 minute granularity (df1), the other is indexed by days (df2). For the sake of this example the days end at 7:10

df1:

            Date    Close
2019-06-20 07:00:00 2927.25
2019-06-20 07:05:00 2927.00
2019-06-20 07:10:00 2926.75
2019-06-21 07:00:00 2932.25
2019-06-21 07:05:00 2932.25
2019-06-21 07:10:00 2931.00
2019-06-24 07:00:00 2941.75
2019-06-24 07:05:00 2942.25
2019-06-24 07:10:00 2941.50
2019-06-25 07:00:00 2925.50
2019-06-25 07:05:00 2926.50
2019-06-25 07:10:00 2926.50

df2:

            range                       
Date                            
2019-06-20  115.0
2019-06-21  86.0    
2019-06-24  52.0
2019-06-25  132.0   

Now I'd like to take the values from 'range' column of df2 and and inject them repetitive in a new column in df1.

It should look like this:

            Date    Close       range
2019-06-20 07:00:00 2927.25     115.0
2019-06-20 07:05:00 2927.00     115.0
2019-06-20 07:10:00 2926.75     115.0
2019-06-21 07:00:00 2932.25     86.0    
2019-06-21 07:05:00 2932.25     86.0    
2019-06-21 07:10:00 2931.00     86.0    
2019-06-24 07:00:00 2941.75     52.0
2019-06-24 07:05:00 2942.25     52.0
2019-06-24 07:10:00 2941.50     52.0
2019-06-25 07:00:00 2925.50     132.0
2019-06-25 07:05:00 2926.50     132.0
2019-06-25 07:10:00 2926.50     132.0

Unfortunately I don't know how to start that's why there's no 'my attempt' code How would you do this?

Answer 1

First convert the date like columns to pandas datetime series:

df1['Date'] = pd.to_datetime(df1['Date'])
df2.index = pd.to_datetime(df2.index)

Use Series.dt.date + Series.map to map range values from df2 to df1 :

df1['range'] = df1['Date'].dt.date.map(df2.set_index(df2.index.date)['range'])

OR its also possible to use DataFrame.merge :

df1.assign(k=df1['Date'].dt.date).merge(df2.assign(k=df2.index.date), on='k').drop('k', 1)

Result:

                  Date    Close  range
0  2019-06-20 07:00:00  2927.25  115.0
1  2019-06-20 07:05:00  2927.00  115.0
2  2019-06-20 07:10:00  2926.75  115.0
3  2019-06-21 07:00:00  2932.25   86.0
4  2019-06-21 07:05:00  2932.25   86.0
5  2019-06-21 07:10:00  2931.00   86.0
6  2019-06-24 07:00:00  2941.75   52.0
7  2019-06-24 07:05:00  2942.25   52.0
8  2019-06-24 07:10:00  2941.50   52.0
9  2019-06-25 07:00:00  2925.50  132.0
10 2019-06-25 07:05:00  2926.50  132.0
11 2019-06-25 07:10:00  2926.50  132.0

Answer 2

If you want to make a loop, do something like that:

for i in df2["Date"]:
     for j in df1["Date"]:
          if i==j:
              df1['range'] = df2['range']

Answer 3

1. need to ensure your Date columns are dates
2. floor them to day granularity so they are same across data frames
3. cleanup the columns

df1 = pd.DataFrame({"Date":["2019-06-19T23:00:00.000Z","2019-06-19T23:05:00.000Z","2019-06-19T23:10:00.000Z","2019-06-20T23:00:00.000Z","2019-06-20T23:05:00.000Z","2019-06-20T23:10:00.000Z","2019-06-23T23:00:00.000Z","2019-06-23T23:05:00.000Z","2019-06-23T23:10:00.000Z","2019-06-24T23:00:00.000Z","2019-06-24T23:05:00.000Z","2019-06-24T23:10:00.000Z"],"Close":[2927.25,2927,2926.75,2932.25,2932.25,2931,2941.75,2942.25,2941.5,2925.5,2926.5,2926.5]})
df2 = pd.DataFrame({"Date":["2019-06-19T16:00:00.000Z","2019-06-20T16:00:00.000Z","2019-06-23T16:00:00.000Z","2019-06-24T16:00:00.000Z"],"range":[115,86,52,132]})

df1.Date = pd.to_datetime(df1.Date)
df2.Date = pd.to_datetime(df2.Date)


df1.assign(day=df1.Date.dt.floor("D"))\
    .merge(df2.assign(day=df2.Date.dt.floor("D")), on="day")\
    .drop(["day","Date_y"],1).rename({"Date_x":"Date"},axis=1)

output

                     Date    Close  range
2019-06-19 23:00:00+00:00  2927.25    115
2019-06-19 23:05:00+00:00  2927.00    115
2019-06-19 23:10:00+00:00  2926.75    115
2019-06-20 23:00:00+00:00  2932.25     86
2019-06-20 23:05:00+00:00  2932.25     86
2019-06-20 23:10:00+00:00  2931.00     86
2019-06-23 23:00:00+00:00  2941.75     52
2019-06-23 23:05:00+00:00  2942.25     52
2019-06-23 23:10:00+00:00  2941.50     52
2019-06-24 23:00:00+00:00  2925.50    132
2019-06-24 23:05:00+00:00  2926.50    132
2019-06-24 23:10:00+00:00  2926.50    132