在最接近特定时间的数据框中查找值

Question

I have a dataframe with a date-time column and a value column, I'd like to find a way to create another column with the value at the time closest to a given interval before the date-time.

What I'd like is to have a column called "Value 2 hours ago", and have the value of this column correspond to the "Value" column at the time that was the closest to 2 hours ago.

For example, if the "Date-Time" column shows "01/01/2014 12:10:00", this new column will return the number in "Value" in the line for which "Date-Time" is closest to "01/01/2014 10:10:00"

Even better if I can apply some conditions on this value based on how far the real time interval is from the desired "2 hours" interval. For example, "return the value closest to 2 hours ago, except if it's less than 1 hour ago or more than 3 hours ago, then return nothing"

To illustrate, here is a sample input dataframe. I can easily get the value 2 hours ago, and then self-merge it on the two date-time columns. The challenge is to have this merge be on the nearest match, rather than an exact match.

df = pd.DataFrame({'Date-Time' : pd.Series(["01/01/2014 04:11:00", "01/01/2014 08:10:00","01/01/2014 09:11:00","01/01/2014 12:10:00"], index=['1', '2','3', '4']),'Value' : pd.Series([9,12,3,21], index=['1', '2','3','4'])})
df["Time"]=pd.to_datetime(df["Time"])
df["t_2h_ago"]=df["Time"]-pd.to_timedelta('2h')
merged=pd.merge(df,df,how='left',left_on='Time',right_on='t_2h_ago')

Answer 1

Take the cartesian product. Then find the difference between the timestamps. Note I assumed that each date-time is unique in the function named nearest_time. Then group by and calculate the min of each group. For each group, this gives you the closest timestamp in seconds. Then join back.

from datetime import datetime
import time
import pandas as pd
import numpy as np
df = pd.DataFrame({'Date-Time' : pd.Series(["01/01/2014 04:11:00", "01/01/2014 08:10:00","01/01/2014 09:11:00","01/01/2014 12:10:00"], index=['1', '2','3', '4']),'Value' : pd.Series([9,12,3,21], index=['1', '2','3','4'])})

def nearest_time(x):
    row_i= datetime.strptime(x['Date-Time_x'], "%m/%d/%Y %H:%M:%S")
    row_j = datetime.strptime(x['Date-Time_y'], "%m/%d/%Y %H:%M:%S")
    diff = time.mktime(row_i.timetuple()) - time.mktime(row_j.timetuple()) #seconds ex(2 hrs)
    if diff == 0: diff = float('inf')
    return abs(diff)

df = df.copy()
df['key']=1
df = pd.merge(df,df,on='key')
df['diff'] = df.apply(nearest_time,axis=1)
df2 = df.copy()
df2= df2.groupby(['Date-Time_x']).agg({'diff': np.min})
df2 = df2[['diff']]
df2['Date-Time_x']=df2.index

df3 = pd.merge(df2,df, on=['diff',"Date-Time_x"])
print df3