当 pandas 中的时间增量索引不规则时，如何获取时间序列值的每日差异？

Question

I have a dataframe containing a time series indexed by time but with irregular time deltas as below

df
time                  x
2018-08-18 17:45:08   1.4562
2018-08-18 17:46:55   1.4901
2018-08-18 17:51:21   1.8012
...
2020-03-21 04:17:19   0.7623
2020-03-21 05:01:02   0.8231
2020-03-21 05:02:34   0.8038

What I want to do is get the daily difference between the two (chronologically) closest values , ie the closest time the next day. For example, if we have a sample at time 2018-08-18 17:45:08, and the next day we do not have a sample at the same time, but the closest sample is at, say, 2018-08-19 17:44:29, then I want to get the difference in x between these two times. How is that possible in pandas?

• There will always be a sample for every single day between first day and last day in the time series.
• The difference should be taken as (current x) - (past x) eg x_day2 - x_day1
• The output's first n rows will be NaN given how the difference is taken, where n is the number of samples in the first day

EDIT: The code below works if the time deltas are regular

def get_daily_diff(data):
    """
    Calculate daily difference in time series

    Args:
        data (pandas.Series): a pandas series of time series values indexed by pandas.Timestamp

    Returns:
        pandas.Series: daily difference in values
    """
    df0 = data.index.searchsorted(data.index - pd.Timedelta(days=1))
    df0 = df0[df0 > 0]
    df0 = pd.Series(data.index[df0 - 1], index=data.index[data.shape[0] - df0.shape[0]:])
    out = data.loc[df0.index] - data.loc[df0.values]
    return out

However, if using irregular time delats, a ValueError is thrown when defining the variable out as we get a length mismatch between data.loc[df0.index] and data.loc[df0.values] . So the issue is to expand this function to work when the time deltas are irregular.

Answer 1

I would use pd.merge_asof with direction='nearest' :

df['time_1d'] = df['time']+pd.Timedelta('1D')
tmp = pd.merge_asof(df, df, left_on='time', right_on ='time_1d',
           direction='nearest', tolerance=pd.Timedelta('12H'), suffixes=('', '_y'))
tmp['delta'] = tmp['x_y'] - tmp['x']
tmp = tmp[['time', 'x', 'delta']]

Here I have used a tolerance of 12H to make sure to have NaN for first days but you could use a more appropriate value.