I have a df that contains multiple values at duplicate time points. I want to interpolate values for two specific columns but only between unique time points. Using the df below, I want to interpolate X
and Y
only but between unique time points.
import pandas as pd
import numpy as np
df = pd.DataFrame({
'Time' : ['09:00:00.1','09:00:00.1','09:00:00.2','09:00:00.2','09:00:00.3'],
'X' : [1,np.nan,np.nan,np.nan,3],
'Y' : [1,np.nan,np.nan,np.nan,3],
'A' : [5,np.nan,np.nan,np.nan,6],
'B' : [5,np.nan,np.nan,np.nan,6],
})
df1 = df.groupby('Time').apply(lambda x: x.interpolate(method='linear'))
Note: I don't want,
df[['X','Y']] = df[['X','Y']].interpolate(method = 'linear')
The intended output is:
Time X Y A B
0 09:00:00.1 1.0 1.0 5.0 5.0
1 09:00:00.1 1.0 1.0 Nan NaN
2 09:00:00.2 2.0 2.0 NaN NaN
3 09:00:00.2 2.0 2.0 NaN NaN
4 09:00:00.3 3.0 3.0 6.0 6.0
First we drop_duplicates
based on Time
to get unique rows, then we interpolate, and update our original dataframe with these values.
Finally we use ffill
to forwardfill our values:
interpolation = df.drop_duplicates('Time')[['X', 'Y']].interpolate()
df.loc[interpolation.index, ['X', 'Y']] = interpolation
df.loc[:, ['X', 'Y']] = df[['X', 'Y']].ffill()
Time X Y A B
0 09:00:00.1 1.00 1.00 5.00 5.00
1 09:00:00.1 1.00 1.00 nan nan
2 09:00:00.2 2.00 2.00 nan nan
3 09:00:00.2 2.00 2.00 nan nan
4 09:00:00.3 3.00 3.00 6.00 6.00
Another method would be to use np.floor
but this only works if you have the scenario as in your example dataframe (and this will probably not be the case):
df[['X', 'Y']] = np.floor(df[['X', 'Y']].interpolate())
Time X Y A B
0 09:00:00.1 1.00 1.00 5.00 5.00
1 09:00:00.1 1.00 1.00 nan nan
2 09:00:00.2 2.00 2.00 nan nan
3 09:00:00.2 2.00 2.00 nan nan
4 09:00:00.3 3.00 3.00 6.00 6.00
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.