根據列值復制 dataframe 行

Question

我有一個 dataframe 的 2 個月數據，有 20 列，其中一列是'date' 。 有 3 個不連續的日期沒有數據。 我想復制前一天的數據來為那些缺失的日子創建條目。

這是我嘗試過的：

df_replicate=df[(df['date']=='2021-07-27') | (df['date']=='2021-08-18') | (df['date']=='2021-08-22')]

df_replicate.loc[df_replicate['date']=='2021-07-27']='2021-07-28'
df_replicate.loc[df_replicate['date']=='2021-08-18']='2021-08-19'
df_replicate.loc[df_replicate['date']=='2021-08-22']='2021-08-23'

然后連接df和df_replicate

什么是更簡單的方法來做到這一點？

Answer 1

您可以將 reindex 與“ffill”參數一起使用：

import pandas as pd
import numpy as np


date_index = pd.date_range('2021-07-27', periods=7, freq='D')

# set data to date time index
df = pd.DataFrame({"prices": [100, np.nan, 100, 89, 88, np.nan, np.nan ]},index=date_index)

2021-07-27  100.0
2021-07-28    NaN
2021-07-29  100.0
2021-07-30   89.0
2021-07-31   88.0
2021-08-01    NaN
2021-08-02    NaN

# remove one of the date values to represent missing data
df = df[~(df.index=='2021-07-28')]

2021-07-27  100.0
2021-07-29  100.0
2021-07-30   89.0
2021-07-31   88.0
2021-08-01    NaN
2021-08-02    NaN

# Second date index with correct number of days
date_index2 = pd.date_range('2021-07-27', periods=7, freq='D')

# df with missing row foward filled
df.reindex(date_index2, method="ffill")

2021-07-27  100.0 #This value is carried to the next date
2021-07-28  100.0 
2021-07-29  100.0
2021-07-30   89.0
2021-07-31   88.0
2021-08-01    NaN
2021-08-02    NaN

https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.reindex.html#pandas.DataFrame.reindex

Answer 2

使用由Index.shift創建的下一個值按列表過濾匹配的行，並為這些對使用前向填充缺失值：

df = pd.DataFrame({"prices": [100, np.nan, 100, 89, 88, np.nan, np.nan ],
                  'date': pd.date_range('2021-07-27', periods=7, freq='D')})


df['date'] = pd.to_datetime(df['date'])

dates = pd.to_datetime(['2021-07-27','2021-08-18','2021-08-22'])

mask = df['date'].isin(dates.append(dates.shift(freq='d')))
df[mask] = df[mask].ffill()
print (df)
   prices       date
0   100.0 2021-07-27
1   100.0 2021-07-28
2   100.0 2021-07-29
3    89.0 2021-07-30
4    88.0 2021-07-31
5     NaN 2021-08-01
6     NaN 2021-08-02

如果只需要用所有以前的非 NaN 替換下一行（由 NaN 填充）：

df['date'] = pd.to_datetime(df['date'])

dates = pd.to_datetime(['2021-07-27','2021-08-18','2021-08-22'])

mask = df['date'].isin(dates.shift(freq='d'))
df[mask] = df.ffill()

如果輸入列表不同，則下一個值 ( ['2021-07-28','2021-08-19','2021-08-23'] ) 必須為之前的匹配值移動：

df['date'] = pd.to_datetime(df['date'])
dates = pd.to_datetime(['2021-07-28','2021-08-19','2021-08-23'])

mask = df['date'].isin(dates.append(dates.shift(-1, freq='d')))
df[mask] = df[mask].ffill()