熊猫-在保留列/索引值的同时向DataFrame添加缺少的日期？
[英]Pandas- adding missing dates to DataFrame while keeping column/index values?
问题描述
I have a pandas dataframe that incorporates dates, customers, items, and then dollar value for purchases. 
我有一个熊猫数据框，其中包含日期，客户，物品，然后是购买的美元价值。
date customer product amt
1/1/2017 tim apple 3
1/1/2017 jim melon 2
1/1/2017 tom apple 5
1/1/2017 tom melon 4
1/4/2017 tim melon 3
1/4/2017 jim apple 2
1/4/2017 tom melon 1
1/4/2017 tom orange 4
I'm trying to just look at performance, but I want to forward fill all dates from my min and max date range, and also fill for each customer for each product 我只想查看效果，但我想从最小和最大日期范围中向前填充所有日期，并为每个产品的每个客户填充
Something like: 就像是：
date customer product amt
1/1/2017 tim apple 3
1/1/2017 tim melon 0
1/1/2017 tim orange 0
1/1/2017 jim melon 2
1/1/2017 jim apple 0
1/1/2017 jim orange 0
1/1/2017 tom apple 5
1/1/2017 tom melon 4
1/1/2017 tom orange 0
1/2/2017 tim apple 0
1/2/2017 tim melon 0
1/2/2017 tim orange 0
1/2/2017 jim melon 0
1/2/2017 jim apple 0
1/2/2017 jim orange 0
1/2/2017 tom apple 0
1/2/2017 tom melon 0
1/2/2017 tom orange 0
1/3/2017 tim apple 0
1/3/2017 tim melon 0
1/3/2017 tim orange 0
1/3/2017 jim melon 0
1/3/2017 jim apple 0
1/3/2017 jim orange 0
1/3/2017 tom apple 0
1/3/2017 tom melon 0
1/3/2017 tom orange 0
1/4/2017 tim melon 3
1/4/2017 tim apple 0
1/4/2017 tim orange 0
1/4/2017 jim apple 2
1/4/2017 jim melon 0
1/4/2017 jim orange 0
1/4/2017 tom melon 1
1/4/2017 tom orange 4
1/4/2017 tom apple 0
I know that I can create a reindex based off of the max and min dates, but this also makes my customer and product values 0. Is there any other way to go about this? 我知道我可以根据最大日期和最小日期创建一个重新索引，但这也使我的客户和产品值均为0。还有其他解决方法吗？ Am I missing a step or something? 我错过了一步吗？ Appreciate the help 感谢帮助
4 个解决方案
解决方案1
2 已采纳 2017-12-18 22:03:37
Notice ,this using the stack and unstack couple of times 请注意，这个使用stack和unstack几次
df.set_index(['date','customer','product']).amt.unstack(-3).\
reindex(columns=pd.date_range(df['date'].min(),
df['date'].max()),fill_value=0).\
stack(dropna=False).unstack().stack(dropna=False).\
unstack('customer').stack(dropna=False).reset_index().\
fillna(0).sort_values(['level_1','customer','product'])
Out[314]:
product level_1 customer 0
0 apple 2017-01-01 jim 0.0
12 melon 2017-01-01 jim 2.0
24 orange 2017-01-01 jim 0.0
1 apple 2017-01-01 tim 3.0
13 melon 2017-01-01 tim 0.0
25 orange 2017-01-01 tim 0.0
2 apple 2017-01-01 tom 5.0
14 melon 2017-01-01 tom 4.0
26 orange 2017-01-01 tom 0.0
3 apple 2017-01-02 jim 0.0
15 melon 2017-01-02 jim 0.0
27 orange 2017-01-02 jim 0.0
4 apple 2017-01-02 tim 0.0
16 melon 2017-01-02 tim 0.0
28 orange 2017-01-02 tim 0.0
5 apple 2017-01-02 tom 0.0
17 melon 2017-01-02 tom 0.0
29 orange 2017-01-02 tom 0.0
6 apple 2017-01-03 jim 0.0
18 melon 2017-01-03 jim 0.0
30 orange 2017-01-03 jim 0.0
7 apple 2017-01-03 tim 0.0
19 melon 2017-01-03 tim 0.0
31 orange 2017-01-03 tim 0.0
8 apple 2017-01-03 tom 0.0
20 melon 2017-01-03 tom 0.0
32 orange 2017-01-03 tom 0.0
9 apple 2017-01-04 jim 2.0
21 melon 2017-01-04 jim 0.0
33 orange 2017-01-04 jim 0.0
10 apple 2017-01-04 tim 0.0
22 melon 2017-01-04 tim 3.0
34 orange 2017-01-04 tim 0.0
11 apple 2017-01-04 tom 0.0
23 melon 2017-01-04 tom 1.0
35 orange 2017-01-04 tom 4.0
解决方案2
1 2017-12-18 21:52:55
IIUC you can do it this way: IIUC您可以通过以下方式进行操作：
In [63]: dates = pd.date_range(df['date'].min(), df['date'].max())
In [64]: idx = pd.MultiIndex.from_product((dates,
df['customer'].unique(),
df['product'].unique()))
In [72]: (df.set_index(['date','customer','product'])
.reindex(idx, fill_value=0)
.reset_index()
.set_axis(df.columns, axis=1, inplace=False))
Out[72]:
date customer product amt
0 2017-01-01 tim apple 3
1 2017-01-01 tim melon 0
2 2017-01-01 tim orange 0
3 2017-01-01 jim apple 0
4 2017-01-01 jim melon 2
5 2017-01-01 jim orange 0
6 2017-01-01 tom apple 5
7 2017-01-01 tom melon 4
8 2017-01-01 tom orange 0
9 2017-01-02 tim apple 0
.. ... ... ... ...
26 2017-01-03 tom orange 0
27 2017-01-04 tim apple 0
28 2017-01-04 tim melon 3
29 2017-01-04 tim orange 0
30 2017-01-04 jim apple 2
31 2017-01-04 jim melon 0
32 2017-01-04 jim orange 0
33 2017-01-04 tom apple 0
34 2017-01-04 tom melon 1
35 2017-01-04 tom orange 4
[36 rows x 4 columns]
解决方案3
1 2017-12-18 22:29:31
Maybe because of my SQL mindset, consider a left join merge on an expanded helper dataframe: 也许是由于我的SQL思维方式，考虑在扩展的帮助器数据帧上进行左联接merge ：
helper_df_list = [pd.DataFrame({'date': pd.date_range(df['date'].min(), df['date'].max()),
'customer': c, 'product': p })
for c in df['customer'].unique()
for p in df['product'].unique()]
helper_df = pd.concat(helper_df_list, ignore_index=True)
final_df = pd.merge(helper_df, df, on=['date', 'customer', 'product'], how='left')\
.fillna(0).sort_values(['date', 'customer']).reset_index(drop=True)
Output 输出量
print(final_df)
# customer date product amt
# 0 jim 2017-01-01 apple 0.0
# 1 jim 2017-01-01 melon 2.0
# 2 jim 2017-01-01 orange 0.0
# 3 tim 2017-01-01 apple 3.0
# 4 tim 2017-01-01 melon 0.0
# 5 tim 2017-01-01 orange 0.0
# 6 tom 2017-01-01 apple 5.0
# 7 tom 2017-01-01 melon 4.0
# 8 tom 2017-01-01 orange 0.0
# 9 jim 2017-01-02 apple 0.0
# 10 jim 2017-01-02 melon 0.0
# 11 jim 2017-01-02 orange 0.0
# 12 tim 2017-01-02 apple 0.0
# 13 tim 2017-01-02 melon 0.0
# 14 tim 2017-01-02 orange 0.0
# 15 tom 2017-01-02 apple 0.0
# 16 tom 2017-01-02 melon 0.0
# 17 tom 2017-01-02 orange 0.0
# 18 jim 2017-01-03 apple 0.0
# 19 jim 2017-01-03 melon 0.0
# 20 jim 2017-01-03 orange 0.0
# 21 tim 2017-01-03 apple 0.0
# 22 tim 2017-01-03 melon 0.0
# 23 tim 2017-01-03 orange 0.0
# 24 tom 2017-01-03 apple 0.0
# 25 tom 2017-01-03 melon 0.0
# 26 tom 2017-01-03 orange 0.0
# 27 jim 2017-01-04 apple 2.0
# 28 jim 2017-01-04 melon 0.0
# 29 jim 2017-01-04 orange 0.0
# 30 tim 2017-01-04 apple 0.0
# 31 tim 2017-01-04 melon 3.0
# 32 tim 2017-01-04 orange 0.0
# 33 tom 2017-01-04 apple 0.0
# 34 tom 2017-01-04 melon 1.0
# 35 tom 2017-01-04 orange 4.0
解决方案4
1 2017-12-18 22:36:58
Let's use product from itertools, pd.date_range , and merge : 让我们使用itertools的product pd.date_range和merge ：
from itertools import product
daterange = pd.date_range(df['date'].min(), df['date'].max(), freq='D')
d1 = pd.DataFrame(list(product(daterange,
df['customer'].unique(),
df['product'].unique())),
columns=['date', 'customer', 'product'])
d1.merge(df, on=['date', 'customer', 'product'], how='left').fillna(0)
Output: 输出：
date customer product amt
0 2017-01-01 tim apple 3.0
1 2017-01-01 tim melon 0.0
2 2017-01-01 tim orange 0.0
3 2017-01-01 jim apple 0.0
4 2017-01-01 jim melon 2.0
5 2017-01-01 jim orange 0.0
6 2017-01-01 tom apple 5.0
7 2017-01-01 tom melon 4.0
8 2017-01-01 tom orange 0.0
9 2017-01-02 tim apple 0.0
10 2017-01-02 tim melon 0.0
11 2017-01-02 tim orange 0.0
12 2017-01-02 jim apple 0.0
13 2017-01-02 jim melon 0.0
14 2017-01-02 jim orange 0.0
15 2017-01-02 tom apple 0.0
16 2017-01-02 tom melon 0.0
17 2017-01-02 tom orange 0.0
18 2017-01-03 tim apple 0.0
19 2017-01-03 tim melon 0.0
20 2017-01-03 tim orange 0.0
21 2017-01-03 jim apple 0.0
22 2017-01-03 jim melon 0.0
23 2017-01-03 jim orange 0.0
24 2017-01-03 tom apple 0.0
25 2017-01-03 tom melon 0.0
26 2017-01-03 tom orange 0.0
27 2017-01-04 tim apple 0.0
28 2017-01-04 tim melon 3.0
29 2017-01-04 tim orange 0.0
30 2017-01-04 jim apple 2.0
31 2017-01-04 jim melon 0.0
32 2017-01-04 jim orange 0.0
33 2017-01-04 tom apple 0.0
34 2017-01-04 tom melon 1.0
35 2017-01-04 tom orange 4.0
声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.
相关问题
如何在缺少日期的情况下重新采样重新采样的熊猫DataFrame以保持当前值 - How to reindex resampled pandas DataFrame with missing dates keeping current values
Pandas-通过对列和索引的值求和来合并两个数据框 - Pandas- merging two dataframe by sum the values of columns and index
pandas-向数据框添加一系列会导致出现NaN值 - pandas- adding a series to a dataframe causes NaN values to appear
python pandas-在您所在的行上方添加一列的值 - python pandas- adding values of a column above the row you're on
熊猫-将列转换为日期时间格式时包含不必要的值 - Pandas- Unnecessay values included while converting a column into datetime format
Pandas-拆分多索引数据帧 - Pandas- split a multiple index dataframe
更改数据框索引值，同时保持其他列数据相同 - Change dataframe index values while keeping other column data same
在Pandas DataFrame中添加行以保持索引顺序 - Adding row in Pandas DataFrame keeping index order
熊猫-将值设置为空数据框 - Pandas- set values to an empty dataframe
我如何从pandas数据框中的每一列获取最大（x）值，同时保持每个索引的索引？ - How can I get the max (x) number of values from each column in a pandas dataframe while keeping the index for each?
相关标签