[英]Filling missing dates by group
I have a data set that looks like this: 我有一个如下所示的数据集:
shop_id,item_id,time,value
150,1,2015-07-10,3
150,1,2015-07-11,5
150,1,2015-07-13,2
150,2,2015-07-10,15
150,2,2015-07-12,12
Within each group, defined by "shop_id and "item_id", there are missing dates. 在每个组中,由“shop_id和”item_id“定义,缺少日期。
I wish to expand this irregular the time series to a regular, with consecutive dates, within each group: 我希望将这个不规则的时间序列扩展到每个组中的常规连续日期:
shop_id,item_id,time,value
150,1,2015-07-10,3
150,1,2015-07-11,5
150,1,2015-07-12,0 # <~~ added
150,1,2015-07-13,2
150,2,2015-07-10,15
150,2,2015-07-11,0 # <~~ added
150,2,2015-07-12,12
For the dates which are added, the corresponding values should by zero. 对于添加的日期,相应的值应为零。 I've read very similar questions though (either using R or SQL coalescing), but most of the solutions I've seen doesn't involve GROUP BYs. 我读过非常相似的问题(使用R或SQL合并),但我见过的大多数解决方案都不涉及GROUP BY。
Basically I have access to the SQL database/I can export as CSV for manipulation preferably in C#. 基本上我可以访问SQL数据库/我可以导出为CSV,最好在C#中进行操作。 Was hoping to find C# libraries that can do such data manipulation but couldn't find any. 希望找到可以进行此类数据操作但无法找到的C#库。
Any advice or help is appreciated! 任何建议或帮助表示赞赏!
You can use data.table
from R
. 您可以使用R
data.table
。 Assuming that 'time' column is of 'Date' class, 假设'time'列是'Date'类,
library(data.table)#v1.9.5+
DT1 <- setDT(df1)[, list(time=seq(min(time), max(time), by ='day')),
by =.(shop_id, item_id)]
setkeyv(df1, names(df1)[1:3])[DT1][is.na(value), value:=0]
# shop_id item_id time value
#1: 150 1 2015-07-10 3
#2: 150 1 2015-07-11 5
#3: 150 1 2015-07-12 0
#4: 150 1 2015-07-13 2
#5: 150 2 2015-07-10 15
#6: 150 2 2015-07-11 0
#7: 150 2 2015-07-12 12
In the devel version, you can also do this without setting the 'key'. 在devel版本中,您也可以在不设置“密钥”的情况下执行此操作。 Instructions to install the devel version are here
安装devel版本的说明在here
df1[DT1, on =c('shop_id', 'item_id', 'time')][is.na(value), value:=0]
# shop_id item_id time value
#1: 150 1 2015-07-10 3
#2: 150 1 2015-07-11 5
#3: 150 1 2015-07-12 0
#4: 150 1 2015-07-13 2
#5: 150 2 2015-07-10 15
#6: 150 2 2015-07-11 0
#7: 150 2 2015-07-12 12
Or as @Arun suggested, a more efficient option would be 或者@Arun建议,一个更有效的选择
DT1[, value := 0L][df1, value := i.value, on = c('shop_id', 'item_id', 'time')]
DT1
This is a Sql based solution 这是一个基于Sql的解决方案
First you need a dates
table 首先你需要一个dates
表
Date table query. 日期表查询。 Note this will create a physical table in your database. 请注意,这将在数据库中创建物理表 。
;with cte as
(
select cast('2000-01-01' as datetime) as Dates -- Start date
union all
select dateadd(MM,1,Dates)
from cte
where Dates < '2099-12-01' -- End date
)
select *
INTO Date_table
from CTE
Then you need to left outer join
your table with Date_table
to get the missing dates. 然后你需要使用Date_table
将你的表left outer join
到left outer join
以获取缺少的日期。
SELECT A.shop_id,
A.item_id,
DT.dates,
Isnull(Y.value, 0)
FROM date_table DT
CROSS JOIN(SELECT DISTINCT shop_id,
item_id
FROM yourtable) A
LEFT OUTER JOIN yourtable Y
ON t.[time] = DT.dates
AND A.shop_id = Y.shop_id
AND A.item_id = Y.item_id
Here's a solution with fill_by_value
from padr
: 这里有一个解决方案fill_by_value
从padr
:
library(dplyr)
library(tidyr)
library(padr)
df %>%
mutate(time = as.Date(time)) %>%
group_by(item_id) %>%
pad() %>% # from padr
fill(shop_id) %>% # from tidyr
fill_by_value(value) # from padr
Result: 结果:
# A tibble: 7 x 4
# Groups: item_id [2]
shop_id item_id time value
<int> <int> <date> <dbl>
1 150 1 2015-07-10 3
2 150 1 2015-07-11 5
3 150 1 2015-07-12 0
4 150 1 2015-07-13 2
5 150 2 2015-07-10 15
6 150 2 2015-07-11 0
7 150 2 2015-07-12 12
Data: 数据:
df = read.table(text = "shop_id,item_id,time,value
150,1,2015-07-10,3
150,1,2015-07-11,5
150,1,2015-07-13,2
150,2,2015-07-10,15
150,2,2015-07-12,12", header = TRUE, sep = ",")
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.