如何根据R中另一列的条件过滤列

Question

I have a huge data table with millions of rows and dozens columns, so performance is a crucial issue for me. The data describes visits to a content site. I want to compute the ContentId of the earliest (ie minimum hit time) hit of each visit. What I did is: dt[,.(FirstContentOfVisit=ContentID[ContentID != ""][which.min(HitTime)]), by=VisitId,.SDcols=c("ContentID","HitTime")]

the problem is that I don't know if which.min first computes the min on all the HitTime vector (which I don't want!) or does it only on the filtered HitTime vector (the one which is corresponding to the non-empty ContentID).

In addition, after I compute it - how can I get the minimal HitTime of the ContentIDs that are different from the first (ie the earliest hit time of the non-first content id).

When I tried to have both actions with user-defined functions (first - sort the sub data table and then extract the desired value) it took ages (and actually never stopped), although I have a very strong machine (virtual) with 180 GB RAM. So I'm looking for an inline solution.

Answer 1

dplyr makes this much easier. You didn't share a sample of your data, but I assume the variables of interest look something like this.

web <- tibble(
  HitTime = sample(seq(as.Date('2010/01/01'), as.Date('2017/02/23'), by="day"), 1000),
  ContentID = 1:1000,
  SessionID = sample(1:100, 1000, replace = TRUE)
)

Then you can just use group_by and summarise to find the earliest value of HitTime for each SessionID .

web %>%
  group_by(SessionID) %>%
  summarise(HitTime = min(HitTime))