为什么预计算列最大值的 dplyr 代码比在 mutate 调用中计算它的 dplyr 代码慢？

Question

Sample data frame:

ngroups <- 100
nsamples <- 1000
foo <- data.frame(engine = rep(seq(1, ngroups), each = nsamples), cycles = runif(ngroups*nsamples, 0, nsamples))

I want to find the max of cycles for each engine group, and create a new variable tte = max(cycles) - cycles with mutate . I thought that if I would precompute the column of maximum cycles, instead than recomputing it inside the mutate command for each row, the code would be faster. Turns out I'm wrong:

library(microbenchmark)
library(dplyr)
library(magrittr)

add_tte <- function(dataset){
  dataset %<>% group_by(engine) %>% mutate(max_cycles = max(cycles)) %>% 
    mutate(tte = max_cycles - cycles) %>% select(-max_cycles) %>% ungroup
}

add_tte_old <- function(dataset){
  dataset %<>% group_by(engine) %>% mutate(tte = max(cycles) - cycles) %>% ungroup
}

microbenchmark(add_tte(foo), add_tte_old(foo), times = 500)
# Unit: milliseconds
# expr      min        lq     mean   median       uq       max neval
# add_tte(foo) 17.45324 21.107264 26.50535 24.52625 28.75208 113.98433   500
# add_tte_old(foo)  8.10376  9.949188 13.35830 12.18336 14.52474  77.64578   500

Why is this happening? Is the reason that dplyr computes the maximum just once for group, instead that once for row?

EDIT : even if I use a single mutate statement in add_tte , and I create a bigger example, add_tte_old is still faster

# these are the only lines of code modified, the rest is as before
nsamples <- 10000

foo <- data.frame(engine = rep(seq(1, ngroups), each = nsamples), cycles = runif(ngroups*nsamples, 0, nsamples))

add_tte <- function(dataset){
  dataset %<>% group_by(engine) %>% mutate(max_cycles = max(cycles), tte = max_cycles - cycles) %>%
  select(-max_cycles) %>% ungroup
}

# the new results are:
microbenchmark(add_tte(foo), add_tte_old(foo), times = 500)
# Unit: milliseconds
# expr      min        lq      mean    median        uq      max neval
# add_tte(foo) 90.46658 107.14015 139.13570 131.83689 158.24358 411.3272   500
# add_tte_old(foo) 39.38357  46.13531  62.57386  52.00782  69.26815 176.1512   500

Answer 1

You have made some wrong assumptions, but besides that, more importantly, you are not comparing like-wise.

It would make more sense to look at the two variants below:

add_tte <- function(dataset) {
  dataset %<>% group_by(engine) %>% mutate(max_cycles = rep(max(cycles), times = n()), tte = max_cycles - cycles) %>%
    select(-max_cycles) %>% ungroup
}

add_tte_old <- function(dataset) {
  dataset %<>% group_by(engine) %>% mutate(extra = rep(1, times = n()), tte = max(cycles) - cycles) %>%
    select(-extra) %>% ungroup
}

microbenchmark(add_tte(foo), add_tte_old(foo), times = 100)

On my machine, these two are pretty similar.

It is kind of ironic that with your way of attempting to pre-compute the max(cycles) , you probably did what you were trying to avoid :)

In the case here, you should really use the explicit rep() to fill up the column, whereas in the subtraction max(cycles) - cycles the auto-recycling is alright.