How can I compute a transition table in R using data.table?

Question

I was wondering how I can compute a transition table for when new ID's are introduced (+1) and ID's are removed (-1).

For example:

library(data.table)

dt <- data.table(id = c(1,2,3,1,2,3,4,5,1,2,4,5),
           year = c(2015, 2015, 2015, 2016, 2016, 2016, 2016, 2016, 2017, 2017, 2017, 2017))

ids <- unique(dt$id)

So for the first year (2016), it should be:

0, 0, 0, 1, 1

and the second year (2017) it should be:

0, 0, -1, 0, 0

Answer 1

Here is an alternative approach which uses a cross join and aggregates in a join . It is essentially equivalent to the dcast() and table() solutions above but keeps the data in long form:

dt[CJ(year = year, id = id, unique = TRUE), on = .(id, year), .N, by = .EACHI][
  , change := N - shift(N), by = id][]

  id year N change 1: 1 2015 1 NA 2: 2 2015 1 NA 3: 3 2015 1 NA 4: 4 2015 0 NA 5: 5 2015 0 NA 6: 1 2016 1 0 7: 2 2016 1 0 8: 3 2016 1 0 9: 4 2016 1 1 10: 5 2016 1 1 11: 1 2017 1 0 12: 2 2017 1 0 13: 3 2017 0 -1 14: 4 2017 1 0 15: 5 2017 1 0