data.table合并的“模糊密钥匹配”

Question

I'm trying to match workers from year to year using name strings and a measure of experience. Experience can only increase by at most one from year to year, so I'd like to use this to help matching when other metrics fail.

For example:

dt1<-data.table(name=c("jane doe","jane doe",
                       "john doe","jane smith"),
                exp=c(0.,5,1,2),id=1:4,key="name")
dt2<-data.table(name=c("jane doe","jane doe",
                       "john doe","jane smith"),
                exp=c(0,30,1.5,2),key="name")

I want to match the first "jane doe" in dt1 to the first "jane doe" in dt2 . The latter "jane doe"s don't match, because they're clearly different people (based on vastly different experience levels).

I'd also like to add some flags to know I matched these people in this way later on down the line. Here's my first pass:

dt2[dt1,`:=`(id=ifelse(exp<=i.exp+1,i.id,NA),
             flag=ifelse(exp<=i.exp+1,i.id,NA))]

But this is not working--here's what that gives me:

> dt2
         name  exp id flag
1:   jane doe  0.0  2    2
2:   jane doe 30.0 NA   NA
3: jane smith  2.0  4    4
4:   john doe  1.5  3    3

It seems properly to have missed matching the latter "jane doe", but appears to have matched the first "jane doe" to the wrong prior "jane doe". I'm not quite sure why this is; anyway, it seems preferable to have a way to incorporate the matching on exp before instead of after joining--this would also clean up the ifelse mess in defining the new variables. Any suggestions?

---

For clarity, here's the desired output:

> dt2
         name  exp id flag
1:   jane doe  1.0  1    1
2:   jane doe 30.0 NA   NA
3: jane smith  2.0  4    1
4:   john doe  1.5  3    1

Answer 1

In your case the join isn't really "fuzzy". All you trying to do is to join by name by exp while allowing one 1 year distance per match. This is good use for a rolling join with a -1L specification.

First we will correctly key the data sets

setkey(dt1, name, exp) 
setkey(dt2, name, exp) 

Then, we will perform the rolling join while passing -1L as its value

dt2[dt1, `:=`(id = i.id, flag = 1L), roll = -1L]
df2
#          name  exp id flag
# 1:   jane doe  0.0  1    1
# 2:   jane doe 30.0 NA   NA
# 3: jane smith  2.0  4    1
# 4:   john doe  1.5  3    1

---

In future, if you''ll need to conduct an interval join such as c(1L, -1L) you can take a look here for some examples of the foverlaps function.