dplyr 包的inner_join 输出不正确

Question

I have two datasets, named "results" and "support2", available here .

I want to merge the two datasets by the only common column name "SNP". Code below:

> library(dplyr)
> results <- read_delim("<path>\\results", delim = "\t", col_name = T)
> support2 <- read_delim("<path>\\support2", delim = "\t", col_name = T)

> head(results)
# A tibble: 6 x 2
  SNP        p.value
  <chr>        <dbl>
1 rs28436661   0.334
2 rs9922067    0.322
3 rs2562132    0.848
4 rs3930588    0.332
5 rs2562137    0.323
6 rs3848343    0.363

 > head(support2)
# A tibble: 6 x 2
  SNP         position
  <chr>          <dbl>
1 rs62028702     60054
2 rs190434815    60085
3 rs62028703     60087
4 rs62028704     60095
5 rs181534180    60164
6 rs186233776    60177

> dim(results)
[1] 188242      2
> dim(support2)
[1] 1210619       2

# determine the number of common SNPs
length(Reduce(intersect, list(results$SNP, support2$SNP)))
[1] 187613

I would expect that after inner_join, the new data would have 187613 rows.

> newdata <- inner_join(results, support2)
Joining, by = "SNP"
> dim(newdata)
[1] 1409812       3

Strangely, instead of have 187613 rows, the new data have 1409812 rows, which is even larger than the sum of the number of rows of the two dataframes.

I switched to the merge function as below:

> newdata2 <- merge(results, support2)
> dim(newdata2)
[1] 1409812       3

This second new dataframe has the same issue. No idea why.

I wish to know how should I obtain a new dataframe whose rows represent the common rows of the two dataframes (should have 187613 rows) and whose columns contain columns of both dataframes.

Answer 1

It could be a result of duplicate elements

results <- data.frame(col1 = rep(letters[1:3], each = 3), col2 = rnorm(9))
support2 <- data.frame(col1 = rep(letters[1:5],each = 2), newcol = runif(10))

library(dplyr)
out <- inner_join(results, support2)
nrow(out)
#[1] 18

Here, the initial datasets in the common column ('col1') are duplicated which confuses the join statement as to which row it should take as a match resulting in a situation similar to a cross join but not exactly that

Answer 2

As already pointed out by @akrun, the data may have duplicates, possibly that is the only explanation of this behavior.

From the documentation of intersect, it always returns a unique value but inner join can have duplicates if the "by" value has duplicates, Hence the count mismatch.

If you truly want to see its right, see the unique counts of by variable (unique key in your case), it should match with your intersect result. But that doesn't mean your join/merge is right, ideally any join which has duplicates in both table A and B is not recommended(unless offcourse you have business/other justification). So, check if the duplicates are present in both the tables or only one of them. If it only found in one of the tables then probably your merge/join should be alright. I hope I am able to explain the scenario.

Please let me know if it doesn't answer your question, I shall remove it.

From Documentations:

intersect:

Each of union, intersect, setdiff and setequal will discard any duplicated values in the arguments, and they apply as.vector to their arguments

inner_join():

return all rows from x where there are matching values in y, and all columns from x and y. If there are multiple matches between x and y, all combination of the matches are returned.