计算由第三个变量分组的两个变量之间的相关系数

Question

I have an excel spreadsheet with 3 columns. The first column is the id of a picture which groups the data together, and the 2nd and 3rd columns are the values I am trying to find a correlation coefficient for.

For example:

ID  Dat1 Dat2
130 4   4.3
130 7.5 5
130 6.6 6
180 5.6 
180 5   8.7
180 7.1 5

In that example, the data is grouped by the values in the 1st column and then they have separate data in the 2nd and 3rd columns. I'm not sure whether it would be easier to find the correlation coefficients for each grouping using excel or R.

I have tried the Data Analysis add-in in Excel but it won't work for 3 columns.

Thanks in advance!

The real data has hundreds of thousands of lines of data. This is just an example.

Answer 1

Solution using data.table

# install.packages("data.table")
library(data.table)
df <- data.table(df)
df[,cor(Dat1,Dat2),by="ID"]

Answer 2

You could try

library(dplyr)
df1 %>% 
   group_by(ID) %>% 
   summarise(Cor= cor(Dat1, Dat2, use= "na.or.complete"))
#   ID        Cor
#1 130  0.6407453
#2 180 -1.0000000

data

df1 <- structure(list(ID = c(130L, 130L, 130L, 180L, 180L, 180L),
Dat1 = c(4, 
7.5, 6.6, 5.6, 5, 7.1), Dat2 = c(4.3, 5, 6, NA, 8.7, 5)), .Names = c("ID", 
"Dat1", "Dat2"), class = "data.frame", row.names = c(NA, -6L))

Answer 3

Two base R solutions, using @akrun's data:

with(df1, by(cbind(Dat1, Dat2), ID, cor, use = "complete"))
# INDICES: 130
#           Dat1      Dat2
# Dat1 1.0000000 0.6407453
# Dat2 0.6407453 1.0000000
# ----------------------------------------------------------------------------------------------------------------------- 
# INDICES: 180
#      Dat1 Dat2
# Dat1    1   -1
# Dat2   -1    1

lapply(split(df1, df1$ID), function(x) cor(x$Dat1, x$Dat2, use = "complete"))
# $`130`
# [1] 0.6407453
# 
# $`180`
# [1] -1