[英]How can I create a mapping between two dataframes to calculate spearman's correlation?
[英]How to calculate Spearman's rank correlation between two datasets
如果我们有:
X = pd.DataFrame({"A":[34,12,78,84,26], "B":[54,87,35,25,82], "C":[56,78,0,14,13], "D":[0,23,72,56,14], "E":[78,12,31,0,34]})
Y = pd.DataFrame({"A":[45,24,65,65,65], "B":[45,87,65,52,12], "C":[98,52,32,32,12], "D":[0,23,1,365,53], "E":[24,12,65,3,65]})
我们如何计算两个数据集之间的Spearman秩相关性 (但不在每个数据集中),以便最终我们有一个5x5矩阵? 像这样:
A B C D E
A . . . . .
B . . . . .
C . . . . .
D . . . . .
E . . . . .
使用pandas的concat
和corr
函数,您可以通过将所有内容放在一个DataFrame
转换为DataFrame
:
import pandas as pd
X = pd.DataFrame({"A":[34,12,78,84,26], "B":[54,87,35,25,82], "C":[56,78,0,14,13], "D":[0,23,72,56,14], "E":[78,12,31,0,34]})
Y = pd.DataFrame({"A1":[45,24,65,65,65], "B1":[45,87,65,52,12], "C1":[98,52,32,32,12], "D1":[0,23,1,365,53], "E1":[24,12,65,3,65]})
pd.concat([X,Y], axis=1).corr(method="spearman").iloc[5:,:5]
请注意,在我的示例中,我为第二组列赋予了不同的名称,以使它们更容易区分。 使用pandas的索引功能,您可以提出一种更复杂的方法,从相关表中挑选出比我的.iloc[5:,:5]
所需的行/列,但在这种情况下它可以工作。
编辑添加结果:
这应该做的伎俩! 可能会缩短,但可能会缩短:
import pandas as pd
import numpy as np
from scipy.stats import linregress
X = pd.DataFrame({"A":[34,12,78,84,26], "B":[54,87,35,25,82], "C":[56,78,0,14,13], "D":[0,23,72,56,14], "E":[78,12,31,0,34]})
Y = pd.DataFrame({"A":[45,24,65,65,65], "B":[45,87,65,52,12], "C":[98,52,32,32,12], "D":[0,23,1,365,53], "E":[24,12,65,3,65]})
row = 0
col = 0
m = np.zeros( (len(X), len(Y) ))
for key_x, val_x in X.iteritems():
for key_y, val_y in Y.iteritems():
if( col == 5 ):
col = 0
m[row][col] = linregress(val_x, val_y).rvalue
col += 1
row += 1
print m
为了计算相关性,我正在使用linregress,但还有其他替代方案,例如:
numpy.corrcoef
pandas.DataFrame.corr
也许还有其他一些;)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.