熊猫数据框上的成对检验统计显着性

Question

I have a pandas dataframe (100x10), where each column represents some quantity and I would like to pair-wise test all columns using t-test. Instead of looping over the columns:

stats.ttest_rel(df.iloc[:,i], df.iloc[:,j])

where i!=j , is there a cleaner way to do it? Something similar to correlations:

df.corr()

where it computes all pair-wise correlations.

Answer 1

I think there is no direct method to create pair-wise t-test, you can try this one

from scipy.stats import ttest_ind
import pandas as pd
import csv
df=pd.read_csv('input.csv')


fo = open('result.csv','wb+')
outfile = csv.writer(fo, delimiter=',')
outfile.writerow((df.columns).insert(0,''))

for i in df.columns:
    t=[]
    for j in df.columns:

        t.append(ttest_ind(df[i], df[j]))
    (t).insert(0,i)
    outfile.writerow(t)

this script will give you output file as result.csv that tells your pair-wise t-test calculation

Answer 2

No need to do a double for-loop yourself. You can use itertools.combinations

results = pd.DataFrame(columns=df.columns, index=df.columns)
for (label1, column1), (label2, column2) in itertools.combinations(df.items(), 2):
    results.loc[label1, label2] = results.loc[label2, label1] = stats.ttest_rel(column1, column2)