R - Repeated measure analysis - Different results for LME and Tukey post hoc test

Question

I'm currently running a repeated measure analysis in R on 4 sub-factors: SF1, SF2, SF3, SF4

First, it is to be noted that the assumption of sphericity is violated, sample size is conisdered as reasonable large (N = 188). Group size is however not equal.

Contrasts are set to show that SF1 and SF2(combined) are signifcantly higher than SF3 and SF4(combined). Whereas the values for SF1 and SF2(between) and SF3 and SF4(between) do not differ significantly. Ie

Contr1<-c(1, 1, -1, -1)
Contr2<-c(1, -1, 0, 0)
Contr3<-c(0, 0, 1, -1)
contrasts(rep_table_long$Subfactor)<-cbind(Contr1, Contr2, Contr3)

The general model code is the following

rep_model <- lme(Value ~ Subfactor, random = ~1|Subject/Subfactor, data = rep_table_long, method ="ML")

By executing summary(rep_model) I received the following (truncated) output Fixed effects: Value ~ Subfactor Value Std.Error DF t-value p-value (Intercept) 5.498910 0.07229032 561 76.06703 0.0000 SubfactorContr1 0.459601 0.03066438 561 14.98811 0.0000 SubfactorContr2 0.085266 0.04336598 561 1.96619 0.0498 SubfactorContr3 0.093617 0.04336598 561 2.15877 0.0313

Thus, showing SF1&SF2 are significantly larger than SF3&SF4. But SF1 is also significantly larger than SF2 and so is SF3 > SF4.

However, and here comes the reason for my question, the post hoc Tukey test showed different results:

> postHocs <- glht (rep_model, linfct = mcp(Subfactor = "Tukey"))
> summary(postHocs)

Simultaneous Tests for General Linear Hypotheses

Multiple Comparisons of Means: Tukey Contrasts

Fit: lme.formula(fixed = Value ~ Subfactor, data = rep_table_long, random = ~1 | Subject/Subfactor, method = "ML")

Linear Hypotheses:
              Estimate Std. Error  z value   Pr(>|z|)    
SF2- SF1 == 0   -0.1872     0.0865  -2.165    0.133    
SF3- SF1 == 0   -0.9275     0.0865 -10.723   <0.001 
SF4- SF1 == 0   -1.0981     0.0865 -12.694   <0.001 
SF3- SF2 == 0   -0.7403     0.0865  -8.559   <0.001 
SF4- SF2 == 0   -0.9109     0.0865 -10.530   <0.001
SF4- SF3 == 0   -0.1705     0.0865  -1.971    0.199`

The result of the post hoc Tukey tests shows that the difference between SF2 and SF1 as well as between SF4 and SF3 are not significantly different.

Why do I get different results in both tests? Is it because sphericity is violated? Or am I doing something wrong here?

Any help is very appreciated.

Answer 1

I read about that in Andy Field´s book. The reason why you get different results from planned contarsts vs. a post hoc test is that post hoc tests are two-tailed and hence are suitable for explorative analyses (= no hypotheses). Contrasts, however, are one-tailed. When you think about t-tests it makes a difference whether you conduct a two-tailed test instead of a one-tailed test. Also, post-hoc tests like Tukey are conservative (they lack statistical power). These might be the reason why you did not find any significane with tukey.

Also, only use Tukey if your sample sizes are equal and you are confident that your population variances are similiar. (And Tukey is better with larger number of means but this is the case).

Hope this helped.

Reference: Field, A., Miles, J., & Field, Z. (2012). Discovering statistics using R. Sage publications.