I read this post today by John Quick. I was a little taken back when he used a pairwise t-test for post hoc analysis. In a contradiction the t-test did not show differences in the treatment means when the ANOVA model did. This is because the pairwise.t.test does not take into account the two-way anova, it only looks marginally, and so gives erroneous results. The more appropriate analysis should be TukeyHSD applied to the fitted model.
> model1<-aov(StressReduction~Treatment+Age, data ) > TukeyHSD(model1, "Treatment") Tukey multiple comparisons of means 95% family-wise confidence level Fit: aov(formula = StressReduction ~ Treatment + Age, data = data) $Treatment diff lwr upr p adj mental-medical 2 0.92885267 3.07114733 0.0003172 physical-medical 1 -0.07114733 2.07114733 0.0702309 physical-mental -1 -2.07114733 0.07114733 0.0702309
Here I already had the data read in as data, then I fit the model and applied a post-hoc pairwise test. This yielded that the mental and medical are different, but no other treatments. This is shown by the plot of the data.