Tukey Test

An ANOVA test is used to find out if there is a significant difference between three or more group means. However, the ANOVA analysis simply indicates there is a difference between two or more group means, but it does not tell you which means there is a significant difference between. For example, in the table below there are five groups listed. An ANOVA indicated there is a significant difference! Is it between mean I & II; means II & IV? Means III and IV? In order to find out between which means there is a significant difference, a post hoc test needs to be done. The Tukey Test is a post hoc test designed to perform a pairwise comparison of the means to see where a significant difference lies!

The first step in doing a Tukey Test is to arrange the means in ascending order in a comparison table and to calculate the difference between each pair of means.

 Means in Ascending Order Grp 4 Grp 5 Grp 3 Grp 2 Grp 1 38.7 56.8 63.4 63.6 64.8 Grp 4 38.7 ---- 18.0 24.6 24.8 26.1 Grp 5 56.8 ---- ---- 6.6 6.8 8.0 Grp 3 63.4 ---- ---- ---- 0.2 1.4 Grp 2 63.6 ---- ---- ---- ---- 1.2 Grp 1 64.8 ---- ---- ---- ----- ----

The next step is to calculate the minimum pairwise difference needed. Use the following formula.

Dmin= Qt (sqrt(MSwithin group))/(sqrt(n))

where

Dmin is the minimum pairwise difference needed for significance

MSwithin group is the within group Mean Square Error value found in the ANOVA output

n is the sample size of each group (number of data points in each group in the data set or, if the groups have different numbers of data points, use the smallest group number for a conservative analysis)

Qt is the t statistic you will need. It requires the following:

1) the df WITHIN group (total number of data points minus the number of groups),
2) the number of means being tested (number of groups), and
3) the alpha level (most often set at .05)

Example

Dmin= Qt (sq rt(MSwithin group))/(sq rt(n))

Dmin= 3.92(sq rt(219.28))/(sqrt(25))

Dmin= 3.92(14.81)/(5)

Dmin= 11.61

The final step is to compare the difference between the means in the table you constructed to the minimum pairwise difference. The ones that are larger than the minimum are the means pairs that are significantly different. In this example, group 4 is statistically significantly different than every other group.
NOTE: The Tukey test is a weaker statistical test than the ANOVA. What this means is that an ANOVA might show a statistically significant difference with a p-value relatively close to the alpha, but the Tukey difference table might not have any differences which are greater than the minimum difference (Dmin). When this happens, the statistically significant difference lies between the groups with the greatest difference between their means, even though that difference is less than the Dmin.

Copyright © 1997 Central Virginia Governor's School for Science and Technology Lynchburg, VA