Analysing a nominal and scale variable
Part 3b: Post-hoc test
The one-way ANOVA result on the previous page showed us that the location has a significant effect on the grade students gave the course. The next thing to do would then be to find out which locations are significantly different from each other. We can guess from the split histogram, but there is a proper way for doing this. A post-hoc analysis that will compare each possible pair of locations.
Since we are doing multiple tests, we have a danger of making a wrong decision of 5% each time. Although this seems low, it can quickly compound to making at least one wrong decision. Therefor the regular significance for each pairwise test gets adjusted. There are various methods to do this adjustment, and they are divided into two camps: those if the variance is the same in the population, and those for if the variances are not the same. Before we can go to testing the means, we therefor first need to test if the variances in the population could be the same. One possible test for this is the Levene F-test (Levene, 1960).
Click here to see how to perform a Levene F-test, with SPSS, R (Studio), Excel, or Python
with SPSS
with R (Studio)
with Excel
with Python
The significance in the example of the Levene F-test is .014. This indicates there is a 1.4% chance of having a Levene Statistic of 4.683 or even higher in a sample, if in the population the variances would be equal. This is such a low chance (below 5%) that we will therefor assume that the variances in the population will not be equal (across the three locations).
If the variance is not assumed equal (as in the example), I would use the Games-Howell procedure (Games & Howell, 1976) for the post-hoc analyses, while I’d use the Bonferroni method in case they are assumed to be equal.
In the example I’ll go for the Games-Howell procedure.
Click here to see how to perform a post-hoc test (Bonferroni and Games-Howell), with SPSS, R (Studio), Excel, or Python.
with SPSS
equal variances Bonferroni
unequal variances Games-Howell
with R (Studio)
equal variances Bonferroni
unequal variances Games-Howell
with Excel
equal variances Bonferroni
unequal variances Games-Howell
with Python
The .001 at Diemen-Haarlem indicates that the chance of having a sample with a mean difference of 22.678 or even higher, if there would be no difference in the population, is almost 0. This is below the usual 0.050, so we can say that indeed the average grade given for the course, in Diemen is significantly different from the students in Haarlem. The same goes for Diemen vs Rotterdam, but Haarlem vs Rotterdam is not significantly different.
From the descriptive measurements we already saw that Diemen scored the highest average, so we could report something like the following:
The one-way ANOVA showed that Location had significant effect on how students evaluated the course, F(2, 45) = 8.04, p = .001. The Levene test showed that the variances across the three locations were significantly different, F(2, 45) = 4.68, p = .014. A Games-Howell post hoc analysis showed that there was a significant difference between Diemen and Haarlem, p = .001, and also between Diemen and Rotterdam, p = .010.
The last part we need to complete the report is an indication of how strong the effect actually is. This will be the topic on the next page.
Nominal vs Scale
Google adds