Multiple paired ordinal variables
Post-hoc
The Friedman test shows if there might be differences in mean ranks between the various ordinal variables. If there are, we would often like to know which ones are then different. This is then known as a post-hoc test.
With post hoc tests we are going to compare each time one of the variables with another one. Since we are going to perform multiple tests, with each test we reject the assumption about the population if the significance is below .05. So with each test we take a risk of less than .05 that we are rejecting the assumption although we shouldn't. Now this .05 seems very low, but the more tests we do, the bigger the chance we make at least once the wrong decision. There are various techniques to control for this. The most common one used is probably the Bonferroni adjustment.
Besides the type of adjustment, there are different methods to perform the post-hoc test itself for a Friedman test. In the appendix below a bit more detail on these. I'd recommend to choose between Wilcoxon, Dunn, and Fisher LSD (see appendix below for argumentation for this). Which is up to you to decide upon.
For the example the Dunn test was used with Bonferroni adjustment. The results show that 'the teacher was able to answer questions about the course' scores significantly higher than, his/her stimulation to participate, his/her stimulation to use discussion boards, and his/her ability to motivate students. It also showed that there the teacher ability to motivate students scored significant lower than his/her competence. So we can add this to our report:
The Friedman test indicated that there are differences between the average ranks among the seven different questions about the teacher, χ2(6, N = 52) = 49.79, p < .001. A post-hoc Dunn test with Bonferroni adjustment showed that 'the teacher was able to answer questions about the course' scored significant higher than his/her stimulation to participate in online activities (z = 3.27, p = .023), his/her stimulation to use discussion boards (z = 3.25, p = .025), and also his/her ability to motivate students (z = -3.81, p = .003). It also showed that the teacher ability to motivate students scored significant lower than his/her competence (z = -3.56, p = .008).
Click here to see how to perform these tests with SPSS, R (Studio), Excel, or Python.
with SPSS
Dunn test
Wilcoxon test
Sign test
with R (Studio)
Dunn test
Wilcoxon test
Sign test
Fisher's LSD
Conover method
Nemenyi method
with Excel
Dunn test
to be uploaded
Wilcoxon test
to be uploaded
Sign test
to be uploaded
with Python (Dunn)
Jupyter Notebook from video available here.
The last addition would be the strength of the differences. This is the topic for the next page.
Appendix: More on all the different types of post-hoc tests (click to expand)
Unfortunately there are a few different approaches for a post-hoc analysis for a Friedman test. One approach to post-hoc test for a Friedman test is to only use the data of the two variables (the pair). Since all variables are ordinal, this would mean we simply perform the test described at two variables - paired, multiple times. The test used there was the Wilcoxon signed rank test, or as alternative the two-sample sign test. The first two possible post-hoc tests for a Friedman test are therefor a pairwise-Wilcoxon signed rank test, and the pairwise two-sample Sign test.
Another approach to the post-hoc test is to use the ranks as determined by the Friedman test. However various complications arise from this and with it different solutions. Conover uses a t-distribution, Nemenyi a Studentized range distribution, and Dunn a z-distribution.
Pereira, Afonso, and Medeiros (2015) compared most of the options mentioned in the appendix. They concluded that the pairwise sign test should probably never be used since it is extremely conservative, and has much lower power than the other tests. Then the Wilcoxon test with Bonferroni correction would be more conservative, but has lower power, while the others (Dunn, Nemenyi, and Fisher LSD) are more liberal but have larger power. Singh (2013, p. 72) recommends not to use the Nemenyi procedure.
3+ Ordinal variables
Google adds