Analysing a nominal and ordinal variable

Part 3c: Effect size

We found out that the nominal variable has an effect/influence on the ordinal. We saw however it is not that one location gave the teacher only ‘fully agrees’, and the other only ‘fully disagrees’. To indicate how strong the influence is, it is a good habit to also report a so-called effect size. Unfortunately for the Kruskal-Wallis test there is not a single agreed upon method of calculating this, however Epsilon square (ε²) (Kelley, 1935) seems to be a good choice (see King & Minium (2009), as cited in Tomczak & Tomczak, 2014).

An epsilon square of 0 would mean no differences (and no influence), while one of 1 would indicate a full dependency. Unfortunately there is no formal way to determine if 0.40 is high or low, and I have not been able to find any rule of thumbs for the interpretation. Since this is a squared variable, I would use the same rule of thumb as for a correlation coefficient, but then squaring the upper and lower bounds of each bin. This would give if we use from Rea & Parker (1992) their interpretation for r, the following:

0.00 < 0.01 - Negligible
0.01 < 0.04 - Weak
0.04 < 0.16 - Moderate
0.16 < 0.36 - Relatively strong
0.36 < 0.64 - Strong
0.64 < 1.00 - Very strong

In the example epsilon square was 0.402 which would indicate a strong effect. We could add this to our report:

A Kruskal-Wallis test showed that Location had a significant relatively strong effect on how motivated students were by the teacher, χ²(2, N = 54) = 21.33, p < .001, ε² = .40. A post-hoc test using Dunn's test with Bonferroni correction showed the significant differences between Diemen and Haarlem, p < .05, and between Diemen and Rotterdam, p < .001.

Click here to see how you can determine epsilon square, with SPSS, R (studio), Excel, Python, an Online calculator, or Manually.

with SPSS

with R (Studio)

with Excel

two videos, a short one for only using the adjusted for tied ranks Kruskal-Wallis H value, and a longer one showing also the unadjusted and with more details

short version

long version

with Python

Online calculator

Enter the requested information below:

Manually (formula and example)

Formula

The formula for epsilon square is:

$\epsilon_{KW}^2=H\times\frac{n+1}{n^2-1}$

Where H is the H-statistic, or the adjusted H-statistic, and n is the total sample size.

The formula can also be done using Pearson correlation coefficient:

$\epsilon_{KW}^2=R_{r,\bar{r}_i}^2$

In this formula $R_{r,\bar{r}_i}^2$ is the square of the Pearson correlation coefficient between the ranks and the average rank of the category the score belongs to.

Example

Note: different example than the one used in the rest of this section, but same as used in the example for the manual calculation of the test.

We are given scores on an ordinal scale from three categories:

$X_1=(5,4,5,3,2), X_2=(1,2,2,4), X_3=(1,3,5,2)$

In total we have 5 + 4 + 4 = 13 scores, so:

$n=13$

In the manual example of the test, we also determined:

$=\frac{3\times3169}{10\times347} =\frac{9507}{3470} \approx2.7398$

Now we can fill out the formula for epsilon square:

$\epsilon_{KW}^2 =H\times\frac{n+1}{n^2-1} =\frac{9507}{3470}\times\frac{13+1}{13^2-1} =\frac{9507}{3470}\times\frac{14}{169-1} =\frac{9507}{3470}\times\frac{14}{168}$

$=\frac{9507}{3470}\times\frac{1}{12} =\frac{9507\times1}{3470\times12} =\frac{3169\times3}{3470\times4\times3} =\frac{3169}{3470\times4} =\frac{3169}{13880} \approx0.2283$

We can now wrap things up and combine all the parts for a full report on this on the next page.

Besides epsilon-squared, also eta-squared is often mentioned as an effect size measure for the Kruskal-Wallis H test. However it seems that eta-squared is more biased than epsilon-squared (Lakens, 2015). Another alternative is Freeman’s theta.

Nominal vs. Ordinal

Google adds