One-Sample Binomial Test
Introduction
The one-sample binomial test, is a test that could be used to compare two proportions. It is therefor a suitable test when analysing a single binary variable.
Performing the Test
with Excel
Excel file from videos TS - Binomial (one-sample) (E).xlsm.
with stikpetE
without stikpetE
with Flowgorithm
A basic implementation for a one-sample binomial test is shown in the flowchart in figure 1
Figure 1
Flowgorithm for one-sample binomial test
It takes as input the frequency of one of the categories (k) and the sample size (n). This makes use of the binomial distribution cumulative density function.
Flowgorithm file: TS - Binomial (one-sample).fprg.
with Python
Jupyter Notebook from videos TS - Binomial Exact Test.ipynb.
with stikpetP
with other libraries
without libraries
or without using any libraries:
Basic code example:
# libraries needed
import pandas as pd
from scipy.stats import binom_test
# some data
myDf = pd.read_csv('../../Data/csv/StudentStatistics.csv', sep=';')
myCd = myDf['Gen_Gender'].value_counts()
# the test
binom_test(myCd.values[0], sum(myCd.values), 1/2, alternative='two-sided')
with R (Studio)
with stikpetR
Jupyter Notebook from video TS - Binomial (one-sample) (R).ipynb.
without stikpetR
R script from video TS - Binomial Exact Test.R.
Datafile used in video: StudentStatistics.sav
Basic code example:
#one sample binomial test
#Preparation
#Getting some data
#install.packages("foreign")
library(foreign)
myData <- read.spss("../Data Files/StudentStatistics.sav", to.data.frame = TRUE)
#Remove na's
myVar <- na.omit(myData$Gen_Gender)
#Determine number of successes
k <- sum(myVar==myVar[1])
#Determine total sample size
n <- length(myVar)
#Test if expected both groups to be equal
#Perform binomial test
binom.test(k,n)
#Or use binomial distribution directly
2*pbinom(k,n,.5)
with SPSS
using non-parametric tests
Datafile used in video: StudentStatistics.sav
using Legacy Dialogs
Datafile used in video: StudentStatistics.sav
using compare means
Datafile used in video: StudentStatistics.sav
Manually (Formula's)
A one-sample binomial test, is almost 'just' the same as using the binomial distribution.
Given a probability of success (p), which for the binomial test is the expected proportion in the population, the number of trials (n), which for the binomial test is the total sample size, and the number of successes (k), which for the binomial test is number of occurences in one of the categories.
The formula for the cumulative binomial distribution (F(k; n,p)) is:
\(F\left(k;n,p\right)=\sum_{i=0}^{\left\lfloor k\right\rfloor}\binom{n}{i}\times p^{i}\times\left(1-p\right)^{n-i}\)
If p = 0.5 the formula could be simplfied into:
\(F\left(k;n,0.5\right)=0.5^{n}\times\sum_{i=0}^{\left\lfloor k\right\rfloor}\binom{n}{i}\)
In the formula ⌊k⌋ is the 'floor' function. This gives the greatest integer (whole number) less than or equal to k. So for example ⌊2.8⌋ = 2, and ⌊-2.2⌋=-3.
\(\binom{n}{i}\) is the binomial coefficient, this can be calculated using:
\(\binom{n}{i}=\frac{n!}{i!\times\left(n-i\right)!}\)
In this formula the ! indicates the factorial operation:
\(n!=\prod_{i=1}^{n}i\), and 0! is defined as 0! = 1.
These formulas are discussed in more detail in the binomial distribution section
Interpreting the Result
The assumption about the population for this test (the null hypothesis) is that the proportion for one category is X. Where X can be any value between 0 and 1. Very often X is set to 0.5.
The test provides a p-value, which is the probability of a test statistic as from the sample, or even more extreme, if the assumption about the population would be true. If this p-value (significance) is below a pre-defined threshold (the significance level \(\alpha\) ), the assumption about the population is rejected. We then speak of a (statistically) significant result. The threshold is usually set at 0.05. Anything below is then considered low.
If the assumption is rejected, we conclude that the proportion the category will have in the population will be different than the one used in the test.
Note that if we do not reject the assumption, it does not mean we accept it, we simply state that there is insufficient evidence to reject it.
The test uses the cumulative density function (cdf) of the binomial distribution to determine the probability.
Lets for example assume we have a sample that showed that 34 of the respondents chose for male, and 12 for female. That is 74% male, and 26% female. We can use the one-sample binomial test, to test if in the population these two could be equal. The assumption about the population is then that the proportion of female is 0.5 (50%).
The cdf of the binomial distribution can inform us, that the chance of having 12 or less out of 46 females, if in the population the distribution is 50/50, is only 0.0008. This is the one-sided significance.
For the two-sided significance, we need to realise that we are looking for 'or more extreme' at either end. So having 12 or less female, is just as unlikely as having (46/2 - 12 + 46/2 = ) 34 or more females. If the assumption is that the proportion is 0.5, this will be the same probability. This means we can then simply double the one-sided probability, to get 0.0016 as the two-sided significance. In case the assumption is not .5, doubling the one-sided result will not work. See the section below on how to resolve this.
Writing the results
When reporting the results of a one-sample binomial test, only the p-value is needed, since there is no test statistic. This is usually reported with three decimal places, e.g. p = 0.002. If the p-value is below .00005, it is reported as p < .001.
Next step and Alternatives
APA (2019, p. 88) states to also report an effect size measure. If the assumption was 0.5 one possible option could be Cohen g, otherwise Cohen h2 or the Alternative Ratio could also be used.
Alternative tests could be a one-sample Wald, or a one-sample score test. These are discussed at the one-sample proportion tests page.
Appendix
What if the expected proportion is not 0.50
If in the example we’d expected 30% to be Female, our assumption about the population would change from 0.5 to 0.3. There are two methods we can then use to interpret ‘or more extreme’. The method of equal distance, and the method of small p-values.
The method of equal distance
This method looks at the number of cases. In a sample of 46 people, we’d then expect 46 x 0.3 = 13.8 Female respondents. We only had 12, so a difference of 13.8 – 12 = 1.8. The ‘equal distance method’ now means to look for the chance of having 12 Female or less, and 13.8 + 1.8 = 15.6 Female or more. Each of these two probabilities can be found using a binomial distribution. The ’12 or less’ probability is 0.3448, and the ’15.6 or more’ probability is 0.4031 (note that the 15.6 is always rounded down). Adding these two together than gives the two-sided significance of 0.7479.
The method of small p-values
This method looks at the probabilities itself. The probability of having exactly 12 Female scores out of a group of 46, if the chance of Female is 0.3, is 0.1119 (this is again a binomial distribution). The method of small p-values now considers ‘or more extreme’ any number between 0 and 49 (the sample size) that has a probability less or equal to the 0.1119. This means we need to go over each option, determine the probability and check if it is lower or equal. So, the probability of 0 Female, the probability of 1 Female, etc. In the example all counts of 12 or less, and all of 16 or more, each have a probability of 0.1119 or less. In total this results in a two-sided significance of 0.6300.
Google adds