Sign test - overview

This page offers structured overviews of one or more selected methods. Add additional methods for comparisons (max. of 3) by clicking on the dropdown button in the right-hand column. To practice with a specific method click the button at the bottom row of the table

Sign test
Binomial test for a single proportion
McNemar's test
McNemar's test
Independent variableIndependent variableIndependent variableIndependent variable
2 paired groupsNone2 paired groups2 paired groups
Dependent variableDependent variableDependent variableDependent variable
One of ordinal levelOne categorical with 2 independent groupsOne categorical with 2 independent groupsOne categorical with 2 independent groups
Null hypothesisNull hypothesisNull hypothesisNull hypothesis
  • H0: P(first score of a pair exceeds second score of a pair) = P(second score of a pair exceeds first score of a pair)
If the dependent variable is measured on a continuous scale, this can also be formulated as:
  • H0: the population median of the difference scores is equal to zero
A difference score is the difference between the first score of a pair and the second score of a pair.
H0: $\pi = \pi_0$

Here $\pi$ is the population proportion of 'successes', and $\pi_0$ is the population proportion of successes according to the null hypothesis.

Let's say that the scores on the dependent variable are scored 0 and 1. Then for each pair of scores, the data allow four options:

  1. First score of pair is 0, second score of pair is 0
  2. First score of pair is 0, second score of pair is 1 (switched)
  3. First score of pair is 1, second score of pair is 0 (switched)
  4. First score of pair is 1, second score of pair is 1
The null hypothesis H0 is that for each pair of scores, P(first score of pair is 0 while second score of pair is 1) = P(first score of pair is 1 while second score of pair is 0). That is, the probability that a pair of scores switches from 0 to 1 is the same as the probability that a pair of scores switches from 1 to 0.

Other formulations of the null hypothesis are:

  • H0: $\pi_1 = \pi_2$, where $\pi_1$ is the population proportion of ones for the first paired group and $\pi_2$ is the population proportion of ones for the second paired group
  • H0: for each pair of scores, P(first score of pair is 1) = P(second score of pair is 1)

Let's say that the scores on the dependent variable are scored 0 and 1. Then for each pair of scores, the data allow four options:

  1. First score of pair is 0, second score of pair is 0
  2. First score of pair is 0, second score of pair is 1 (switched)
  3. First score of pair is 1, second score of pair is 0 (switched)
  4. First score of pair is 1, second score of pair is 1
The null hypothesis H0 is that for each pair of scores, P(first score of pair is 0 while second score of pair is 1) = P(first score of pair is 1 while second score of pair is 0). That is, the probability that a pair of scores switches from 0 to 1 is the same as the probability that a pair of scores switches from 1 to 0.

Other formulations of the null hypothesis are:

  • H0: $\pi_1 = \pi_2$, where $\pi_1$ is the population proportion of ones for the first paired group and $\pi_2$ is the population proportion of ones for the second paired group
  • H0: for each pair of scores, P(first score of pair is 1) = P(second score of pair is 1)

Alternative hypothesisAlternative hypothesisAlternative hypothesisAlternative hypothesis
  • H1 two sided: P(first score of a pair exceeds second score of a pair) $\neq$ P(second score of a pair exceeds first score of a pair)
  • H1 right sided: P(first score of a pair exceeds second score of a pair) > P(second score of a pair exceeds first score of a pair)
  • H1 left sided: P(first score of a pair exceeds second score of a pair) < P(second score of a pair exceeds first score of a pair)
If the dependent variable is measured on a continuous scale, this can also be formulated as:
  • H1 two sided: the population median of the difference scores is different from zero
  • H1 right sided: the population median of the difference scores is larger than zero
  • H1 left sided: the population median of the difference scores is smaller than zero
H1 two sided: $\pi \neq \pi_0$
H1 right sided: $\pi > \pi_0$
H1 left sided: $\pi < \pi_0$

The alternative hypothesis H1 is that for each pair of scores, P(first score of pair is 0 while second score of pair is 1) $\neq$ P(first score of pair is 1 while second score of pair is 0). That is, the probability that a pair of scores switches from 0 to 1 is not the same as the probability that a pair of scores switches from 1 to 0.

Other formulations of the alternative hypothesis are:

  • H1: $\pi_1 \neq \pi_2$
  • H1: for each pair of scores, P(first score of pair is 1) $\neq$ P(second score of pair is 1)

The alternative hypothesis H1 is that for each pair of scores, P(first score of pair is 0 while second score of pair is 1) $\neq$ P(first score of pair is 1 while second score of pair is 0). That is, the probability that a pair of scores switches from 0 to 1 is not the same as the probability that a pair of scores switches from 1 to 0.

Other formulations of the alternative hypothesis are:

  • H1: $\pi_1 \neq \pi_2$
  • H1: for each pair of scores, P(first score of pair is 1) $\neq$ P(second score of pair is 1)

AssumptionsAssumptionsAssumptionsAssumptions
  • Sample of pairs is a simple random sample from the population of pairs. That is, pairs are independent of one another
  • Sample is a simple random sample from the population. That is, observations are independent of one another
  • Sample of pairs is a simple random sample from the population of pairs. That is, pairs are independent of one another
  • Sample of pairs is a simple random sample from the population of pairs. That is, pairs are independent of one another
Test statisticTest statisticTest statisticTest statistic
$W = $ number of difference scores that is larger than 0$X$ = number of successes in the sample$X^2 = \dfrac{(b - c)^2}{b + c}$
Here $b$ is the number of pairs in the sample for which the first score is 0 while the second score is 1, and $c$ is the number of pairs in the sample for which the first score is 1 while the second score is 0.
$X^2 = \dfrac{(b - c)^2}{b + c}$
Here $b$ is the number of pairs in the sample for which the first score is 0 while the second score is 1, and $c$ is the number of pairs in the sample for which the first score is 1 while the second score is 0.
Sampling distribution of $W$ if H0 were trueSampling distribution of $X$ if H0 were trueSampling distribution of $X^2$ if H0 were trueSampling distribution of $X^2$ if H0 were true
The exact distribution of $W$ under the null hypothesis is the Binomial($n$, $P$) distribution, with $n =$ number of positive differences $+$ number of negative differences, and $P = 0.5$.

If $n$ is large, $W$ is approximately normally distributed under the null hypothesis, with mean $nP = n \times 0.5$ and standard deviation $\sqrt{nP(1-P)} = \sqrt{n \times 0.5(1 - 0.5)}$. Hence, if $n$ is large, the standardized test statistic $$z = \frac{W - n \times 0.5}{\sqrt{n \times 0.5(1 - 0.5)}}$$ follows approximately the standard normal distribution if the null hypothesis were true.
Binomial($n$, $P$) distribution.

Here $n = N$ (total sample size), and $P = \pi_0$ (population proportion according to the null hypothesis).

If $b + c$ is large enough (say, > 20), approximately the chi-squared distribution with 1 degree of freedom.

If $b + c$ is small, the Binomial($n$, $P$) distribution should be used, with $n = b + c$ and $P = 0.5$. In that case the test statistic becomes equal to $b$.

If $b + c$ is large enough (say, > 20), approximately the chi-squared distribution with 1 degree of freedom.

If $b + c$ is small, the Binomial($n$, $P$) distribution should be used, with $n = b + c$ and $P = 0.5$. In that case the test statistic becomes equal to $b$.

Significant?Significant?Significant?Significant?
If $n$ is small, the table for the binomial distribution should be used:
Two sided:
  • Check if $W$ observed in sample is in the rejection region or
  • Find two sided $p$ value corresponding to observed $W$ and check if it is equal to or smaller than $\alpha$
Right sided:
  • Check if $W$ observed in sample is in the rejection region or
  • Find right sided $p$ value corresponding to observed $W$ and check if it is equal to or smaller than $\alpha$
Left sided:
  • Check if $W$ observed in sample is in the rejection region or
  • Find left sided $p$ value corresponding to observed $W$ and check if it is equal to or smaller than $\alpha$

If $n$ is large, the table for standard normal probabilities can be used:
Two sided: Right sided: Left sided:
Two sided:
  • Check if $X$ observed in sample is in the rejection region or
  • Find two sided $p$ value corresponding to observed $X$ and check if it is equal to or smaller than $\alpha$
Right sided:
  • Check if $X$ observed in sample is in the rejection region or
  • Find right sided $p$ value corresponding to observed $X$ and check if it is equal to or smaller than $\alpha$
Left sided:
  • Check if $X$ observed in sample is in the rejection region or
  • Find left sided $p$ value corresponding to observed $X$ and check if it is equal to or smaller than $\alpha$
For test statistic $X^2$:
  • Check if $X^2$ observed in sample is equal to or larger than critical value $X^{2*}$ or
  • Find $p$ value corresponding to observed $X^2$ and check if it is equal to or smaller than $\alpha$
If $b + c$ is small, the table for the binomial distribution should be used, with as test statistic $b$:
  • Check if $b$ observed in sample is in the rejection region or
  • Find two sided $p$ value corresponding to observed $b$ and check if it is equal to or smaller than $\alpha$
For test statistic $X^2$:
  • Check if $X^2$ observed in sample is equal to or larger than critical value $X^{2*}$ or
  • Find $p$ value corresponding to observed $X^2$ and check if it is equal to or smaller than $\alpha$
If $b + c$ is small, the table for the binomial distribution should be used, with as test statistic $b$:
  • Check if $b$ observed in sample is in the rejection region or
  • Find two sided $p$ value corresponding to observed $b$ and check if it is equal to or smaller than $\alpha$
Equivalent ton.a.Equivalent toEquivalent to
Two sided sign test is equivalent to -
Example contextExample contextExample contextExample context
Do people tend to score higher on mental health after a mindfulness course?Is the proportion of smokers amongst office workers different from $\pi_0 = 0.2$?Does a tv documentary about spiders change whether people are afraid (yes/no) of spiders?Does a tv documentary about spiders change whether people are afraid (yes/no) of spiders?
SPSSSPSSSPSSSPSS
Analyze > Nonparametric Tests > Legacy Dialogs > 2 Related Samples...
  • Put the two paired variables in the boxes below Variable 1 and Variable 2
  • Under Test Type, select the Sign test
Analyze > Nonparametric Tests > Legacy Dialogs > Binomial...
  • Put your dichotomous variable in the box below Test Variable List
  • Fill in the value for $\pi_0$ in the box next to Test Proportion
Analyze > Nonparametric Tests > Legacy Dialogs > 2 Related Samples...
  • Put the two paired variables in the boxes below Variable 1 and Variable 2
  • Under Test Type, select the McNemar test
Analyze > Nonparametric Tests > Legacy Dialogs > 2 Related Samples...
  • Put the two paired variables in the boxes below Variable 1 and Variable 2
  • Under Test Type, select the McNemar test
JamoviJamoviJamoviJamovi
Jamovi does not have a specific option for the sign test. However, you can do the Friedman test instead. The $p$ value resulting from this Friedman test is equivalent to the two sided $p$ value that would have resulted from the sign test. Go to:

ANOVA > Repeated Measures ANOVA - Friedman
  • Put the two paired variables in the box below Measures
Frequencies > 2 Outcomes - Binomial test
  • Put your dichotomous variable in the white box at the right
  • Fill in the value for $\pi_0$ in the box next to Test value
  • Under Hypothesis, select your alternative hypothesis
Frequencies > Paired Samples - McNemar test
  • Put one of the two paired variables in the box below Rows and the other paired variable in the box below Columns
Frequencies > Paired Samples - McNemar test
  • Put one of the two paired variables in the box below Rows and the other paired variable in the box below Columns
Practice questionsPractice questionsPractice questionsPractice questions