Paired sample $t$ test - sampling distribution of the sample mean of the difference scores, and its standard error

Definition of the sampling distribution of the sample mean of the difference scores $\bar{y}$, and its standard error


Sampling distribution of the sample mean of the difference scores $ \bar{y}$:

When we have a sample consisting of $ N$ difference scores (e.g., difference scores of pre- and post-measurements before and after an intervention), we can compute the sample mean of the difference scores: $ \bar{y}$. Now suppose that we repeated our study several times. Specifically, suppose that we repeated our study an infinite number of times, so we have an infinite number of samples of difference scores, each of size $ N$. In each sample, we could compute the sample mean of difference scores $ \bar{y}$. Different samples would give different sample means. The distribution of all these sample means is the sampling distribution of $ \bar{y}$. Note that this sampling distribution is purely hypothetical. We would never really repeat our study an infinite number of times, but hypothetically, we could.

Standard error:

Suppose that the assumptions of the paired sample $ t$ test hold:

  • The difference scores are normally distributed in the population, with mean $\mu$ and standard deviation $\sigma$
  • The sample of difference scores is a simple random sample from the population of difference scores. That is, the difference scores are independent of one another
Then the sampling distribution of $ \bar{y}$ is normal with mean $\mu$ and standard deviation $\sigma / \sqrt{N}$. The difference scores in the population can be conceived of as the difference scores we would find if we would apply our study (e.g., applying an intervention and measuring pre-post scores) to all individuals in the population. Since the paired sample $ t$ test does not make the assumption that the value of $\sigma$ is known (like the $ z$ test does), we need to:
  • estimate $\sigma$ with $ s$: the standard deviation of the difference scores in the sample
  • estimate $\sigma / \sqrt{N}$ with $ s / \sqrt{N}$
That is, we estimate the standard deviation of the sampling distribution of $\bar{y}$, $\sigma / \sqrt{N}$, with $ s / \sqrt{N}$. We call this estimated standard deviation of the sampling distribution of $\bar{y}$ the standard error of $\bar{y}$.

Note that the $ t$ statistic $ t = \frac{\bar{y} - \mu_0}{s / \sqrt{N}}$ thus indicates how many standard errors the observed sample mean of the difference scores $\bar{y}$ is removed from $\mu_0$: the population mean of the difference scores according to the null hypothesis.