The Null Model as a Reference Distribution

Why a Null Hypothesis Needs a Model

A null hypothesis states a specific claim about a population parameter, such as “the population mean is 70 minutes” or “the population proportion is 0.25.” But that claim does not, by itself, describe how much a sample mean or sample proportion would vary from one sample to another. To judge a result against the null, we need a probability model for the data as well as the null claim.

That model gives a reference distribution: the distribution of a chosen sample statistic across repeated samples, assuming the null hypothesis and the model’s assumptions hold. It describes what values the statistic could take under that specified situation. At this stage, the goal is to construct and understand that distribution—not yet to measure how extreme an observed result is.

Definition: A null model combines the null hypothesis with assumptions about the population and the sampling process. The reference distribution is the probability distribution of a specified statistic under that null model. It describes how the statistic would vary over repeated samples if the null model were correct.

A useful way to keep the ideas separate is to ask three questions. What population parameter does \(H_0\) specify? What statistic will be calculated from the sample? What assumptions determine the statistic’s distribution when \(H_0\) holds? For example, \(H_0:\mu=70\) specifies a population mean, but says nothing about the population standard deviation or shape. Those details can matter to the reference distribution of \(\bar{x}\).

The reference distribution is not the distribution of individual observations. It is the distribution of a statistic computed from samples. Individual measurements may vary widely while sample means from adequately sized samples vary less. The sampling plan matters too: a random sample taken independently is modeled differently from a design with dependence or clustering.

A Reference Distribution for a Sample Mean

Suppose a quantitative measurement is taken on each of \(n\) individuals, and the statistic of interest is the sample mean \(\bar{x}\). If observations are independent and come from a normal population with mean \(\mu\) and standard deviation \(\sigma\), then the sampling distribution of \(\bar{x}\) is normal with mean \(\mu\) and variance \(\sigma^2/n\). Under the null hypothesis \(H_0:\mu=70\), the null model therefore sets the center of this distribution at 70. The model also needs the value of \(\sigma\) and the normal-population and independent-sampling assumptions to specify its spread and shape.

\bar{x}\sim N\left(\mu,\frac{\sigma^2}{n}\right)

In this notation, the second argument of \(N\) is the variance. The standard deviation of the reference distribution of \(\bar{x}\), called its standard error, is \(\sigma/\sqrt{n}\). It measures the sample-to-sample variability of the mean under the model; it is not the standard deviation of individual measurements.

Worked Example: A Mean Under a Normal Null Model

Suppose a hypothetical quality-control process measures the fill volume, in millilitres, of each container in a random sample of \(n=16\) containers. Consider the null hypothesis \(H_0:\mu=70\) mL. For this example, assume individual fill volumes are independent observations from a normal population with known \(\sigma=8\) mL. A sample has \(\bar{x}=73\) mL. Find the reference distribution of the sample mean under the null model.

Check the model. The null hypothesis supplies the mean, 70 mL. The stated normal-population and independent-sampling assumptions make the normal sampling distribution exact. The known population standard deviation supplies its spread. These assumptions are part of this hypothetical setup; the null hypothesis alone would not establish them.

Calculate the variance and standard error.

\operatorname{Var}(\bar{x})=\frac{\sigma^2}{n} =\frac{8^2}{16} =\frac{64}{16} =4\ \text{mL}^2

SE(\bar{x})=\frac{\sigma}{\sqrt{n}} =\frac{8}{\sqrt{16}} =2\ \text{mL}

State the reference distribution. Under \(H_0\), the sample mean has a normal distribution centered at 70 mL with variance \(4\ \text{mL}^2\):

\bar{x}\sim N(70,4)

The observed mean of 73 mL is one possible value of the statistic described by this distribution. The reference distribution is defined by the null model, not by moving its center to the observed sample mean.

Why the Mean Distribution Has This Form

The mean model above follows from the behavior of a sum of independent observations. Let \(X_1,\ldots,X_n\) be independent observations from a normal population with mean \(\mu\) and variance \(\sigma^2\). The sum of independent normal random variables is normal. Its mean is the sum of their means, and its variance is the sum of their variances. Since \(\bar{X}=(X_1+\cdots+X_n)/n\), scaling that sum by \(1/n\) gives the result below.

E(\bar{X})= \frac{E(X_1)+\cdots+E(X_n)}{n} =\frac{n\mu}{n} =\mu

\operatorname{Var}(\bar{X})= \frac{\operatorname{Var}(X_1+\cdots+X_n)}{n^2} =\frac{n\sigma^2}{n^2} =\frac{\sigma^2}{n}

Because a constant multiple of a normal random variable is normal, \(\bar{X}\) is normal as well. Substituting the null value for \(\mu\) gives the reference distribution under \(H_0\). Independence is essential to the variance calculation: if observations are dependent, the variance of their sum can include additional covariance terms.

If individual observations are not normally distributed, the sample mean is not necessarily exactly normal for a finite sample. Under the central limit theorem, the sampling distribution of the mean is approximately normal for sufficiently large samples when observations are independent and identically distributed with finite variance. How adequate that approximation is depends on the population distribution and sample size. The null value still specifies the center, but the approximation must be justified rather than assumed to be exact.

Worked Example: A Large-Sample Mean Approximation

In a hypothetical setting, independent random arrivals have waiting times with a moderately right-skewed distribution, mean \(\mu=12\) minutes under \(H_0\), and known standard deviation \(\sigma=3\) minutes. Assume the population has finite variance and no extreme heavy tail. A sample of \(n=100\) arrivals is taken. Describe the reference distribution for \(\bar{x}\).

Check the model. The waiting times are not stated to be normally distributed, so an exact normal distribution for the mean does not follow. However, the sample is large, observations are independent, and the stated population has finite variance and only moderate skew. These conditions support a central-limit approximation.

Calculate the center, variance, and standard error.

E(\bar{x})=12\ \text{minutes}

\operatorname{Var}(\bar{x})=\frac{3^2}{100} =\frac{9}{100} =0.09\ \text{minutes}^2

SE(\bar{x})=\frac{3}{\sqrt{100}} =0.3\ \text{minutes}

State the approximation. Under the null model, \(\bar{x}\) is approximately distributed as \(N(12,0.09)\). This is an approximation to the sampling distribution of the mean, not a claim that individual waiting times are normally distributed. A different sample size or a more severely skewed population could make the approximation less adequate.

A Reference Distribution for a Sample Proportion

For a binary outcome, let each sampled individual either have or not have the characteristic of interest. If the individuals are sampled independently and each has the same probability \(p\) of having that characteristic, the count \(X\) of individuals with the characteristic has a binomial distribution. The sample proportion is \(\hat{p}=X/n\). Under a null hypothesis specifying \(p\), this binomial model gives an exact reference distribution for \(X\), and therefore for \(\hat{p}\).

X\sim\operatorname{Binomial}(n,p), \qquad \hat{p}=\frac{X}{n}

The possible values of \(X\) are the integers from 0 through \(n\), so the possible values of \(\hat{p}\) are \(0,1/n,2/n,\ldots,1\). The distribution is discrete. Its mean and variance follow from the binomial model:

E(\hat{p})=p, \qquad \operatorname{Var}(\hat{p})=\frac{p(1-p)}{n}, \qquad SE(\hat{p})=\sqrt{\frac{p(1-p)}{n}}

Worked Example: A Proportion Under a Binomial Null Model

A hypothetical survey asks \(n=40\) independently sampled people whether they support a proposal. Let \(p\) be the population proportion who support it, and consider \(H_0:p=0.25\). Suppose 14 of the 40 sampled people support the proposal. Describe the reference distribution of the sample proportion under the null model.

Check the model. The null sets each person’s support probability to 0.25. The exact binomial model additionally assumes independent responses and a common probability of support for all sampled people. Given those assumptions, the number of supporters follows a binomial distribution.

State the count distribution and connect it to the statistic. Under \(H_0\), \(X\sim\operatorname{Binomial}(40,0.25)\), and \(\hat{p}=X/40\). Thus \(\hat{p}\) can take the values 0, \(1/40\), \(2/40\), through 1. The observed value is calculated as

\hat{p}=\frac{14}{40}=0.35

Calculate the model’s center and spread.

E(\hat{p})=0.25

\operatorname{Var}(\hat{p}) =\frac{0.25(1-0.25)}{40} =\frac{0.1875}{40} =0.0046875

SE(\hat{p})=\sqrt{0.0046875}\approx0.06847

The null reference distribution is the exact distribution of \(X/40\) when \(X\sim\operatorname{Binomial}(40,0.25)\). Its center is 0.25 and its standard error is about 0.06847. The observed proportion, 0.35, is one possible value on its discrete scale.

What the Model Must Specify

The examples illustrate an important distinction: the null hypothesis sets a parameter value, while the sampling model supplies the probability structure that turns that value into a distribution of sample statistics. For a mean, this may require assumptions about the population distribution, \(\sigma\), and how observations are sampled. For a proportion, it requires a binary outcome and assumptions that support a binomial model.

The sampling design cannot be ignored. If observations are related—for example, if responses are grouped within households—the independence assumption may fail. If sampling is without replacement from a small finite population, the outcomes need not be independent. In such settings, simply applying an independent-observation reference distribution may misrepresent sample-to-sample variation. The model must match both the null claim and the way data could be generated.

Common Mistake

Wrong reasoning: “The null hypothesis says the mean is 70, so the sample mean must be 70 under the null.” This confuses the population parameter with the statistic calculated from a sample. Even when the population mean is exactly 70, individual samples can have means above or below 70.

Correction: The null specifies the center of the sample-mean distribution when the model’s assumptions hold; it does not force every sample mean to equal that center. The distribution’s spread depends on sample size and population variability. Likewise, a null proportion does not force a sample proportion to equal that proportion in every sample.

Key takeaway: A reference distribution is built by combining a null parameter value with a sampling model and a specified statistic. State the assumptions, identify the statistic, and then describe the distribution that statistic would have under the null model.

Check Your Understanding

Answer each question using the distinction between a null value and a reference distribution.

Why does \(H_0:\mu=50\) fail, by itself, to specify the full sampling distribution of \(\bar{x}\)?
For independent normal observations with population standard deviation 6 and sample size 9, what is the standard error of \(\bar{x}\)?
Under a binomial model with \(n=20\) and null proportion \(p=0.30\), what values can \(\hat{p}\) take?
What additional assumptions allow the sample mean in the large-sample waiting-time example to be treated as approximately normal?
Explain why a null hypothesis about a population mean does not require every sample mean to equal the null value.