Significance Levels and Test Size

Why Significance Level and Size Need Careful Distinction

A significance level is often stated before data are collected: for example, a study may specify a nominal level of \(0.05\). That number describes a tolerance for false rejection under the null hypothesis; it does not automatically equal the test’s actual rejection probability. The distinction is especially important when the null hypothesis allows several parameter values.

A test’s rejection probability is calculated under a specified model and parameter value, before the data are observed. For a composite null hypothesis, that probability can differ across null parameter values. The test’s size summarizes those probabilities by taking their supremum over the null parameter set. The supremum may be attained, but it need not be.

Definition: Let \(X\) denote the observed data, let \(\theta\) be a model parameter, and let \(\Theta_0\) be the parameter values permitted by the null hypothesis. A test rejects for data in a specified rejection region \(R\). Its rejection probability at \(\theta\) is \(P_\theta(X\in R)\). Its size is

\sup_{\theta\in\Theta_0} P_\theta(X\in R).

A test has level \(\alpha\) if its size is at most \(\alpha\). The chosen \(\alpha\) is the nominal significance level, or nominal error bound. If the size equals \(\alpha\), the test attains that level; if the size is smaller, the test is conservative at the stated level.

Some texts use “significance level” to mean the test’s size itself; in practical descriptions, “a test at level \(\alpha\)” often means a test whose size is no greater than \(\alpha\). The distinction to retain is between the stated bound \(\alpha\) and the test’s actual size. When the null hypothesis is simple, it specifies one parameter value, so the size is just the rejection probability at that value. When the null is composite, the size is a supremum over all its permitted parameter values.

The supremum is the least upper bound. It is a maximum only when at least one permitted null value actually gives that rejection probability. If rejection probabilities approach a value without ever reaching it, that value is still the size. Using “largest probability” in every case would therefore be inaccurate.

Rejection Probability Before and After Data Are Observed

Before sampling, \(X\) is random under the assumed model, so the probability \(P_\theta(X\in R)\) is meaningful. Once the data have been observed, the result either lies in the rejection region or it does not: the test’s decision for that data set is fixed. The nominal significance level is not the probability that this particular decision is wrong, nor is it the probability that the null hypothesis is true.

The size instead describes the test procedure under repeated sampling, evaluated at null parameter values. For a composite null, the size is its worst-case rejection probability in the sense of a supremum. It does not say that every null value produces that probability, or that a particular observed rejection has that probability of being erroneous.

Worked Example: A Simple Null and a Conservative Test

A quality-control test counts \(X\), the number of defective units in a random sample of four. Assume independent sampling and a common defect probability \(p\), so \(X\sim\operatorname{Binomial}(4,p)\). The null hypothesis is \(H_0:p=0.5\), and the procedure rejects only when all four units are defective. The plan states a nominal significance level of \(0.10\). In the observed sample, all four units are defective. Find the test’s size, assess its level, and interpret the observed decision.

Method and model. Under the null, \(X\sim\operatorname{Binomial}(4,0.5)\), and the rejection region is \(R=\{4\}\). Because the null contains only \(p=0.5\), the size is the rejection probability at that value.

Calculate the size. The binomial probability of four defectives is

P_{0.5}(X=4) =\binom{4}{4}(0.5)^4(1-0.5)^0 =1(0.0625)(1) =0.0625.

Thus the size is \(0.0625\). Since \(0.0625\leq 0.10\), the test has level \(0.10\), but it is conservative at that nominal level; its size does not equal the bound.

Interpret the observation. The observed value is \(X=4\), which is in \(R\), so the test rejects. That decision is fixed for these observed data. The value \(0.10\) is not the probability that this rejection is wrong. Under the null model, the procedure’s rejection probability is \(0.0625\).

Common mistake avoided. Equating the nominal \(0.10\) with the actual null rejection probability would give the wrong size. The rejection region and null distribution determine the size; the stated bound only tells us the maximum allowed size.

Calculating Size for a Composite Null

With a composite null, first find the rejection probability as a function of the parameter. Then take its supremum over the null parameter set. A parameter value outside that set cannot determine the size, even if it would give a larger rejection probability.

Worked Example: A Composite Binomial Null With an Attained Supremum

A hypothetical laboratory tests a process by recording \(X\), the number of flagged readings among five independent readings. Each reading is flagged with probability \(p\), so \(X\sim\operatorname{Binomial}(5,p)\). The null is \(H_0:0\leq p\leq0.4\). The procedure rejects when \(X\geq4\), and the nominal significance level is \(0.10\). Find the size and determine whether the test has the stated level.

Method and model. For any fixed \(p\) in the null set, the rejection probability is the probability of four or five flags:

g(p)=P_p(X\geq4) =\binom{5}{4}p^4(1-p)+\binom{5}{5}p^5 =5p^4(1-p)+p^5 =5p^4-4p^5.

Locate the supremum. For \(0<p\leq0.4\),

$$ g'(p)=20p^3-20p^4=20p^3(1-p)>0. $$

So \(g(p)\) increases across the null parameter range. Also \(g(0)=0\), and the upper endpoint \(p=0.4\) is included in the null. Therefore the supremum is attained at \(p=0.4\).

Evaluate it. At the boundary,

\begin{aligned} g(0.4) &=5(0.4)^4(0.6)+(0.4)^5\\ &=5(0.0256)(0.6)+0.01024\\ &=0.0768+0.01024\\ &=0.08704. \end{aligned}

The size is \(0.08704\), or \(0.0870\) to four decimal places. Since \(0.08704\leq0.10\), the procedure has level \(0.10\), and it does not attain the nominal bound.

Common mistake avoided. Evaluating the rejection probability at a convenient interior value, such as \(p=0.2\), would not establish the size. The size requires the supremum over every null value. Here monotonicity proves that the upper endpoint controls it.

Why the Supremum May Not Be a Maximum

A supremum need not be attained when the null parameter set excludes a limiting value. The following example is deliberately simple so the distinction is visible without lengthy probability calculations.

Worked Example: A Composite Null With an Unattained Size

Let \(X\) be one Bernoulli observation with \(P_\theta(X=1)=\theta\), where the null hypothesis is \(H_0:0<\theta<1\). Consider the procedure that rejects when \(X=1\). Find the size and decide whether any null parameter value gives that rejection probability.

Method and model. The rejection region is \(R=\{1\}\). For any permitted null value,

P_\theta(X\in R)=P_\theta(X=1)=\theta,\qquad 0<\theta<1.

As \(\theta\) approaches \(1\) from below, these rejection probabilities approach \(1\). No permitted value equals \(1\), because the null set is open at that endpoint. Thus

\sup_{0<\theta<1}P_\theta(X\in R)=1,

but there is no null parameter value at which the rejection probability equals \(1\). The size is \(1\), and it is a supremum rather than a maximum. Calling it the “largest rejection probability under the null” would falsely imply that some permitted \(\theta\) attains it.

Common mistake avoided. A sequence of null values can make rejection probabilities arbitrarily close to the size without any one null value achieving it. The supremum, not an assumed maximum, is the correct definition.

A Short Justification of the Level Condition

The definition of size explains exactly why it controls the rejection probability at every null value. Write \(r(\theta)=P_\theta(X\in R)\). For every \(\theta\in\Theta_0\), the definition of supremum gives \(r(\theta)\leq\sup_{\eta\in\Theta_0}r(\eta)\). Therefore, if the size is at most \(\alpha\), then every null parameter value satisfies the same bound:

\sup_{\eta\in\Theta_0}P_\eta(X\in R)\leq\alpha \quad\Longrightarrow\quad P_\theta(X\in R)\leq\alpha \quad\text{for every }\theta\in\Theta_0.

Conversely, if \(P_\theta(X\in R)\leq\alpha\) for every \(\theta\in\Theta_0\), then \(\alpha\) is an upper bound for all those probabilities, so their supremum is also at most \(\alpha\). Hence the test has level \(\alpha\) exactly when its size is no greater than \(\alpha\). This argument does not require the supremum to be attained.

Reading a Significance Statement Correctly

When a report says that a test uses a nominal significance level of \(0.05\), the statement identifies a design bound. To determine the actual size, one also needs the null model and the rejection region, and for a composite null one must evaluate the rejection probability across the null parameter set. The possible relationships are:

Relationship	Interpretation
Size equals nominal \(\alpha\)	The test attains the stated level.
Size is less than nominal \(\alpha\)	The test is conservative at that stated level.
Size exceeds nominal \(\alpha\)	The test does not have the claimed level.

Keep the distinction between a repeated-sampling probability and an observed outcome in view. A test does not assign a probability to its already-observed reject-or-not-reject decision. Its size describes how often the procedure would reject under null parameter values, with the composite-null size summarizing those rates by their supremum. It is not a probability that the null is true, and it is not the probability that a particular rejection is an error.

Key takeaway: The nominal significance level \(\alpha\) is a chosen upper bound. The size is the supremum of the test’s rejection probabilities over the null parameter set. That supremum may or may not be attained, and neither quantity is the probability that a particular observed decision is wrong.

Check Your Understanding

For each problem, identify the rejection probability over the null and compare the resulting size with the stated nominal level.

Let \(X\sim\operatorname{Binomial}(3,p)\) under the simple null \(p=0.2\). A test rejects when \(X\geq2\) and has nominal level \(0.12\). What is its size, and is it conservative? (Answer: Size \(=0.1040\); yes, it is conservative at level \(0.12\).)
Let \(X\sim\operatorname{Binomial}(4,p)\), with \(H_0:0\leq p\leq0.3\). A test rejects when \(X\geq3\) and has nominal level \(0.09\). Its rejection probability is \(4p^3(1-p)+p^4\). Find the size and state whether the nominal level is satisfied. (Answer: Size \(=0.0837\), attained at \(p=0.3\); yes, the test has level \(0.09\).)
Let \(X\sim N(\mu,9)\), with \(H_0:\mu\leq50\), and reject when \(X>55\). Assuming this model for every null value, find the size to four decimal places and compare it with nominal level \(0.05\). (Answer: Size \(=P_{50}(X>55)=1-\Phi(5/3)\approx0.0478\), attained at \(\mu=50\); the test is conservative at level \(0.05\).)
Let \(X\) be Bernoulli with parameter \(\theta\), and let \(H_0:0<\theta<1\). A procedure rejects when \(X=0\). Find the size and say whether any null value attains it. (Answer: The rejection probability is \(1-\theta\), so the size is \(1\); it is not attained by any permitted null value.)
A test has nominal level \(0.08\) and size \(0.06\). For a particular observed data set it rejects the null. What does \(0.08\) state, and what can be concluded from these two values about the test’s level? (Answer: \(0.08\) is the nominal upper bound on null rejection probability; the size \(0.06\) means the test has level \(0.08\) and is conservative. Neither value is the probability that this particular rejection is wrong.)

What You'll Learn