One Threshold, Two Possible Errors
A hypothesis test turns a statistic into a decision: reject the null hypothesis when the statistic falls in a designated rejection region, and otherwise do not reject it. For an upper-tail test, the rule often has a single threshold \(c\): reject when \(T>c\). Moving \(c\) changes which outcomes lead to rejection. Because the sampling distributions under the null and alternative can overlap, that change affects both kinds of error.
This tutorial considers a specified null distribution and a specified alternative distribution, with the same test statistic and an upper-tail rejection rule. The key is to calculate each error under the hypothesis that makes that error possible. The threshold does not change the underlying distributions; it changes how outcomes from those distributions are classified.
These probabilities answer different conditional questions. To calculate \(\alpha\), use the null distribution and measure its probability above the cutoff. To calculate \(\beta\), use the alternative distribution and measure its probability at or below the cutoff. A numerical threshold by itself is not enough: the hypothesis and the distribution used for each calculation must also be specified.
Normal Sampling Distributions and the Threshold Tradeoff
Suppose the statistic \(T\) has a normal sampling distribution under each hypothesis, with a common standard deviation \(\tau>0\):
Here \(N(\mu,\tau^2)\) uses \(\tau^2\) as the variance, so \(\tau\) is the standard deviation. The test rejects \(H_0\) when \(T>c\). Let \(\Phi(z)\) denote the standard normal cumulative distribution function, \(P(Z\le z)\) for \(Z\sim N(0,1)\). Standardizing each probability gives:
The arguments of \(\Phi\) differ because the same cutoff is measured from a different mean under each hypothesis. Raising \(c\) moves the rejection boundary farther into the upper tail of both distributions. That reduces the fraction of the null distribution above the boundary, but it also leaves more of the alternative distribution at or below it.
Why the Error Probabilities Move in Opposite Directions
The formulas show the tradeoff directly. If \(c\) increases, then \((c-\mu_0)/\tau\) increases. Since \(\Phi\) increases with its argument, \(1-\Phi((c-\mu_0)/\tau)\) decreases. Meanwhile, \((c-\mu_1)/\tau\) also increases, so \(\Phi((c-\mu_1)/\tau)\) increases.
A derivative makes the direction explicit. Write \(\phi\) for the standard normal density, which is positive at every finite argument. Differentiating the two error probabilities with respect to \(c\) gives:
This also explains why one threshold generally cannot make both errors smaller than they were at another threshold: moving the cutoff in the direction that reduces one error moves it in the direction that increases the other. The amount of change depends on how much probability density lies near the cutoff under each distribution. If the distributions overlap substantially, outcomes near the threshold are plausible under both hypotheses, so changing the classification boundary can materially affect both errors.
Calculating the Tradeoff
Worked Example: Comparing Two Thresholds for a Sample Mean
A hypothetical process produces independent normally distributed measurements with known standard deviation \(8\). A sample of \(16\) measurements is taken, and the test statistic is the sample mean \(\bar X\). Test \(H_0:\mu=100\) against the specified alternative \(H_1:\mu=104\), rejecting for \(\bar X>c\). Compare thresholds \(c=102\) and \(c=104\).
Method and model. With independent normal observations and known standard deviation \(8\), the sample mean has standard deviation \(8/\sqrt{16}=8/4=2\). Thus the two sampling distributions are \(\bar X\mid H_0\sim N(100,2^2)\) and \(\bar X\mid H_1\sim N(104,2^2)\).
At \(c=102\): Under the null, the standardized cutoff is \((102-100)/2=1\), so \(\alpha=1-\Phi(1)=1-0.8413=0.1587\). Under the alternative, it is \((102-104)/2=-1\), so \(\beta=\Phi(-1)=0.1587\).
At \(c=104\): Under the null, the standardized cutoff is \((104-100)/2=2\), so \(\alpha=1-\Phi(2)=1-0.9772=0.0228\). Under the alternative, it is \((104-104)/2=0\), so \(\beta=\Phi(0)=0.5000\).
Raising the cutoff from \(102\) to \(104\) reduces the Type I error from \(0.1587\) to \(0.0228\), while increasing the Type II error from \(0.1587\) to \(0.5000\). The power at the specified alternative correspondingly falls from \(1-0.1587=0.8413\) to \(1-0.5000=0.5000\). The calculation avoids the mistake of using one sampling distribution for both errors: each probability was evaluated under its own hypothesis.
Check: At \(c=104\), the alternative distribution is centered exactly at the cutoff. For a continuous symmetric normal distribution, half its probability lies above and half below, confirming \(\beta=0.5000\).
Worked Example: Tracking Several Cutoffs
A hypothetical laboratory measures a response whose sample mean has standard deviation \(2\). Consider \(H_0:\mu=50\) and the specified alternative \(H_1:\mu=54\), with the rule to reject for \(\bar X>c\). Find both errors at \(c=51,52,\) and \(53\), and describe the trend.
Method and model. Assume the sample mean is normally distributed with standard deviation \(2\) under both hypotheses. At any cutoff, standardize relative to \(50\) for \(\alpha\) and relative to \(54\) for \(\beta\).
At \(c=51\), the null z-score is \((51-50)/2=0.5\), giving \(\alpha=1-\Phi(0.5)=1-0.6915=0.3085\). The alternative z-score is \((51-54)/2=-1.5\), giving \(\beta=\Phi(-1.5)=0.0668\).
At \(c=52\), the null z-score is \((52-50)/2=1\), giving \(\alpha=1-\Phi(1)=1-0.8413=0.1587\). The alternative z-score is \((52-54)/2=-1\), giving \(\beta=\Phi(-1)=0.1587\).
At \(c=53\), the null z-score is \((53-50)/2=1.5\), giving \(\alpha=1-\Phi(1.5)=1-0.9332=0.0668\). The alternative z-score is \((53-54)/2=-0.5\), giving \(\beta=\Phi(-0.5)=0.3085\).
The results are \((\alpha,\beta)=(0.3085,0.0668)\), \((0.1587,0.1587)\), and \((0.0668,0.3085)\), in increasing order of cutoff. The lower cutoff is more likely to reject under either hypothesis; consequently it has greater Type I error but smaller Type II error. The common mistake avoided here is to treat a rising threshold as improving both error rates.
Check: Each increase in the threshold lowers the null upper-tail probability and raises the alternative lower-tail probability, as the formulas predict.
Worked Example: A Midpoint Cutoff Does Not Eliminate Error
A hypothetical instrument yields a statistic \(T\) with \(T\mid H_0\sim N(20,3^2)\) and \(T\mid H_1\sim N(23,3^2)\). The rule rejects \(H_0\) when \(T>c\). Use the midpoint \(c=21.5\) and calculate both error probabilities.
Method and model. The means are \(20\) and \(23\), and their midpoint is \((20+23)/2=21.5\). The cutoff is \(1.5\) above the null mean and \(1.5\) below the alternative mean; with standard deviation \(3\), the standardized distances are \(1.5/3=0.5\) in magnitude.
Under \(H_0\), \(\alpha=P(T>21.5\mid H_0)=1-\Phi((21.5-20)/3)=1-\Phi(0.5)=1-0.6915=0.3085\).
Under \(H_1\), \(\beta=P(T\le21.5\mid H_1)=\Phi((21.5-23)/3)=\Phi(-0.5)=0.3085\).
The midpoint makes these two errors equal in this equal-standard-deviation normal model, but neither error is zero. The distributions overlap, so some values that occur under the null also occur under the alternative. The method avoids assuming that a cutoff halfway between the means guarantees correct classification; it balances these errors here but does not remove them.
Check: The standardized distances are equal in magnitude, and the standard normal symmetry identity \(1-\Phi(0.5)=\Phi(-0.5)\) confirms the matching probabilities.
Reading the Tradeoff Carefully
A threshold is a choice about how much evidence is required before rejecting the null. For an upper-tail test, a higher threshold requires a more extreme value of \(T\). That makes rejection less frequent when the null is true, but also makes rejection less frequent under the specified alternative. Because power is \(1-\beta\), the higher threshold lowers power at that alternative.
There is no universally best threshold based only on the two distributions. A decision-maker may place greater importance on limiting false rejections, or on detecting the specified alternative. The chosen threshold determines the corresponding pair \((\alpha,\beta)\); comparing thresholds is therefore a comparison of error tradeoffs, not a claim that one error has disappeared.
The formulas above rely on the stated upper-tail rule and the specified distributions. If the rejection region is lower-tail or two-sided, the regions used to calculate each error differ; do not apply the upper-tail formulas without changing the probabilities. The essential principle remains that Type I error is evaluated under the null, while Type II error is evaluated under the alternative. Keeping that distinction clear is aligned with the SOA VEE Mathematical Statistics syllabus.
Check Your Understanding
For each problem, identify the relevant null and alternative probabilities before comparing or calculating errors.
- A statistic has distributions \(T\mid H_0\sim N(40,4^2)\) and \(T\mid H_1\sim N(43,4^2)\). The rule rejects for \(T>42\). Calculate \(\alpha\) and \(\beta\). (Answer: \(\alpha=0.3085,\ \beta=0.4013\))
- A sample mean has standard deviation \(2\) under \(H_0:\mu=12\) and the specified alternative \(H_1:\mu=15\). With rejection rule \(\bar X>14\), calculate both errors and the power. (Answer: \(\alpha=0.1587,\ \beta=0.3085,\ \text{power}=0.6915\))
- For \(T\mid H_0\sim N(0,1)\) and \(T\mid H_1\sim N(2,1)\), compare cutoffs \(1\) and \(1.5\) for an upper-tail test. Give \((\alpha,\beta)\) at each cutoff. (Answer: at \(c=1\), \((0.1587,0.1587)\); at \(c=1.5\), \((0.0668,0.3085)\))
- A statistic has distributions \(T\mid H_0\sim N(30,4^2)\) and \(T\mid H_1\sim N(36,4^2)\). Find the midpoint cutoff and the two errors for the rule \(T>c\). (Answer: \(c=33,\ \alpha=\beta=0.2266\))
- Under \(H_0\), a statistic has distribution \(N(100,10^2)\); under the specified alternative it has distribution \(N(108,10^2)\). Choose the upper-tail cutoff to give \(\alpha=0.0500\), then find \(\beta\). Use \(z_{0.95}=1.6449\). (Answer: \(c=116.449\), approximately \(116.45\); \(\beta=\Phi(0.8449)\approx0.8008\))