Why Neyman–Pearson Construction Errors Matter
A Neyman–Pearson test is built to distinguish one specified null distribution from one specified alternative distribution. Its construction can be concise, but small errors have substantial effects: reversing a likelihood ratio reverses the rejection rule, a support difference can create evidence missed by an ordinary density comparison, and a discrete cutoff can have a size different from the one requested. A correct-looking critical region is not enough; it must be checked under the null.
The standard result applies to two simple hypotheses. It does not, by itself, establish optimality against a collection of alternatives. The diagnostic habit throughout is to state the hypotheses, define the ratio, inspect the supports, and calculate size and power under the correct distributions.
The Ratio Direction and the Rejection Rule
Use the convention \( \Lambda(x)=f_1(x)/f_0(x) \) where \(f_0(x)>0\). Large values of this ratio mean that the observation is relatively more likely under \(H_1\), so the Neyman–Pearson rule rejects \(H_0\) for large ratios. If a problem instead defines the reciprocal \(f_0/f_1\), the inequality reverses. Either convention works, but combining one ratio with the other convention’s rejection direction does not.
The support must be checked before forming or simplifying the ratio. Where \(f_0=0\) and \(f_1>0\), an observation is impossible under the null but possible under the alternative; it should be treated as having infinite likelihood ratio and rejected. Where \(f_1=0\) and \(f_0>0\), the ratio is zero. Where both are zero, the observation has probability zero under both hypotheses and does not affect size or power.
Identify which distribution is \(H_0\) and which is \(H_1\). Do not infer the null from which parameter value looks more familiar.
Use the same dominating measure under both hypotheses. Record any region possible only under \(H_1\) or only under \(H_0\).
Check whether the ratio increases or decreases with the statistic before choosing an upper or lower tail.
Calculate rejection probability under \(H_0\) for the size, then under \(H_1\) for power. In a discrete model, consider randomization at a boundary.
Why the Likelihood-Ratio Rule Is Most Powerful
The key comparison is between the candidate likelihood-ratio test \(\phi^*\) and any other test \(\phi\) whose size is no larger. Suppose \(\phi^*\) has exact size \(\alpha\), rejects above \(c\), and accepts below \(c\). On the rejection region, \(\phi-\phi^*\leq0\) and \(f_1-cf_0>0\). On the acceptance region, \(\phi-\phi^*\geq0\) and \(f_1-cf_0<0\). On the boundary, \(f_1-cf_0=0\). Therefore the first term in the following expression is nonpositive:
The second term is also nonpositive because \(E_0[\phi]\leq\alpha=E_0[\phi^*]\) and \(c\geq0\). Thus no test of size at most \(\alpha\) has greater power against this particular simple \(H_1\). This argument also shows why the ratio direction matters: the signs in the comparison depend on rejecting where \(f_1-cf_0\) is positive.
Worked Example: Reversing the Ratio Reverses the Tail
Let \(X\sim N(0,1)\) under \(H_0\), and \(X\sim N(2,1)\) under \(H_1\). Construct a most powerful test of size \(0.05\), calculate its power, and identify the error made by using \(f_0/f_1\) with an upper-tail rule.
Method and ratio. Both hypotheses have variance \(1\). Their densities give \(f_1(x)/f_0(x)=\exp(2x-2)\), which increases with \(x\). Therefore the rule rejects for sufficiently large \(X\), not sufficiently small \(X\).
Calibrate size under \(H_0\). Under the null, \(X\sim N(0,1)\). The upper 5% standard-normal cutoff is \(z_{0.95}=1.6448536\), so reject when \(X>1.6448536\). The null rejection probability is \(0.05\).
Calculate power under \(H_1\). Under the alternative, \(X-2\sim N(0,1)\). Hence \(P_1(X>1.6448536)=P(Z> -0.3551464)=\Phi(0.3551464)\approx0.6388\), where \(Z\sim N(0,1)\). The power is \(0.6388\) to four decimal places.
Common mistake and correction. The reciprocal \(f_0/f_1=\exp(-2x+2)\) decreases with \(x\). Rejecting for large values of this reciprocal would incorrectly select small observations. If the reciprocal is used, reject for small values instead; this is the same upper-tail test. The size is \(0.05\) and the power is \(0.6388\), as checked above.
A rounding trap is to write \(2(1.6449)=3.2897\). Using the displayed rounded value gives \(2(1.6449)=3.2898\); using the unrounded quantile gives \(2z_{0.95}\approx3.2897\). Keeping the precision consistent avoids a false arithmetic discrepancy.
Support Differences Can Determine the Test
A ratio calculated only on the overlap of the supports can conceal the most informative observations. In particular, an observation lying outside the null support has null probability zero. Omitting that region from the rejection rule can discard power without using any of the permitted size.
Worked Example: A Null-Impossible Observation
Under \(H_0\), let \(X\sim\operatorname{Uniform}(0,1)\); under \(H_1\), let \(X\sim\operatorname{Uniform}(0,2)\). Construct a most powerful test of size \(0.10\). For a concrete test, use \(X<0.10\) for the boundary portion of the rejection region.
Method and supports. Under \(H_0\), the density is \(1\) on \((0,1)\) and zero elsewhere. Under \(H_1\), it is \(1/2\) on \((0,2)\) and zero elsewhere. On \((0,1)\), the ratio \(f_1/f_0=1/2\). On \((1,2)\), \(f_0=0\) while \(f_1=1/2\), so the ratio is infinite: reject throughout \((1,2)\).
On the common support the ratio is constant, so any subset of \((0,1)\) having null probability \(0.10\) can supply the remaining rejection probability. The chosen interval \((0,0.10)\) has null probability \(0.10\).
Check size. The rejection region is \((1,2)\cup(0,0.10)\). Under \(H_0\), the first interval has probability zero and the second has probability \(0.10\). Thus the size is \(0.10\).
Calculate power. Under \(H_1\), \(P_1(1<X<2)=1/2\), and \(P_1(0<X<0.10)=0.10/2=0.05\). These disjoint regions give total power \(0.50+0.05=0.55\).
Common mistake and correction. Comparing densities only on \((0,1)\) and ignoring \((1,2)\) could lead to rejecting only for \(X<0.10\). That rule has the correct size \(0.10\), but its power is only \(0.05\). Including the null-impossible interval raises power to \(0.55\) without increasing size. The size and power calculations confirm the correction.
Discrete Tests Need Explicit Size Calibration
In a discrete model, a cutoff may give a null rejection probability greater than the allowed size, while the next more conservative cutoff may give a probability much smaller. A randomized decision at the boundary can attain the requested size exactly. If randomization is disallowed, use a nonrandomized test with size no greater than the limit and report its actual size.
Worked Example: Calibrating a Binomial Test at a Boundary
A hypothetical quality audit records \(K\), the number of flagged units among \(3\) independently inspected units. Under \(H_0\), \(K\sim\operatorname{Binomial}(3,0.30)\); under \(H_1\), \(K\sim\operatorname{Binomial}(3,0.60)\). Find a most powerful test of size \(0.20\), allowing randomization, and calculate its power.
Method and ratio ordering. For \(k=0,1,2,3\), the likelihood ratio is \(\binom{3}{k}(0.60)^k(0.40)^{3-k}/\{\binom{3}{k}(0.30)^k(0.70)^{3-k}\}\). The binomial coefficient cancels. Replacing one failure by one success multiplies the ratio by \((0.60/0.30)/(0.40/0.70)=3.5\). Thus the ratio increases with \(k\), and larger counts are rejected first.
Under \(H_0\), \(P_0(K=3)=0.30^3=0.027\), and \(P_0(K=2)=\binom{3}{2}(0.30)^2(0.70)=3(0.09)(0.70)=0.189\). Reject for \(K=3\), and reject with probability \(\gamma\) when \(K=2\). To attain size \(0.20\), solve \(0.027+0.189\gamma=0.20\), giving \(\gamma=0.173/0.189=173/189\approx0.9153\).
Check size and power. The size is \(0.027+0.189(173/189)=0.2000\). Under \(H_1\), \(P_1(K=3)=0.60^3=0.216\) and \(P_1(K=2)=3(0.60)^2(0.40)=0.432\). Therefore power is \(0.216+0.432(173/189)=0.216+0.3954286=0.6114286\), or \(0.6114\) to four decimal places.
Common mistake and correction. Rejecting whenever \(K\geq2\) gives size \(0.027+0.189=0.216\), which exceeds \(0.20\). Rejecting only when \(K=3\) has size \(0.027\), but leaves permitted size unused. Randomizing at \(K=2\) gives the stated size exactly and is most powerful by the likelihood-ratio ordering.
Simple-Alternative Optimality Does Not Automatically Extend
The Neyman–Pearson result compares tests against a specified simple alternative. If the alternative is a set of distributions, a test most powerful against one member need not be most powerful against another. A claim of optimality over a composite alternative needs a separate argument; it does not follow merely from applying the lemma to one chosen parameter value.
Worked Example: Different Simple Alternatives Require Different Tails
Let \(X\sim N(0,1)\) under the simple null \(H_0\). Consider the composite alternative consisting of the two possibilities \(N(2,1)\) and \(N(-2,1)\). At size \(0.05\), compare the Neyman–Pearson tests for these two alternatives and assess whether either test is most powerful against both.
Against \(N(2,1)\). The likelihood ratio is \(\exp(2x-2)\), increasing in \(x\). The size-\(0.05\) test rejects for \(X>1.6448536\). As in the normal calculation, its power is \(\Phi(0.3551464)\approx0.6388\).
Against \(N(-2,1)\). The likelihood ratio is \(\exp(-2x-2)\), decreasing in \(x\). The size-\(0.05\) test instead rejects for \(X<-1.6448536\). By symmetry, its power under \(N(-2,1)\) is also \(\Phi(0.3551464)\approx0.6388\).
The two most powerful regions are opposite tails. The upper-tail test’s power against \(N(-2,1)\) is \(P(Z>3.6448536)\approx0.0001\) to four decimal places; the lower-tail test has the same small power against \(N(2,1)\). Because the likelihood ratios are strictly monotone, the respective most powerful regions are unique up to probability-zero boundaries. They cannot both be the same test. Thus neither simple-hypothesis result establishes one test as most powerful against both members of this composite alternative.
Common mistake and correction. A test derived for \(N(2,1)\) is most powerful against that specified distribution, not automatically against every alternative with a nonzero mean. State the alternative being optimized over, and claim composite optimality only when it has been separately established.
Diagnostic Checks Before Finalizing a Test
- Ratio direction: Write \(f_1/f_0\) or \(f_0/f_1\) explicitly, then verify which observations make the ratio large.
- Support: Check regions where either density is zero before cancelling terms or describing a common-support ratio.
- Size: Compute rejection probability under \(H_0\), not \(H_1\). In a discrete problem, do not assume a threshold produces the requested size.
- Power: Compute rejection probability under the stated alternative, keeping its distribution and parameters distinct from the null.
- Scope of optimality: A Neyman–Pearson test for two simple hypotheses is not automatically optimal for a composite alternative.
Check Your Understanding
For each problem, identify the relevant construction issue and verify the stated size or power under the correct hypothesis.
- Under \(H_0\), \(X\sim N(0,1)\); under \(H_1\), \(X\sim N(-1,1)\). At size \(0.05\), is the most powerful rejection region an upper or lower tail? Give its cutoff. (Answer: Lower tail, \(X<-1.6449\).)
- Under \(H_0\), \(X\sim\operatorname{Uniform}(0,3)\); under \(H_1\), \(X\sim\operatorname{Uniform}(0,6)\). At size \(0.10\), reject throughout the null-impossible part of the alternative support and use \(0<X<0.30\) on the common support. Find the power. (Answer: \(0.50+0.30/6=0.55\).)
- Suppose under \(H_0\), \(K\sim\operatorname{Binomial}(2,0.20)\), and under \(H_1\), \(K\sim\operatorname{Binomial}(2,0.50)\). For size \(0.10\), reject for \(K=2\) and randomize at \(K=1\). Find the randomization probability. (Answer: \(\gamma=(0.10-0.04)/0.32=0.1875\).)
- For the binomial test in question 3, calculate its power using the randomization probability you found. (Answer: \(0.25+0.50(0.1875)=0.34375\).)
- Under \(H_0:N(0,1)\), compare the simple alternatives \(N(1,1)\) and \(N(-1,1)\) at size \(0.05\). State the two Neyman–Pearson rejection regions and explain why the result for either one alone does not establish optimality against both. (Answer: \(X>1.6449\) for \(N(1,1)\) and \(X<-1.6449\) for \(N(-1,1)\); the simple alternatives require opposite-tail tests, so the separate results do not prove a single test most powerful against both.)