What the Neyman–Pearson Guarantee Does—and Does Not—Say
A likelihood-ratio rule can identify the most powerful test when the null specifies one distribution and the alternative specifies one distribution. But many practical questions specify a range of parameter values: for example, a null that includes every success probability at or below a threshold. In that setting, choosing a likelihood ratio for one pair of parameter values is not, by itself, enough. The test must also control its rejection probability across the entire null set, and its performance may need to be assessed across an entire alternative set.
The distinction is between a guarantee for two simple hypotheses and a testing problem with one or both hypotheses composite. The Neyman–Pearson lemma is a powerful result, but its optimality comparison is made between tests for a particular pair of probability distributions. Extending that comparison to parameter families requires additional arguments.
For a simple null distribution \(f_0\), the test's size is its probability of rejection under that null, \(E_0[\phi(X)]\). For a composite null indexed by a parameter \(\theta\in\Theta_0\), its size is the largest rejection probability over the null:
A level-\(\alpha\) test must have size no greater than \(\alpha\). Its power at a specified alternative value \(\theta\) is \(E_\theta[\phi(X)]\). A test is uniformly most powerful (UMP) at level \(\alpha\) for a composite alternative if it has size at most \(\alpha\) and its power is at least that of every other level-\(\alpha\) test at every parameter value in the alternative. A test can be most powerful at one alternative value without being UMP for a whole set of alternatives.
The Simple-Hypothesis Result
Suppose \(H_0\) and \(H_1\) specify densities or probability mass functions \(f_0\) and \(f_1\), respectively, with respect to a common measure. A test of size at most \(\alpha\) is most powerful against \(H_1\) when it rejects for large values of the likelihood ratio \(f_1(x)/f_0(x)\), with randomization at a boundary if needed to attain size \(\alpha\). Values with \(f_0(x)=0\) and \(f_1(x)>0\) have an infinite ratio and are treated as strong evidence for \(H_1\).
The result compares the proposed test with every other test satisfying the same size constraint under \(f_0\). It does not say that one likelihood ratio automatically solves a problem involving multiple null distributions or multiple alternative distributions. The hypotheses must first be simple for the direct guarantee to apply.
Worked Example: A Randomized Most Powerful Test for Two Simple Hypotheses
Let \(X\sim\operatorname{Binomial}(2,p)\). Test \(H_0:p=0.25\) against \(H_1:p=0.75\) at size \(\alpha=0.2500\). Find the Neyman–Pearson test and its power. The model is two independent Bernoulli trials with the stated success probability under each hypothesis.
Method. Compare the likelihoods at each possible count \(x=0,1,2\). Since the hypotheses each specify one value of \(p\), they are simple. The likelihood ratio is \[ \frac{f_1(x)}{f_0(x)} =\frac{\binom{2}{x}(0.75)^x(0.25)^{2-x}} {\binom{2}{x}(0.25)^x(0.75)^{2-x}} =3^{2x-2}. \] The ratios for \(x=0,1,2\) are \(1/9\), \(1\), and \(9\). Thus the evidence for \(H_1\) increases with \(x\), and the test rejects first at \(x=2\).
Under \(H_0\), the probabilities of \(X=0,1,2\) are \(0.5625\), \(0.3750\), and \(0.0625\). Rejecting whenever \(X=2\) uses size \(0.0625\). To reach size \(0.2500\), randomize at \(X=1\) with probability \(\gamma\), where \[ 0.0625+\gamma(0.3750)=0.2500. \] Therefore \(\gamma=0.1875/0.3750=0.5000\). The test rejects for \(X=2\), rejects with probability \(0.5000\) for \(X=1\), and accepts for \(X=0\).
Under \(H_1\), the probabilities of \(X=0,1,2\) are \(0.0625\), \(0.3750\), and \(0.5625\). Its power is \[ P_{0.75}(X=2)+0.5000P_{0.75}(X=1) =0.5625+0.5000(0.3750)=0.7500. \] Check: The size is \(0.0625+0.5000(0.3750)=0.2500\), as required. Randomization at the tied likelihood ratio \(1\) is necessary to use the full size without rejecting at the less favorable outcome \(X=0\). A common mistake is to reject only for \(X=2\) and report that as a size-\(0.2500\) test; its size is only \(0.0625\).
Why the Lemma Holds for a Simple Pair
The key comparison explains both the strength and the limit of the result. Let \(\phi^*\) be a likelihood-ratio test of size \(\alpha\), and let \(\phi\) be any competing test with size at most \(\alpha\). At every data value, the likelihood-ratio rule places rejection probability where \(f_1-cf_0\) is positive and acceptance probability where it is negative. At a boundary where it is zero, randomization does not change the comparison. Consequently,
Integrating over the data gives
The final inequality follows because \(E_0[\phi^*(X)]=\alpha\) and \(E_0[\phi(X)]\leq\alpha\). Hence no competing test of size at most \(\alpha\) has greater power against this particular \(f_1\). The proof depends on a single \(f_0\) and a single \(f_1\): the pointwise comparison uses their likelihood ratio. With a composite null, there is no one null density in that argument unless a further result justifies using a particular one. With a composite alternative, there is no one alternative density for which the comparison automatically proves optimality at every alternative value.
Composite Nulls: Control the Whole Null Set
For a composite null, a test is not level \(\alpha\) merely because its rejection probability is at most \(\alpha\) at one convenient null parameter value. It must be at most \(\alpha\) at every value allowed by the null. Often the largest rejection probability occurs at a boundary value, but that must be established from the model and the test; it cannot simply be assumed.
Worked Example: A Simple-Null Calibration That Fails for a Composite Null
Let \(X\sim\operatorname{Bernoulli}(p)\), so \(X=1\) is a success. Test \(H_0:p\leq0.40\) against the simple alternative \(H_1:p=0.70\), with required size at most \(\alpha=0.2000\). Find a valid boundary-calibrated test, and check why calibrating only at \(p=0.20\) fails.
Method. Write \(\phi(1)=r_1\) and \(\phi(0)=r_0\). At null value \(p\), the rejection probability is \(pr_1+(1-p)r_0\). For the proposed upper-outcome test, set \(r_0=0\). Its rejection probability is \(pr_1\), which is largest over \(p\leq0.40\) at \(p=0.40\). To make the supremum equal to \(0.2000\), choose \(0.40r_1=0.2000\), giving \(r_1=0.5000\).
The test therefore rejects with probability \(0.5000\) on observing \(X=1\), and never rejects on observing \(X=0\). Its size is \[ \sup_{p\leq0.40}P_p(\text{reject}) =\sup_{p\leq0.40}(0.5000p)=0.2000. \] Its power at \(p=0.70\) is \(0.5000(0.70)=0.3500\).
If instead one calibrates at \(p=0.20\), the rule “reject when \(X=1\)” has size \(0.2000\) at that one parameter value. But over the stated null its largest rejection probability is \(0.40\), attained at \(p=0.40\), so it is not a level-\(0.2000\) test. Check: The boundary-calibrated rule has size \(0.2000\) and power \(0.3500\). The mistake is treating a single interior null value as if it controlled the entire composite null.
Composite Alternatives: Optimality May Conflict
A composite alternative creates a different difficulty. A test may be most powerful against one member of the alternative set and perform poorly against another. A UMP test must meet the relevant power comparison at every alternative value, not just at a selected representative value. In some problems no such test exists. One may then need to specify a different goal, such as maximizing the minimum power over the alternative set, but that is a separate optimization criterion and not a consequence of the Neyman–Pearson lemma.
Worked Example: Two Alternatives Favor Opposite Rejection Rules
Let \(X\sim\operatorname{Bernoulli}(p)\) from one trial. Test the simple null \(H_0:p=0.50\) against the composite alternative \(H_1:p\in\{0.10,0.90\}\), at size \(\alpha=0.2000\). Determine the most powerful size-\(0.2000\) test for each simple alternative and show whether one test attains both powers.
Method. Let \(r_0=\phi(0)\) and \(r_1=\phi(1)\). Under the null, the size is \(0.5000r_0+0.5000r_1\), so the constraint is \(r_0+r_1\leq0.4000\).
For \(p=0.10\), power is \(0.9000r_0+0.1000r_1\). Rejection at \(X=0\) contributes more power per unit of null rejection probability than rejection at \(X=1\), so the maximum uses \(r_0=0.4000\), \(r_1=0\). Its power is \(0.9000(0.4000)=0.3600\). For \(p=0.90\), power is \(0.1000r_0+0.9000r_1\); the maximizing test reverses the allocations, using \(r_0=0\), \(r_1=0.4000\), also with power \(0.3600\).
These are incompatible tests: the first requires all available rejection probability at \(X=0\), while the second requires it at \(X=1\). No one test can attain power \(0.3600\) at both alternatives. In fact, the sum of the two powers for any test is \[ (0.9000r_0+0.1000r_1)+(0.1000r_0+0.9000r_1) =r_0+r_1\leq0.4000. \] Thus its smaller power across the two alternatives cannot exceed \(0.2000\). The symmetric test \(r_0=r_1=0.2000\) has power \(0.2000\) at each and attains that maximin value. Check: The simple alternatives each have a most powerful test with power \(0.3600\), but there is no UMP test achieving both; the symmetric maximin test has power \(0.2000\) at each. The trap is to take a test optimized for one simple alternative and call it uniformly most powerful for the composite alternative.
A Practical Reasoning Checklist
When a testing problem includes parameter ranges, separate the required claims before choosing a rejection rule. The following sequence prevents a simple-hypothesis result from being applied beyond its scope.
Write down which parameter values belong to the null and which belong to the alternative. Decide whether either hypothesis specifies more than one distribution.
For a composite null, calculate or bound rejection probability throughout the null set. If you use a boundary value, justify why it controls the supremum for the test being considered.
For a simple alternative, compare power at that distribution. For a composite alternative, a UMP claim requires comparisons at every alternative value; a maximin or average-power goal is different.
A likelihood-ratio test for selected parameter values is optimal for that pair. Further work is needed to show it controls a composite null and, if claimed, is optimal across a composite alternative.
A frequent exam error is to form a likelihood ratio using one null value and one alternative value, then assert that the resulting test is level \(\alpha\) and most powerful for the entire composite problem. Those are separate claims. Check size over the full null first; then establish the required power comparison over the full alternative. The Neyman–Pearson guarantee is exact and useful, but it is not a substitute for those checks. This distinction is aligned with the SOA VEE Mathematical Statistics syllabus.
Check Your Understanding
For each problem, identify the relevant size or power calculation before interpreting the result.
- Let \(X\sim\operatorname{Bernoulli}(p)\). For the simple hypotheses \(H_0:p=0.30\) and \(H_1:p=0.70\), find the non-randomized most powerful size-\(0.3000\) test and its power. (Answer: Reject when \(X=1\); size \(0.3000\), power \(0.7000\).)
- Let \(X\sim\operatorname{Bernoulli}(p)\), with \(H_0:p\leq0.25\) and \(H_1:p=0.60\). A test rejects with probability \(r\) when \(X=1\) and never when \(X=0\). Find \(r\) for level \(0.1000\), then find its power. Why would setting \(r=1\) fail? (Answer: \(r=0.4000\), power \(0.2400\); \(r=1\) gives composite-null size \(0.2500\).)
- One Bernoulli observation has null parameter \(p=0.50\), and alternatives \(p=0.20\) and \(p=0.80\). At size \(0.2000\), find the largest possible minimum power over these two alternatives. Give a test attaining it. (Answer: \(0.2000\), attained by rejecting with probability \(0.2000\) for either observation.)
- Suppose \(X\sim N(\mu,1)\), where the second argument is the variance, and test \(H_0:\mu\leq0\) against the simple alternative \(\mu=1\). A proposed test rejects when \(X>1.6449\). Using \(P(Z>1.6449)=0.0500\) for \(Z\sim N(0,1)\), check its composite-null size and calculate its power at \(\mu=1\). (Answer: Size \(0.0500\), attained at \(\mu=0\); power \(P(Z>0.6449)\approx0.2596\).)