Why the Neyman–Pearson Test Is Most Powerful

Why Compare Tests Outcome by Outcome?

A test with a fixed type I error rate is useful only if it can detect the alternative effectively. For two simple hypotheses, the Neyman–Pearson test ranks possible outcomes by how strongly they favor the alternative over the null, then assigns rejection to outcomes with the largest likelihood ratios. The optimality result is stronger than the observation that this rule seems reasonable: it proves that no competing test with the same size can have greater power.

The proof compares two tests at each possible outcome and then adds or integrates those comparisons over the sample space. This method also shows why a test that uses less than the allowed size cannot gain an advantage over the likelihood-ratio test.

Definition: Consider simple hypotheses $H_0$ and $H_1$, with probability mass functions or densities $f_0$ and $f_1$ relative to the same measure. A test function $\phi(x)$, with $0\leq\phi(x)\leq1$, is the probability of rejecting $H_0$ after observing $x$. Its size is $\alpha(\phi)=E_0[\phi(X)]$, the probability of rejection under $H_0$. Its power at $H_1$ is $\beta(\phi)=E_1[\phi(X)]$, the probability of rejection under $H_1$. A test is most powerful at a specified size if no other test of that size has greater power. It is most powerful at level $\alpha$ if no test with size at most $\alpha$ has greater power.

For a nonrandomized test, $\phi(x)$ is either 0 or 1: the test accepts or rejects at $x$. The more general range from 0 to 1 allows the same proof to cover tests that assign a rejection probability at a boundary. The key comparison does not depend on whether a test is randomized.

The Pointwise Inequality

Write the likelihood ratio as $\Lambda(x)=f_1(x)/f_0(x)$ wherever $f_0(x)>0$. A likelihood-ratio test rejects when $\Lambda(x)$ is large. Equivalently, for a constant $k\geq0$, define a test $\phi^*$ that rejects where $f_1(x)-k f_0(x)>0$ and does not reject where $f_1(x)-k f_0(x)<0$. Where the difference is zero, either boundary choice gives the same contribution to the argument below.

This difference form also covers outcomes with $f_0(x)=0$: if $f_1(x)>0$, then $f_1(x)-k f_0(x)>0$, so the likelihood-ratio test rejects there. Outcomes with both densities zero have no effect on either size or power.

Theorem: Let $\phi^*$ be a likelihood-ratio test with $\phi^*(x)=1$ when $f_1(x)-k f_0(x)>0$, $\phi^*(x)=0$ when $f_1(x)-k f_0(x)<0$, and any value from 0 to 1 when the difference is zero. For any competing test $\phi$, $$ \beta(\phi^*)-\beta(\phi)\ \geq\ k\bigl(\alpha(\phi^*)-\alpha(\phi)\bigr). $$ Consequently, if $\phi^*$ has size $\alpha$, it has at least as much power as every test of size $\alpha$, and at least as much power as every test of size at most $\alpha$.

The important step is to compare the tests at each possible outcome before averaging over the sample space. Where $f_1-kf_0$ is positive, $\phi^*$ rejects, so $\phi^*-\phi\geq0$. Where $f_1-kf_0$ is negative, $\phi^*$ does not reject, so $\phi^*-\phi\leq0$. In both cases their product is nonnegative. Where the difference is zero, the product is zero.

\bigl(\phi^*(x)-\phi(x)\bigr)\bigl(f_1(x)-k f_0(x)\bigr)\geq0

Integrate this inequality over the sample space. The integral of a test function against $f_1$ is its power; the integral against $f_0$ is its size. Rearranging gives the theorem’s comparison.

Compare rejection decisions at each outcome.
The likelihood-ratio test rejects on the side where $f_1-kf_0$ is positive and does not reject on the side where it is negative. Thus the product of the decision difference and $f_1-kf_0$ is nonnegative.

Integrate the pointwise inequality.
Integrating replaces each test function’s product with $f_1$ by its power, and its product with $f_0$ by its size.

Rearrange the result.
The integrated inequality is $\beta(\phi^*)-\beta(\phi)-k[\alpha(\phi^*)-\alpha(\phi)]\geq0$, which gives the stated comparison.

If the two tests have equal size, the size difference is zero, so the inequality reduces to $\beta(\phi^*)\geq\beta(\phi)$. If the likelihood-ratio test has size $\alpha$ and the competitor has size at most $\alpha$, then $\alpha-\alpha(\phi)\geq0$. Since $k\geq0$, the right-hand side is nonnegative, and the likelihood-ratio test still has at least as much power. This proves the most powerful claim.

Worked Comparisons

Worked Example: A Discrete Test and an Equal-Size Competitor

In a hypothetical classification experiment, the observed outcome is one of $A,B,C,D$. Under the simple null hypothesis, the probabilities are $0.4,0.3,0.2,0.1$, respectively. Under the simple alternative, they are $0.1,0.2,0.3,0.4$, respectively. Construct the likelihood-ratio test with size $0.3$, then compare its power with that of the test that rejects only for outcome $B$. No interest or mortality model is involved; the specified probability mass functions define the model.

Method. Calculate the likelihood ratio at each outcome, select the outcomes with the largest ratios until the required null probability is reached, and compute each test’s power under the alternative.

Likelihood ratios. Dividing the alternative probability by the null probability gives

\Lambda(A)=\frac{0.1}{0.4}=0.25,\quad \Lambda(B)=\frac{0.2}{0.3}=\frac{2}{3},\quad \Lambda(C)=\frac{0.3}{0.2}=1.5,\quad \Lambda(D)=\frac{0.4}{0.1}=4.

The largest ratios occur at $D$, then $C$. Under the null, $P_0(D)=0.1$ and $P_0(C)=0.2$, so rejecting for $C$ or $D$ gives size $0.1+0.2=0.3$. Its power is the alternative probability of those outcomes:

\beta(\phi^*)=P_1(C)+P_1(D)=0.3+0.4=0.7.

The competing test rejects only for $B$. Its size is $P_0(B)=0.3$, equal to the likelihood-ratio test’s size, and its power is $P_1(B)=0.2$. Thus the likelihood-ratio test has $0.7-0.2=0.5$ greater power at the same size.

Check. Both tests reject with null probability $0.3$, and $0.7>0.2$, as the theorem requires. The common mistake is to compare the powers without first checking the sizes; here the null probabilities confirm that the comparison is fair.

Worked Example: Continuous Outcomes and a Competing Tail

Suppose $X$ has density $f_0(x)=1$ for $0<x<1$ under $H_0$, and density $f_1(x)=2x$ for $0<x<1$ under $H_1$; each density is zero elsewhere. These are $U(0,1)$ under the null and a distribution with density $2x$ on $(0,1)$ under the alternative. The densities integrate to 1. No interest or mortality model is involved. Compare the likelihood-ratio test of size $0.5$ with the test that rejects when $X<0.5$.

Method. Order outcomes by their density ratio, find the rejection region with the stated null probability, and integrate the alternative density over each rejection region.

For $0<x<1$, the likelihood ratio is $\Lambda(x)=2x$, which increases with $x$. Thus the likelihood-ratio test rejects in the upper part of the interval. Under $H_0$, the density is 1, so the size of the region $X>0.5$ is

\alpha(\phi^*)=\int_{0.5}^{1}1\,dx=1-0.5=0.5.

Under $H_1$, its power is

\beta(\phi^*)=\int_{0.5}^{1}2x\,dx =\left[x^2\right]_{0.5}^{1} =1-(0.5)^2 =0.75.

The lower-tail competitor also has null size $\int_0^{0.5}1\,dx=0.5$. Its power is

\beta(\phi)=\int_0^{0.5}2x\,dx =\left[x^2\right]_0^{0.5} =(0.5)^2 =0.25.

The sizes match, while the powers differ by $0.75-0.25=0.50$. Check: the alternative density puts more probability toward the upper end of the interval, consistent with the likelihood ratio increasing in $x$. The common mistake avoided here is to assume that two opposite-tail tests have different sizes; they do have equal sizes in this uniform null model.

Worked Example: Correctly Matching Normal-Test Sizes

Let $X\sim N(0,1)$ under $H_0$ and $X\sim N(2,1)$ under $H_1$, where the second parameter is the variance. Compare the likelihood-ratio test that rejects for $X>1$ with a lower-tail competitor chosen to have the same size. No interest or mortality model is involved. Give probabilities to four decimal places.

Method. Derive the likelihood ratio, calculate the null size, then use the null symmetry to choose an equal-size competing region and calculate both powers under $H_1$.

The normal densities have the same variance. Their ratio is

\Lambda(x)=\frac{f_1(x)}{f_0(x)} =\exp\!\left(-\frac{(x-2)^2}{2}+\frac{x^2}{2}\right) =\exp(2x-2).

This ratio is increasing, and $\Lambda(1)=\exp(0)=1$, so the specified likelihood-ratio test rejects for $X>1$. Write $\Phi$ for the standard normal cumulative distribution function. Its size is

P_0(X>1)=1-\Phi(1)=1-0.8413=0.1587.

Under $H_1$, write $X=2+Z$, where $Z\sim N(0,1)$. Then

P_1(X>1)=P(2+Z>1)=P(Z>-1)=\Phi(1)=0.8413.

By symmetry of the null distribution, the lower-tail test $X<-1$ also has size $P_0(X<-1)=\Phi(-1)=0.1587$. Its power is

P_1(X<-1)=P(2+Z<-1)=P(Z<-3)=\Phi(-3)=0.0013.

The power difference is approximately $0.8413-0.0013=0.8400$. Check: both sizes are $0.1587$, and the likelihood-ratio test has the greater power. The competitor $X<1$ would not be an equal-size comparison: its null size is $\Phi(1)=0.8413$, not $0.1587$. Its alternative power is $\Phi(-1)=0.1587$, not $0.0013$. The equal-size lower-tail competitor is $X<-1$, whose alternative power is $\Phi(-3)$.

What the Result Does—and Does Not—Say

The theorem compares power subject to a size constraint. It does not say that the likelihood-ratio test has greater power than every conceivable test regardless of size. A test that rejects much more often under the null may also have more power simply because it rejects more readily. The meaningful comparison is at equal size, or among tests whose sizes do not exceed the same level.

The proof also explains the role of the likelihood ratio. On outcomes where the alternative density is large relative to the null density, rejecting contributes relatively much to power for each unit of size. The pointwise inequality makes that intuition exact: the likelihood-ratio test’s favorable allocation of rejection probability cannot be improved by moving rejection probability to less favorable outcomes while keeping the size fixed.

Common Mistakes and Exam Checks

Comparing power without comparing size. First calculate each test’s null rejection probability. A larger power alone does not establish a fair comparison.
Using a familiar cutoff as if it guaranteed equal size. For example, under a standard normal null, $X>1$ and $X<1$ are not equal-size regions. Their probabilities are $1-\Phi(1)$ and $\Phi(1)$. The equal-size opposite tail to $X>1$ is $X<-1$.
Reversing the likelihood ratio. With $\Lambda=f_1/f_0$, large values favor $H_1$. If the ratio is instead written $f_0/f_1$, the ordering reverses; keep the convention explicit.
Claiming strict improvement automatically. The theorem gives at least as much power. Equality can occur, for example, when competing tests differ only on outcomes that have zero probability under both hypotheses.
Forgetting the size constraint. Most powerful at level $\alpha$ means comparison against tests of size at most $\alpha$, not tests of arbitrary size.

Key takeaway: The Neyman–Pearson optimality proof rests on a pointwise inequality. Integrating it gives $\beta(\phi^*)-\beta(\phi)\geq k[\alpha(\phi^*)-\alpha(\phi)]$. Equal sizes make the right-hand side zero; a competitor with smaller size cannot make it negative.

Check Your Understanding

For each problem, identify the size constraint before comparing power. Probabilities in the answers are rounded to four decimal places where needed.

Under $H_0$, outcomes $A,B,C,D$ have probabilities $0.2,0.3,0.3,0.2$; under $H_1$, they have probabilities $0.4,0.3,0.1,0.2$. Find the likelihood ratio for each outcome. At size $0.2$, compare the power of the likelihood-ratio test with the test that rejects only for $D$. (Answer: likelihood ratios $2,1,\frac13,1$; the likelihood-ratio test rejects for $A$, with size $0.2$ and power $0.4$; rejecting only for $D$ has size $0.2$ and power $0.2$.)
Under $H_0$, $X$ has density 1 on $(0,1)$; under $H_1$, it has density $3x^2$ there. Find the likelihood-ratio test with size $0.25$, and compare its power with the test that rejects for $0<X<0.25$. (Answer: reject for $X>\sqrt{3}/2$; its size is $0.25$ and its power is $1-\frac{3\sqrt{3}}{8}\approx0.3505$; the lower-tail test has size $0.25$ and power $1/64=0.0156$.)
Suppose a likelihood-ratio test has size $0.12$ and threshold constant $k=2.5$. A competing test has size $0.08$, and the likelihood-ratio test’s power is $0.50$. Use the power inequality to give an upper bound for the competitor’s power. (Answer: $0.50-\beta(\phi)\geq2.5(0.12-0.08)=0.10$, so $\beta(\phi)\leq0.40$.)
Let $X\sim N(0,1)$ under $H_0$ and $X\sim N(1,1)$ under $H_1$. Compare the likelihood-ratio test $X>0.5$ with the equal-size competitor $X<-0.5$. (Answer: both sizes are $1-\Phi(0.5)\approx0.3085$; powers are $\Phi(0.5)\approx0.6915$ and $\Phi(-1.5)\approx0.0668$, respectively; the power difference is approximately $0.6247$.)
In a model with densities $f_0,f_1$, suppose $\phi^*$ uses threshold $k=1.5$, has size $0.20$, and power $0.62$. A competitor has size $0.16$. What does the theorem imply about its power? (Answer: $0.62-\beta(\phi)\geq1.5(0.20-0.16)=0.06$, so the competitor’s power is at most $0.56$.)

What You'll Learn