Binomial Samples: Upper-Tail Most Powerful Tests

Why an Upper-Tail Test Arises

Suppose a hypothetical quality-control team records whether each of $n$ independently inspected components passes a stress test. The team wants to distinguish a specified baseline pass probability from a specified, larger pass probability. A high count of passes should favor the larger probability, but the test must also control how often it rejects the baseline model when that model is true. The likelihood ratio explains precisely why the evidence is ordered by the pass count, and the null distribution determines which cutoff meets a size requirement.

This tutorial considers a fixed sample size and two specified success probabilities. Its results are aligned with the SOA VEE Mathematical Statistics syllabus. The observations are assumed independent, each with the same success probability, and each is recorded as a success or failure.

Definition: Let $X$ be the number of successes in $n$ independent Bernoulli trials, each with success probability $p$. Then $X\sim\operatorname{Bin}(n,p)$, with $$ P_p(X=x)=\binom{n}{x}p^x(1-p)^{n-x},\qquad x=0,1,\ldots,n. $$ We test the simple null hypothesis $H_0:p=p_0$ against the simple alternative $H_1:p=p_1$, where $0<p_0<p_1<1$. A test’s size is its probability of rejection when $H_0$ is true; its power at $p_1$ is its probability of rejection when $H_1$ is true.

The restrictions $0<p_0<p_1<1$ keep both binomial models positive on every possible count $x=0,\ldots,n$, so their likelihood ratio is defined throughout the sample space. If either probability is at an endpoint, the supports may differ and this likelihood-ratio calculation needs separate treatment.

The Likelihood Ratio Orders Counts

For a possible observed count $x$, the likelihood under each hypothesis is the binomial probability of that count. Use the likelihood ratio of the alternative to the null, $L_1(x)/L_0(x)$. The binomial coefficient appears under both hypotheses and cancels.

\frac{L_1(x)}{L_0(x)} = \frac{\binom{n}{x}p_1^x(1-p_1)^{n-x}} {\binom{n}{x}p_0^x(1-p_0)^{n-x}} = \left(\frac{p_1}{p_0}\right)^x \left(\frac{1-p_1}{1-p_0}\right)^{n-x}.

To see how this ratio changes with the count, compare its value at $x+1$ with its value at $x$. Increasing the observed count by one replaces one failure factor with one success factor:

\frac{L_1(x+1)/L_0(x+1)}{L_1(x)/L_0(x)} = \frac{p_1(1-p_0)}{p_0(1-p_1)}.

This multiplier exceeds 1 because $$ p_1(1-p_0)-p_0(1-p_1)=p_1-p_0>0. $$ Thus the likelihood ratio strictly increases with $x$. The Neyman–Pearson lemma says that, for these two simple hypotheses, a test that rejects for the largest likelihood ratios is most powerful among tests of the same size. Here, that ordering means rejecting for sufficiently large counts. The upper-tail form is a consequence of the likelihood ratio, not a rule that applies to every testing problem.

Theorem: For $X\sim\operatorname{Bin}(n,p)$ and $0<p_0<p_1<1$, the likelihood ratio $L_1(x)/L_0(x)$ increases strictly with $x$. Consequently, a nonrandomized Neyman–Pearson test rejects $H_0$ when $X\ge c$, for an integer cutoff $c$.

Selecting a Cutoff for a Target Size

A cutoff determines the rejection region, but its size must be computed under the null probability $p_0$, not under the alternative. For a target size $\alpha$, the largest nonrandomized upper-tail rejection region whose size does not exceed $\alpha$ uses the smallest cutoff that meets the constraint:

c=\min\left\{k\in\{0,1,\ldots,n+1\}:P_{p_0}(X\ge k)\le\alpha\right\}.

The convention $X\ge n+1$ gives the empty rejection region, whose size is zero. Once $c$ is found, calculate the actual size and the power separately:

\text{actual size}=P_{p_0}(X\ge c) =\sum_{x=c}^{n}\binom{n}{x}p_0^x(1-p_0)^{n-x}, \qquad \text{power at }p_1=P_{p_1}(X\ge c) =\sum_{x=c}^{n}\binom{n}{x}p_1^x(1-p_1)^{n-x}.

Because $X$ takes only integer values, its possible upper-tail probabilities jump from one cutoff to the next. The actual size may therefore be strictly less than the target. If a test is required to have size exactly $\alpha$, randomization at a boundary count can fill the gap; that is distinct from choosing a nonrandomized cutoff with size no greater than $\alpha$.

Worked Cutoff Calculations

Worked Example: Checking a Target Size Before Choosing the Cutoff

Method and mistake to avoid. Use null upper-tail probabilities to select the cutoff, then calculate power under the alternative. Do not select a cutoff just because it looks close to the target size.

A hypothetical component process is tested using $n=6$ independent inspections. Let $X$ be the number that pass. Under $H_0$, $X\sim\operatorname{Bin}(6,0.25)$; under $H_1$, $X\sim\operatorname{Bin}(6,0.50)$. The target size is $\alpha=0.40$. Find the nonrandomized cutoff, its actual size, and its power.

First calculate the null tail for $X\ge2$. Under $p_0=0.25$, the probabilities of zero and one success are $$ P_{0.25}(X=0)=0.75^6=0.177978515625, $$ $$ P_{0.25}(X=1)=6(0.25)(0.75)^5=0.35595703125. $$ Therefore, $$ P_{0.25}(X\ge2)=1-0.177978515625-0.35595703125 =0.466064453125. $$ This exceeds $0.40$, so rejecting for $X\ge2$ is not permitted by the target size.

Next find the tail for $X\ge3$. The probability of exactly two successes is $$ P_{0.25}(X=2)=\binom{6}{2}(0.25)^2(0.75)^4 =15(0.0625)(0.31640625) =0.296630859375. $$ Thus, $$ P_{0.25}(X\ge3)=0.466064453125-0.296630859375 =0.16943359375. $$ The smallest eligible cutoff is $c=3$, because $X\ge2$ exceeds the target and $X\ge3$ does not. The actual size is $0.16943359375$.

Under the alternative, $X\sim\operatorname{Bin}(6,0.50)$. The probability of at least three successes is $$ P_{0.50}(X\ge3) =\frac{\binom{6}{3}+\binom{6}{4}+\binom{6}{5}+\binom{6}{6}}{2^6} =\frac{20+15+6+1}{64} =\frac{42}{64} =0.65625. $$ The power is $0.65625$. As a check, the selected null tail is below $0.40$, whereas the preceding tail is above it; the reported power uses the same cutoff under $p_1$.

Worked Example: Finding the Cutoff From Adjacent Tails

Method and mistake to avoid. Evaluate neighboring null tails around the target to identify the smallest eligible cutoff. Do not report $P_{p_1}(X\ge c)$ as the test size.

A hypothetical inspection program records passes on $n=8$ independent items. Under $H_0$, $X\sim\operatorname{Bin}(8,0.30)$; under $H_1$, $X\sim\operatorname{Bin}(8,0.60)$. For target size $\alpha=0.20$, find the cutoff, actual size, and power.

Under the null, the probabilities for counts zero through three are $$ \begin{aligned} P_{0.30}(X=0)&=0.7^8=0.05764801,\\ P_{0.30}(X=1)&=8(0.3)(0.7)^7=0.19765032,\\ P_{0.30}(X=2)&=28(0.3)^2(0.7)^6=0.29647548,\\ P_{0.30}(X=3)&=56(0.3)^3(0.7)^5=0.25412184. \end{aligned} $$ Their sum is $0.80589565$, so $$ P_{0.30}(X\ge4)=1-0.80589565=0.19410435. $$ The preceding tail is $$ P_{0.30}(X\ge3)=0.19410435+0.25412184=0.44822619. $$ Since $0.19410435\le0.20$ but $0.44822619>0.20$, choose $c=4$. The actual size is $0.19410435$.

Under $H_1$, $X\sim\operatorname{Bin}(8,0.60)$. Compute the probability of at most three successes and subtract it from one: $$ \begin{aligned} P_{0.60}(X=0)&=0.4^8=0.00065536,\\ P_{0.60}(X=1)&=8(0.6)(0.4)^7=0.00786432,\\ P_{0.60}(X=2)&=28(0.6)^2(0.4)^6=0.04128768,\\ P_{0.60}(X=3)&=56(0.6)^3(0.4)^5=0.12386304. \end{aligned} $$ The sum is $0.17367040$, so the power is $$ P_{0.60}(X\ge4)=1-0.17367040=0.82632960. $$ The power is substantially larger than the actual size, as expected for this alternative, and the null tail confirms that the cutoff respects the target.

Worked Example: A Target Near a Discrete Tail Probability

Method and mistake to avoid. Compare the target with both the proposed tail and the next larger rejection region. Do not round a tail probability first and then use the rounded value to decide whether it meets the constraint.

A hypothetical screening process tests $n=5$ independent units. Under $H_0$, $X\sim\operatorname{Bin}(5,0.40)$; under $H_1$, $X\sim\operatorname{Bin}(5,0.70)$. The target size is $\alpha=0.32$. Find the nonrandomized cutoff, actual size, and power.

Under $H_0$, the probability of at most two successes is $$ \begin{aligned} P_{0.40}(X=0)&=0.6^5=0.07776,\\ P_{0.40}(X=1)&=5(0.4)(0.6)^4=0.25920,\\ P_{0.40}(X=2)&=10(0.4)^2(0.6)^3=0.34560. \end{aligned} $$ Thus, $$ P_{0.40}(X\ge3)=1-(0.07776+0.25920+0.34560)=0.31744. $$ This is no greater than $0.32$. Including count two would instead give $$ P_{0.40}(X\ge2)=0.31744+0.34560=0.66304, $$ which exceeds the target. Therefore $c=3$, and the actual size is $0.31744$.

For the alternative $p_1=0.70$, the probability of at most two successes is $$ \begin{aligned} P_{0.70}(X=0)&=0.3^5=0.00243,\\ P_{0.70}(X=1)&=5(0.7)(0.3)^4=0.02835,\\ P_{0.70}(X=2)&=10(0.7)^2(0.3)^3=0.13230. \end{aligned} $$ The power is $$ P_{0.70}(X\ge3)=1-(0.00243+0.02835+0.13230)=0.83692. $$ The actual size is just below the target, and the alternative tail uses the same cutoff $c=3$.

Common Exam Trap: Using the Wrong Tail or Probability

The cutoff is chosen using the null distribution because size is defined under $H_0$. Power is calculated only after the cutoff is set, using the alternative distribution. Reversing those roles can produce a plausible-looking but incorrect answer.

For example, in the six-inspection problem, a tempting calculation is to use $X\ge2$ because its null tail might appear close enough to $0.40$. The actual calculation gives $$ P_{0.25}(X\ge2)=0.466064453125>0.40. $$ That region is too large. The correction is to increase the cutoff to three, giving size $0.16943359375$, and then calculate power under $p_1=0.50$, giving $0.65625$. Reporting the power for $X\ge2$ would answer a different test.

A useful check is to evaluate the tail at the selected cutoff and at the immediately preceding cutoff. For a genuine nonrandomized cutoff, the selected tail is at most $\alpha$, while the preceding tail is greater than $\alpha$ (apart from boundary conventions such as an empty rejection region). This verifies that the selected region is the largest eligible upper tail.

Key takeaway: With $0<p_0<p_1<1$, the binomial likelihood ratio increases with the success count, so the most powerful rejection region is an upper tail. Choose its cutoff using $P_{p_0}(X\ge c)\le\alpha$, then report actual size under $p_0$ and power under $p_1$.

Check Your Understanding

For each problem, state the nonrandomized upper-tail cutoff, actual size, and power when requested. Treat all trials as independent Bernoulli observations.

A hypothetical test uses $n=4$, $p_0=0.25$, $p_1=0.50$, and target size $0.15$. Find the cutoff, actual size, and power. (Answer: Reject for $X\ge3$; size $0.05078125$; power $0.3125$.)
A screening process uses $n=5$, $p_0=0.30$, $p_1=0.60$, and target size $0.25$. Find the cutoff, actual size, and power. (Answer: Reject for $X\ge3$; size $0.16308$; power $0.68256$.)
For $n=7$, $p_0=0.20$, $p_1=0.40$, and target size $0.15$, find the cutoff, actual size, and power. (Answer: Reject for $X\ge3$; size $0.148032$; power $0.580096$.)
For $n=4$, $p_0=0.30$, and $p_1=0.60$, calculate the likelihood ratio $L_1(x)/L_0(x)$ at $x=3$, and state whether it is larger or smaller than the ratio at $x=2$. (Answer: $32/7\approx4.5714$; it is larger, since the successive-count multiplier is $3.5$.)

What You'll Learn