Performing Exact One-Sided Binomial Tests

Why Use an Exact One-Sided Binomial Test?

A one-proportion test asks whether a population success rate is consistent with a specified value, or whether the data support a change in a particular direction. When the number of trials is small or the null success rate is near zero or one, a normal approximation may not describe the sampling distribution accurately. An exact binomial test instead calculates probabilities directly from the binomial model under the null.

The direction of the alternative hypothesis determines which outcomes count as evidence. If the alternative says the population proportion is greater than the null value, unusually large success counts are relevant. If it says the proportion is smaller, unusually small counts are relevant. The exact one-sided p-value is the null probability of the observed count and outcomes at least as extreme in that specified direction.

The Null Model and the Tail Probability

Let \(p\) be the population proportion of successes, and suppose \(n\) binary outcomes are observed. Write \(X\) for the total number of successes and \(x_{\text{obs}}\) for the observed success count. An exact binomial test requires that the number of trials be fixed, each trial have a success or failure outcome, the trials be independent, and each trial have the same success probability. Under the null hypothesis \(H_0:p=p_0\), these assumptions give \(X\sim\operatorname{Bin}(n,p_0)\).

For an integer \(k\) from 0 to \(n\), the null probability of exactly \(k\) successes is \(P(X=k)=\binom{n}{k}p_0^k(1-p_0)^{n-k}\). The binomial coefficient counts the ways to place \(k\) successes among the \(n\) trials. Because the possible counts are separate outcomes, the probability of a set of counts is the sum of their individual probabilities.

Definition: For \(H_0:p=p_0\) against the upper-tail alternative \(H_a:p>p_0\), the exact one-sided p-value is \(P(X\ge x_{\text{obs}}\mid p=p_0)\). For \(H_0:p=p_0\) against the lower-tail alternative \(H_a:p<p_0\), it is \(P(X\le x_{\text{obs}}\mid p=p_0)\), where \(X\sim\operatorname{Bin}(n,p_0)\). The observed count is included in either tail.

\begin{aligned} p\text{-value}_{\text{upper}} &=\sum_{k=x_{\text{obs}}}^{n}\binom{n}{k}p_0^k(1-p_0)^{n-k},\\ p\text{-value}_{\text{lower}} &=\sum_{k=0}^{x_{\text{obs}}}\binom{n}{k}p_0^k(1-p_0)^{n-k}. \end{aligned}

The upper-tail formula includes \(x_{\text{obs}}\) and every larger possible count; the lower-tail formula includes \(x_{\text{obs}}\) and every smaller possible count. This inclusion matters: the p-value is not the probability of getting exactly the observed count. It measures how much null probability lies in the observed direction of departure, counting outcomes at least as extreme in that direction.

Worked Examples

Worked Example: An Upper-Tail Test With Eight Trials

Suppose a hypothetical process produces eight independent binary outcomes, each with success probability \(p_0=0.50\) under the null. Six successes are observed. Test \(H_0:p=0.50\) against \(H_a:p>0.50\) at \(\alpha=0.05\).

Check the model. The number of trials is fixed at \(n=8\), outcomes are binary, and the process is stipulated to have independent trials with the same probability under the null. Thus, under \(H_0\), \(X\sim\operatorname{Bin}(8,0.50)\). The expected number under this null is \(np_0=8(0.50)=4\) successes.

Because the alternative is upper-tailed, the p-value is the probability of six or more successes:

\begin{aligned} P(X\ge 6\mid p=0.50) &=P(X=6)+P(X=7)+P(X=8)\\ &=\binom{8}{6}(0.50)^6(0.50)^2 +\binom{8}{7}(0.50)^7(0.50)^1 +\binom{8}{8}(0.50)^8\\ &=\frac{28+8+1}{256} =\frac{37}{256} \approx 0.1445. \end{aligned}

Since \(0.1445>0.05\), do not reject \(H_0\) at the 5% significance level. The result is not sufficiently unusual under the null model to provide strong evidence, at this level, that the process success probability exceeds 0.50. This does not establish that the probability equals 0.50.

Worked Example: A Lower-Tail Test With No Successes

In a hypothetical sequence of ten independent trials, each trial has success probability \(p_0=0.30\) under the null. No successes are observed. Test \(H_0:p=0.30\) against \(H_a:p<0.30\) at \(\alpha=0.05\).

Check the model. There are a fixed ten trials, each with a success or failure outcome. The scenario specifies independent trials with a common probability under the null, so \(X\sim\operatorname{Bin}(10,0.30)\) when \(H_0\) holds.

The alternative is lower-tailed. Since the observed count is zero, only the count zero is as small as the observation:

\begin{aligned} P(X\le 0\mid p=0.30) &=P(X=0)\\ &=\binom{10}{0}(0.30)^0(0.70)^{10}\\ &=(0.70)^{10} \approx 0.02825. \end{aligned}

The p-value is about 0.02825, which is less than 0.05. Reject \(H_0\) at the 5% level. Under the specified model with success probability 0.30, the chance of observing zero successes is about 2.825%; the data therefore provide evidence in the direction of a success probability below 0.30. The p-value is not the probability that \(H_0\) is true.

Worked Example: Summing an Upper Tail With Several Counts

A hypothetical process is run for 12 independent trials, with a common null success probability \(p_0=0.20\). Six successes are observed. Test \(H_0:p=0.20\) against \(H_a:p>0.20\) at \(\alpha=0.05\).

Check the model. The scenario has a fixed number of binary trials and stipulates independence and a common success probability under the null. Thus \(X\sim\operatorname{Bin}(12,0.20)\) under \(H_0\). The upper-tail p-value includes counts 6 through 12.

\begin{aligned} P(X\ge 6) &=\sum_{k=6}^{12}\binom{12}{k}(0.20)^k(0.80)^{12-k}\\ &\approx 0.0155021+0.00332189+0.000519045\\ &\quad+0.0000576717+0.00000432538 +0.000000196608+0.000000004096\\ &\approx 0.01941. \end{aligned}

The p-value is approximately 0.01941, below 0.05. Reject \(H_0\) at the 5% significance level; the observed count provides evidence in the direction that the process success probability is greater than 0.20. The tail sum is small because six or more successes have low probability when the null success rate is 0.20.

Why the Tail Sum Is the P-Value

A p-value must represent outcomes at least as extreme as the observed result according to the alternative hypothesis. For an upper-tail alternative, every count \(k\ge x_{\text{obs}}\) is at least as large as the observed count. These events \(X=k\) are mutually exclusive, so adding their probabilities gives the probability of their union. The same reasoning applies to all counts \(k\le x_{\text{obs}}\) for a lower-tail alternative. This gives the tail-sum formulas, without any approximation to the binomial distribution.

This calculation is called exact because it uses the probabilities of the discrete binomial model directly. It does not mean the data are free of uncertainty, or that the model assumptions are guaranteed to hold. The result is exact only relative to the stated binomial model and its assumptions.

At a chosen significance level \(\alpha\), reject \(H_0\) when the p-value is at most \(\alpha\); otherwise, do not reject it. Under the null model, the chance that a test rejects is controlled at or below the chosen level. Since a binomial count takes only integer values, the possible p-values change in steps. Consequently, a test may not have a rejection probability exactly equal to \(\alpha\); it can be more conservative.

A Reliable Procedure

State the hypotheses.
Identify \(p_0\) and whether the alternative is \(p>p_0\) or \(p<p_0\). The alternative determines the direction of the tail.

Check the binomial model.
Verify a fixed number of binary trials, independence, and a common success probability. Under \(H_0\), use \(X\sim\operatorname{Bin}(n,p_0)\).

Sum the correct tail.
For \(p>p_0\), calculate \(P(X\ge x_{\text{obs}})\); for \(p<p_0\), calculate \(P(X\le x_{\text{obs}})\). Include the observed count.

Compare and interpret.
Compare the p-value with \(\alpha\), then state the decision and what the data indicate about the population proportion in context.

Common Mistakes and Why the Test Matters

Using the wrong tail. A high observed count does not automatically call for an upper-tail calculation; the alternative hypothesis sets the direction. Read \(H_a\) before summing.
Leaving out the observed count. For an upper-tail test beginning at six, sum \(P(X=6)\) and all larger-count probabilities, not just counts greater than six. The lower-tail rule likewise includes \(x_{\text{obs}}\).
Reporting only the probability of the observed count. \(P(X=x_{\text{obs}})\) is not generally the one-sided p-value. The p-value sums the probabilities of all counts at least as extreme in the specified direction.
Calling the p-value the probability that the null is true. It is calculated assuming the null model; it describes how often that model would produce a result this extreme or more extreme in the specified direction.
Assuming exact means assumption-free. The binomial tail calculation is only appropriate if the trials can reasonably be treated as fixed in number, independent, binary, and governed by a common success probability.

The exact one-sided method is useful when binomial probabilities are more reliable than a normal approximation for the sample size and null proportion at hand. Its central discipline is simple: specify the direction before calculating, use the null distribution, and sum the full tail including the observed count.

Check Your Understanding

For each question, identify the relevant tail or explain the reasoning.

For \(H_0:p=0.40\) against \(H_a:p>0.40\), 7 successes are observed in 10 trials. Which integer counts belong in the exact p-value sum?
For a lower-tail test with \(n=8\), \(p_0=0.25\), and \(x_{\text{obs}}=2\), write the exact p-value as a sum of binomial probabilities.
In a lower-tail test, why is \(P(X=x_{\text{obs}})\) alone usually not the p-value?
A binomial tail p-value is 0.08 and the chosen significance level is 0.05. State the decision and what it does, and does not, establish.
Name the assumptions required to use \(X\sim\operatorname{Bin}(n,p_0)\) for an exact one-proportion test.