Connecting Z Tests and Score Intervals

One Procedure, Two Questions

A two-sided test asks whether the observed sample proportion is sufficiently far from one specified population proportion. A confidence interval answers a related question: which candidate population proportions would not be rejected by that test? For a one-proportion score test, carrying out this reversal gives the score confidence interval, also called the Wilson interval.

The connection is useful because it checks that a test and an interval tell a consistent story. If a null proportion is outside the score interval at a given confidence level, the corresponding two-sided score test rejects it at the matching significance level. If it is inside, the test does not reject it. The two procedures are not separate calculations with unrelated approximations: the interval is obtained by inverting the test.

Definition: For $n$ independent Bernoulli trials with a common success probability $p$, let $X$ be the number of successes and $\hat{p}=X/n$. For testing $H_0:p=p_0$, where $0<p_0<1$, the one-proportion score statistic is $$ z=\frac{\hat{p}-p_0}{\sqrt{p_0(1-p_0)/n}}. $$ A two-sided score test at significance level $\alpha$ compares $|z|$ with the standard normal critical value $z_{1-\alpha/2}$. Its matching two-sided score confidence interval contains the candidate values $p_0$ that are not rejected by this test.

From a Test to an Interval

The assumptions for the score test are a fixed number $n$ of trials, independent outcomes, and the same success probability on every trial. Under $H_0:p=p_0$, the count $X$ has a binomial distribution with success probability $p_0$. The score test uses a normal approximation to that binomial distribution. Its standard error is calculated under the candidate null value $p_0$, not by substituting $\hat{p}$ for $p_0$.

For the normal approximation to be reasonable at a particular null value, the expected success and failure counts under that null, $np_0$ and $n(1-p_0)$, should not be too small. The precise adequacy of a normal approximation depends on the setting; near the boundaries 0 and 1 it can be poor. The algebraic inversion below describes the conventional score interval, but it does not make an inadequate approximation accurate.

Write $c=z_{1-\alpha/2}$. The two-sided test does not reject a candidate $p_0$ when $|z|\leq c$. Substituting the score statistic gives an inequality involving the unknown candidate value. Solving that inequality for every $p_0$ produces the interval.

Start with the test's acceptance rule.
For a candidate null proportion $p_0$, nonrejection means $\left|\hat{p}-p_0\right|\leq c\sqrt{p_0(1-p_0)/n}$.

Square and collect terms.
Both sides are nonnegative, so squaring preserves the inequality. Rearrangement gives a quadratic inequality in $p_0$.

Find the accepted candidate values.
The roots of that quadratic are the score interval endpoints. Candidate values between the roots are not rejected by the corresponding two-sided test.

Deriving the Score Interval

Let $q=\hat{p}$ for the derivation, and suppose first that $0<q<1$. Starting from the nonrejection condition and squaring both sides gives

(q-p_0)^2 \leq \frac{c^2p_0(1-p_0)}{n}

Multiplying by $n$ and moving all terms to the left gives

(n+c^2)p_0^2-(2nq+c^2)p_0+nq^2 \leq 0.

The coefficient of $p_0^2$ is positive, so this quadratic is nonpositive between its two roots. The quadratic formula gives the roots as

p_0= \frac{2nq+c^2\ \pm\ c\sqrt{c^2+4nq(1-q)}}{2(n+c^2)}.

Rewriting the midpoint and half-width of these roots yields the score interval. This derivation also explains why the standard error in the test depends on $p_0$: retaining that dependence is what produces the quadratic and its score interval endpoints.

\frac{\hat{p}+c^2/(2n)}{1+c^2/n} \ \pm\ \frac{c}{1+c^2/n} \sqrt{\frac{\hat{p}(1-\hat{p})}{n}+\frac{c^2}{4n^2}}.

For a confidence level $1-\alpha$, use $c=z_{1-\alpha/2}$. The interval endpoints are the minus and plus versions of this expression. For an observed proportion of 0 or 1, the same formula gives the conventional score interval by substitution; boundary null tests themselves require care because their null standard error is zero.

Theorem: Under the same score-test model and normal approximation, the two-sided score confidence interval at confidence level $1-\alpha$ is the set of candidate proportions $p_0$ for which the two-sided score test at level $\alpha$ does not reject. Equivalently, a candidate null value outside the interval is rejected, and a value inside is not rejected, subject to the chosen convention at the exact critical boundary.

Worked Examples

Worked Example: A 95% Score Interval and a Candidate Null

Suppose a hypothetical sample has $n=40$ independent trials and $x=8$ successes. Thus $\hat{p}=8/40=0.20$. Check whether $p_0=0.12$ is rejected by a two-sided score test at $\alpha=0.05$, then find the matching 95% score interval.

The expected counts under $p_0=0.12$ are $40(0.12)=4.8$ successes and $40(0.88)=35.2$ failures. The expected success count is small, so the normal approximation may be questionable; the calculation below illustrates the score-test and interval connection, not a guarantee of accurate approximation. Using $c=1.96$, the statistic is

z=\frac{0.20-0.12}{\sqrt{0.12(0.88)/40}} =\frac{0.08}{\sqrt{0.00264}} \approx \frac{0.08}{0.05138} \approx 1.557.

Since $|z|=1.557<1.96$, this test does not reject $p_0=0.12$ at the 5% level. To obtain the score interval, use $c^2=3.8416$:

\frac{0.20+3.8416/(2\cdot40)}{1+3.8416/40} \ \pm\ \frac{1.96}{1+3.8416/40} \sqrt{\frac{0.20(0.80)}{40}+\frac{3.8416}{4(40)^2}} \approx 0.2263\pm0.1213.

The interval is approximately $(0.1050,\ 0.3476)$. The candidate $0.12$ lies inside it, matching the test's nonrejection. The interval estimates the population success proportion $p$; it is not a statement that there is a 95% probability that this fixed parameter lies in these particular endpoints.

Worked Example: Matching a 90% Interval to Two Tests

In a hypothetical set of $n=80$ independent trials, $x=44$ are successes, so $\hat{p}=0.55$. Consider two-sided tests of $p_0=0.45$ and $p_0=0.50$ at $\alpha=0.10$, and find the matching 90% score interval.

For $p_0=0.45$, the expected counts are 36 successes and 44 failures; for $p_0=0.50$, each expected count is 40. Both satisfy the usual expected-count check for the normal approximation. For the first candidate,

z=\frac{0.55-0.45}{\sqrt{0.45(0.55)/80}} =\frac{0.10}{\sqrt{0.00309375}} \approx \frac{0.10}{0.05562} \approx 1.798.

The 10% two-sided critical value is $c\approx1.645$. Since $1.798>1.645$, reject $p_0=0.45$. For $p_0=0.50$,

z=\frac{0.55-0.50}{\sqrt{0.50(0.50)/80}} =\frac{0.05}{\sqrt{0.003125}} \approx 0.8944.

This is less than 1.645, so do not reject $p_0=0.50$. Substituting $c=1.645$ into the score formula gives

\frac{0.55+1.645^2/(2\cdot80)}{1+1.645^2/80} \ \pm\ \frac{1.645}{1+1.645^2/80} \sqrt{\frac{0.55(0.45)}{80}+\frac{1.645^2}{4(80)^2}} \approx 0.5484\pm0.0900.

The 90% score interval is approximately $(0.4584,\ 0.6384)$. It excludes $0.45$ and includes $0.50$, exactly matching the two test decisions.

Worked Example: A Null Value Outside the Interval

A hypothetical sample records $x=36$ successes in $n=120$ independent trials, giving $\hat{p}=0.30$. At the 5% level, compare the two-sided score-test decisions for $p_0=0.20$ and $p_0=0.25$ with the 95% score interval.

At $p_0=0.20$, the expected counts are 24 successes and 96 failures; at $p_0=0.25$, they are 30 and 90. These expected counts support using the normal approximation. The test statistics are

z_{0.20}=\frac{0.30-0.20}{\sqrt{0.20(0.80)/120}} =\frac{0.10}{\sqrt{0.0013333}} \approx 2.739, $$ $$ z_{0.25}=\frac{0.30-0.25}{\sqrt{0.25(0.75)/120}} =\frac{0.05}{\sqrt{0.0015625}} \approx 1.265.

With the 5% two-sided critical value $1.96$, reject $p_0=0.20$, but do not reject $p_0=0.25$. The score interval calculation is

\frac{0.30+3.8416/(2\cdot120)}{1+3.8416/120} \ \pm\ \frac{1.96}{1+3.8416/120} \sqrt{\frac{0.30(0.70)}{120}+\frac{3.8416}{4(120)^2}} \approx 0.3062\pm0.0810.

The resulting interval is approximately $(0.2252,\ 0.3872)$. The value $0.20$ is outside, whereas $0.25$ is inside. These locations agree with their respective test decisions.

What the Match Does—and Does Not—Mean

A confidence interval can be understood as a collection of null values that survive a specified family of tests. For the score procedure, each possible null proportion supplies its own null standard error. The interval therefore is not obtained by taking the observed proportion and simply adding and subtracting a fixed margin based on its estimated standard error. Its endpoints result from solving the acceptance condition with $p_0$ still in the standard error.

The confidence level describes the long-run coverage of the interval procedure under repeated sampling when the binomial model and approximation are appropriate. It does not assign a probability to a fixed $p$ after the data have been observed. Likewise, nonrejection means the data do not provide sufficient evidence against that candidate at the chosen level; it does not establish that the null value is true.

Common Mistake

Wrong reasoning: “The null value is close to $\hat{p}$, so it must lie in the interval; the test standard error uses $\hat{p}$, just like an interval margin.”
Correction: For a score test, calculate the standard error under each candidate $p_0$, as $\sqrt{p_0(1-p_0)/n}$. The score interval is found by collecting the candidate values that satisfy the test's nonrejection inequality. Check the computed endpoints rather than judging inclusion by closeness alone.

The equivalence is specific to using the same score procedure on both sides: the same two-sided alternative, significance level, critical value, model assumptions, and standard-error rule. Comparing a score test with a different type of interval need not give matching decisions.

Key takeaway: Inverting a two-sided one-proportion score test means retaining every candidate $p_0$ for which $\left|(\hat{p}-p_0)/\sqrt{p_0(1-p_0)/n}\right|\leq z_{1-\alpha/2}$. Solving this inequality gives the score interval; values outside it are rejected by the matching test.

Check Your Understanding

Use the score-test and inversion ideas to answer the following questions.

In a score test of $p_0=0.30$ with $n=100$, which proportion belongs in the standard error: $0.30$ or the observed $\hat{p}$? Explain.
For a two-sided test at $\alpha=0.05$, what confidence level belongs to the matching score interval?
If a candidate null proportion is outside the matching score interval, what is the decision of its two-sided score test?
Why does inverting the score test produce a quadratic inequality in the candidate null proportion?
What does a 95% score interval say about repeated sampling, and what does it not say about the probability that $p$ lies in the observed interval?