Why Wald Intervals May Not Match Score Tests

Why Compare an Interval With a Test?

A confidence interval and a hypothesis test can answer closely related questions. A two-sided test at significance level \(\alpha\) asks whether a particular null value is compatible with the data. A two-sided confidence interval with confidence level \(1-\alpha\) gives a range of null values that are not rejected by the corresponding procedure. When the procedures are built from the same test, these answers agree: a null value is rejected exactly when it falls outside that test's interval.

For a population proportion \(p\), however, the commonly taught Wald interval is not obtained by inverting the score test. The Wald interval estimates its standard error using the observed proportion \(\hat{p}\); the score test calculates its standard error using the null value \(p_0\). Those different choices matter most when the sample is small or \(\hat{p}\) is near 0 or 1.

Model and procedures: Suppose \(X\) counts successes in \(n\) independent Bernoulli trials, each with the same success probability \(p\), so \(X\sim\operatorname{Binomial}(n,p)\) and \(\hat{p}=X/n\). The Wald interval uses the normal approximation and the estimated standard error \(\sqrt{\hat{p}(1-\hat{p})/n}\). A two-sided score test of \(H_0:p=p_0\) uses the null standard error \(\sqrt{p_0(1-p_0)/n}\). Both are nominal normal-approximation procedures; their accuracy depends on the approximation being reasonable.

The Wald Interval and the Score Test

For a confidence level of \(1-\alpha\), let \(z^*\) be the positive standard normal critical value with probability \(\alpha/2\) above it. The usual Wald interval is

\hat{p}\ \pm\ z^*\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}.

For a proposed null proportion \(p_0\), the score statistic for testing \(H_0:p=p_0\) against \(H_a:p\ne p_0\) is

z_{\text{score}}=\frac{\hat{p}-p_0}{\sqrt{p_0(1-p_0)/n}}.

At significance level \(\alpha\), this two-sided score test rejects when \(\lvert z_{\text{score}}\rvert>z^*\). The key difference is the standard error: the test uses the variability expected if \(p_0\) were true, while the Wald interval uses variability estimated from the observed sample. Consequently, checking whether \(p_0\) lies inside the Wald interval does not, in general, reproduce the score-test decision.

Worked Example: An Extreme Sample Gives Conflicting Decisions

Suppose a hypothetical sequence of 400 independent trials has 8 successes. Test \(H_0:p=0.035\) against a two-sided alternative at \(\alpha=0.05\), and compare the decision with the 95% Wald interval. Use \(z^*=1.96\).

The observed proportion is \(\hat{p}=8/400=0.020\). For the Wald interval, the estimated standard error is \(\sqrt{0.020(0.980)/400}=\sqrt{0.000049}=0.00700\). Thus the interval is \(0.020\pm1.96(0.00700)=0.020\pm0.01372\), or approximately \((0.00628,0.03372)\). Since \(0.035\) is above the upper endpoint, the Wald interval excludes the null value.

For the score test, the standard error under the null is \(\sqrt{0.035(0.965)/400}=\sqrt{0.00008444}\approx0.009189\). Therefore, \(z_{\text{score}}=(0.020-0.035)/0.009189\approx-1.632\). Since \(1.632<1.96\), the two-sided score test does not reject at the 5% level; its normal-approximation p-value is about 0.103.

The decisions conflict: the Wald interval excludes \(0.035\), but the score test does not reject it. The reason is that the observed proportion is low, making the Wald standard error smaller than the standard error calculated under this null value. Inverting the score test gives an interval of approximately \((0.0102,0.0390)\), which includes \(0.035\).

The normal approximation is reasonably supported under the null here: the expected counts are \(400(0.035)=14\) successes and \(400(0.965)=386\) failures. That does not make the two methods identical; it simply makes the score test's normal approximation more defensible for this illustration.

Why the Score Interval Matches the Score Test

The score interval is formed by collecting all values \(p_0\) that the two-sided score test would not reject. This interval is also called the Wilson score interval. Its connection to the test follows directly from solving the test's acceptance condition for \(p_0\).

Write \(c=z^*\). A null value is not rejected when the absolute score statistic is no greater than \(c\):

\left|\frac{\hat{p}-p_0}{\sqrt{p_0(1-p_0)/n}}\right|\le c.

For \(0<p_0<1\), square both sides and multiply by the positive quantity \(np_0(1-p_0)\). This gives

n(\hat{p}-p_0)^2\le c^2p_0(1-p_0).

Expanding and bringing all terms to the left produces a quadratic inequality in \(p_0\):

(n+c^2)p_0^2-(2n\hat{p}+c^2)p_0+n\hat{p}^{\,2}\le0.

Because the coefficient of \(p_0^2\) is positive, the inequality holds between the two roots. The quadratic formula gives the endpoints as

\frac{2n\hat{p}+c^2\ \pm\ c\sqrt{c^2+4n\hat{p}(1-\hat{p})}}{2(n+c^2)}.

Rewriting the same roots as a center plus or minus a half-width gives the score interval:

\frac{\hat{p}+\frac{c^2}{2n}}{1+\frac{c^2}{n}} \ \pm\ \frac{c}{1+\frac{c^2}{n}} \sqrt{\frac{\hat{p}(1-\hat{p})}{n}+\frac{c^2}{4n^2}}.

This derivation shows why the score interval and the two-sided score test agree: its endpoints are precisely the null proportions where the score statistic reaches the critical value. A score test rejects \(p_0\) exactly when \(p_0\) lies outside this interval, apart from the endpoint convention at equality. That is not the construction used for the Wald interval.

Worked Example: The Reverse Disagreement

Consider a hypothetical sample with \(n=10\) independent trials and \(X=5\) successes, so \(\hat{p}=0.50\). Test \(H_0:p=0.20\) against a two-sided alternative at \(\alpha=0.05\). The Wald interval and score test can now disagree in the opposite direction.

The Wald standard error is \(\sqrt{0.50(0.50)/10}=\sqrt{0.025}\approx0.1581\). The 95% Wald interval is \(0.50\pm1.96(0.1581)=0.50\pm0.3099\), or approximately \((0.1901,0.8099)\). It includes \(0.20\), so the interval-based Wald decision would not reject that value.

Under the null, the score standard error is \(\sqrt{0.20(0.80)/10}=\sqrt{0.016}\approx0.1265\). The score statistic is \((0.50-0.20)/0.1265\approx2.372\). Since \(2.372>1.96\), the nominal two-sided score test rejects \(H_0\) at the 5% level; its normal-approximation p-value is about 0.0177. The 95% score interval is approximately \((0.237,0.763)\), which excludes \(0.20\), as it should.

This is an illustration of how the formulas can disagree, not an endorsement of the approximation in this particular small sample. Under \(p_0=0.20\), the expected number of successes is only \(10(0.20)=2\), so the usual normal approximation is questionable. A careful analysis would flag that limitation rather than treating the nominal score-test p-value as reliably calibrated.

When the Decisions Are Similar

The two methods can also give the same decision while reporting different endpoints. For example, suppose a hypothetical sample has \(n=100\) independent trials and \(X=60\) successes. Test \(H_0:p=0.50\) against a two-sided alternative at the 5% level. Here \(\hat{p}=0.60\), and the expected counts under the null are 50 successes and 50 failures, so the normal approximation is reasonable.

The Wald standard error is \(\sqrt{0.60(0.40)/100}\approx0.0490\). The 95% Wald interval is \(0.60\pm1.96(0.0490)\), or approximately \((0.504,0.696)\), which excludes \(0.50\).

The score standard error under the null is \(\sqrt{0.50(0.50)/100}=0.0500\), giving \(z_{\text{score}}=(0.60-0.50)/0.0500=2.00\). The two-sided normal-approximation p-value is about 0.0455, so the score test rejects at 5%. Its score interval is approximately \((0.502,0.691)\), which also excludes \(0.50\). The decisions agree, but the intervals are not the same.

What to Check in Practice

A mismatch is not a mathematical contradiction. Each procedure uses a different estimated or hypothesized standard error, and only the score interval is constructed to invert the score test. In small or extreme samples, the two standard errors can be sufficiently different to change whether a particular value is included.

The Wald interval has additional warning signs near the boundaries. If \(\hat{p}=0\) or \(\hat{p}=1\), its estimated standard error is zero, so the formula produces a degenerate interval even though uncertainty about \(p\) remains. In less extreme cases, the untruncated Wald endpoints can fall outside the possible range from 0 to 1. Truncating such endpoints does not turn the Wald interval into a score interval or guarantee agreement with a score test.

When reporting results, name the procedure used and assess the approximation conditions. If a score test is used, compare the observed counts with what the null predicts and consider whether a normal approximation is credible. Do not infer that a null value is true merely because a test does not reject it; a non-rejection means only that the data do not cross the specified rejection threshold under that procedure.

Key takeaway: A two-sided score test and its score interval are matched by construction. A Wald interval uses \(\hat{p}\) in its standard error, whereas the score test uses \(p_0\); therefore, checking a null value against the Wald interval need not give the score-test decision.

Common Mistake

A common mistake is to say, “The 95% Wald interval contains \(p_0\), so the 5% score test cannot reject \(H_0:p=p_0\).” That conclusion assumes the interval was obtained by inverting that test. The Wald interval was not: its standard error is based on \(\hat{p}\), while the score statistic's standard error is based on \(p_0\). To check consistency with a score test, use the score interval or calculate the score statistic directly.

Check Your Understanding

Answer these questions using the distinctions between the two procedures.

Which proportion appears in the Wald interval's standard error, and which appears in the score test's standard error?
For a two-sided score test at significance level \(\alpha\), what confidence level should its matching score interval have?
Why can a Wald interval exclude a null value that the score test does not reject?
What is a warning sign when the sample has no observed successes or no observed failures?
In the sample with 5 successes out of 10 and null value \(p_0=0.20\), why should the nominal score-test result be interpreted cautiously?