Worked Interval for a Rare Health Outcome

Why Rare Outcomes Need Careful Interval Calculations

When a health outcome is uncommon, a sample may contain only a few people with the outcome—or none at all. The estimated proportion can still be calculated, but the usual Wald confidence interval may be unreliable near zero. A careful analysis checks the data conditions and compares the Wald result with the Wilson score interval, which often behaves more sensibly for small proportions.

Suppose a hypothetical screening program records whether each person experienced a specified adverse outcome during a defined follow-up period. Each observation is binary: the outcome occurred or it did not. Let \(x\) be the number of people with the outcome among \(n\) sampled people. The sample proportion is \(\hat{p}=x/n\), and \(p\) denotes the corresponding proportion in the population of interest.

Definition: A one-proportion confidence interval estimates a population proportion \(p\) from a sample in which each individual has one of two possible outcomes. The Wald interval uses the estimated standard error \(\sqrt{\hat{p}(1-\hat{p})/n}\). The Wilson score interval instead inverts a score test and does not simply add and subtract a margin based on that estimated standard error.

The methods below assume a binary outcome with a consistent definition, a random or otherwise representative sample from the target population, and independent observations. For sampling without replacement from a finite population, independence is usually approximated when the sample is a small fraction of that population. These sampling conditions matter: a more sophisticated formula cannot fix a biased sample or dependent observations.

The Wald Interval and Its Conditions

For a confidence level of 95%, the standard normal critical value is \(z^*=1.96\). The Wald interval is calculated as the sample proportion plus or minus 1.96 estimated standard errors:

\hat{p}\ \pm\ z^*\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}

This formula relies on a normal approximation to the sampling distribution of \(\hat{p}\). A common diagnostic is the success-failure condition: there should be at least 10 expected successes and 10 expected failures, assessed using the sample counts \(x\) and \(n-x\) as a practical check. For rare outcomes, \(x\) is often the limiting count. When it is small, the normal approximation can be poor even when the computed Wald interval looks tidy.

The Wald formula also has a boundary problem. Proportions must lie between 0 and 1, but the formula can produce limits outside that range. Truncating a negative lower endpoint to zero does not make the approximation that produced the interval reliable.

Worked Example: A Rare Outcome With Eight Cases

In a hypothetical random sample of 400 adults, 8 experienced a specified health outcome during follow-up. Estimate the population proportion with a 95% Wald interval, check the success-failure condition, and compare with a Wilson interval.

The estimated proportion is \(\hat{p}=8/400=0.0200\), or 2.00%. The estimated standard error and Wald margin of error are

SE=\sqrt{\frac{0.0200(1-0.0200)}{400}} =\sqrt{0.000049}=0.007000, \qquad ME=1.96(0.007000)=0.01372.

Therefore, the Wald interval is

0.0200\pm0.01372 =(0.00628,\ 0.03372).

In context, the 95% Wald interval runs from about 0.628% to 3.372% of the target population. The sample has 8 people with the outcome and 392 without it. Since \(8<10\), the usual success-failure diagnostic is not met, so the normal approximation behind the Wald interval deserves caution, even though its endpoints happen to fall inside the possible range.

For a 95% Wilson interval, use \(z^*=1.96\). Its center and half-width are

\text{center} =\frac{\hat{p}+z^{*2}/(2n)}{1+z^{*2}/n} =\frac{0.0200+3.8416/800}{1+3.8416/400} \approx0.02457,

\text{half-width} =\frac{z^*}{1+z^{*2}/n} \sqrt{\frac{\hat{p}(1-\hat{p})}{n}+\frac{z^{*2}}{4n^2}} \approx0.01440.

Thus the Wilson interval is approximately \((0.01017,\ 0.03896)\), or 1.017% to 3.896%. Both calculations use the same sample, but the Wilson interval is shifted upward and is wider here. The small number of cases warns us not to treat either approximation as exact; the Wilson result avoids some of the Wald interval's most serious boundary behavior.

Calculating the Wilson Score Interval

The Wilson interval is obtained by considering which candidate population proportions \(p\) would not be rejected by a two-sided score test at the chosen level. Its 95% formula, written for a general critical value \(z^*\), is

\frac{ \hat{p}+\dfrac{z^{*2}}{2n} \ \pm\ z^*\sqrt{\dfrac{\hat{p}(1-\hat{p})}{n}+\dfrac{z^{*2}}{4n^2}} }{ 1+\dfrac{z^{*2}}{n} }.

The minus sign gives the lower endpoint and the plus sign gives the upper endpoint. Unlike the Wald calculation, the expression adjusts both the center and the half-width. Its endpoints remain within the allowable range from 0 to 1.

A short derivation explains why the formula differs. Under a candidate value \(p\), the score statistic for testing that proportion is \((\hat{p}-p)/\sqrt{p(1-p)/n}\). Values of \(p\) retained by a two-sided score test satisfy

\frac{(\hat{p}-p)^2}{p(1-p)/n}\le z^{*2}.

Multiplying by the positive denominator and expanding gives

(n+z^{*2})p^2-(2n\hat{p}+z^{*2})p+n\hat{p}^{\,2}\le0.

The boundary values are the roots of this quadratic. Applying the quadratic formula and dividing the numerator and denominator by \(n\) gives the Wilson endpoints shown above. Thus the interval consists of candidate proportions whose score statistics are no farther than \(z^*\) from zero. This construction incorporates \(p\) in the score standard error being tested, rather than treating the estimated standard error at \(\hat{p}\) as fixed, as the Wald method does.

Worked Example: Two Cases and a Negative Wald Limit

In a different hypothetical survey, 2 of 200 randomly sampled people experienced a rare health outcome. Calculate both 95% intervals and assess the boundary and success-failure concerns.

Here \(\hat{p}=2/200=0.0100\). The Wald standard error and margin of error are

SE=\sqrt{\frac{0.0100(0.9900)}{200}} =\sqrt{0.0000495} \approx0.007036, \qquad ME=1.96(0.007036)\approx0.01379.

The Wald interval is

0.0100\pm0.01379=(-0.00379,\ 0.02379).

A negative population proportion is impossible. The interval is therefore a clear warning that the Wald approximation behaves poorly in this sample. The sample has 2 cases and 198 non-cases, so the success-failure check also fails because there are fewer than 10 cases.

For Wilson, the denominator is \(1+3.8416/200=1.019208\). The center and half-width are

\text{center} =\frac{0.0100+3.8416/400}{1.019208} =\frac{0.019604}{1.019208} \approx0.01923,

\text{half-width} =\frac{1.96}{1.019208} \sqrt{0.0000495+\frac{3.8416}{160000}} \approx0.01649.

The Wilson interval is approximately \((0.00275,\ 0.03572)\), or 0.275% to 3.572%. It stays within 0 and 1, unlike the Wald interval. This is a useful improvement in boundary behavior, not a guarantee that the sample contains enough information for a highly precise estimate.

What If the Sample Has No Cases?

If \(x=0\), then \(\hat{p}=0\). Substituting this estimate into the Wald formula makes its estimated standard error zero, so the Wald interval collapses to \((0,0)\). That result misleadingly suggests no uncertainty. Seeing no cases in a finite sample does not establish that the population proportion is exactly zero.

The Wilson interval does not collapse in this situation. With \(x=0\), its lower endpoint is 0 and its upper endpoint simplifies to \(z^{*2}/(n+z^{*2})\). The interval therefore allows positive population proportions even though no sampled person had the outcome.

Worked Example: No Observed Cases

A hypothetical random sample of 200 people contains no cases of a defined health outcome. Calculate the 95% Wald and Wilson intervals and explain their difference.

The observed proportion is \(\hat{p}=0/200=0\). For Wald,

SE=\sqrt{\frac{0(1-0)}{200}}=0, \qquad 0\pm1.96(0)=(0,\ 0).

This zero-width result is a consequence of the formula's plug-in standard error, not evidence that the population proportion has no uncertainty. The success-failure check also fails: there are zero observed cases.

For Wilson, the lower limit is 0 and the upper limit is

\frac{z^{*2}}{n+z^{*2}} =\frac{3.8416}{200+3.8416} \approx0.01885.

The 95% Wilson interval is approximately \((0,\ 0.01885)\), or 0% to 1.885%. The sample provides evidence compatible with a low population proportion, but observing no cases does not demonstrate that the proportion is zero.

Choosing and Interpreting the Result

For a relatively uncommon outcome, check the observed counts before relying on a Wald interval. If either count is small, or if a Wald endpoint falls outside 0 to 1, the Wald approximation is questionable. The Wilson score interval is generally preferable to Wald in these situations because it respects the proportion's boundaries and typically has better coverage behavior. It is still an approximate interval and still depends on appropriate sampling and independence assumptions.

The success-failure check is a diagnostic, not a mechanical guarantee. Passing it does not prove that every approximation is accurate, and failing it does not make the sample proportion meaningless. It signals that the normal approximation used by Wald may be weak. For rare health outcomes, report the method used and the observed count as well as the estimated proportion; the count reveals information that a rounded percentage can conceal.

A confidence interval describes uncertainty about a population proportion under the method's repeated-sampling interpretation. It does not give the probability that the fixed population proportion lies in this particular calculated interval, and it does not establish that an exposure caused the outcome. The interval estimates a proportion for the population represented by the sampling design.

Common mistake: “There were no cases, so the population proportion is zero; the Wald interval confirms it.” The sample proportion is zero, but the population proportion may be positive. The Wald interval collapses because its estimated standard error is zero at \(\hat{p}=0\); use an interval such as Wilson that remains informative at the boundary, and interpret it with the sampling conditions in view.

Check Your Understanding

Use these questions to check whether you can assess and calculate intervals for rare outcomes.

A random sample has \(n=300\) people, of whom 6 experienced an outcome. What are the observed success and failure counts, and does the usual success-failure diagnostic pass?
Why is a Wald interval extending below zero a warning rather than a reason simply to report zero as its lower limit?
For a sample with \(\hat{p}=0\), why does the Wald interval have zero width, and what does a Wilson interval do differently?
In the score-test derivation, which candidate values of \(p\) are retained in the Wilson interval?
Can choosing the Wilson interval correct bias caused by a sample that does not represent the target population? Explain.