Bootstrap Intervals for a Proportion

Why Bootstrap a Proportion?

Many questions concern the fraction of a population with a particular characteristic: the proportion of patients who improve, of parts that fail inspection, or of households that use a service. A sample gives an estimate of that proportion, but a different sample would usually give a different estimate. A bootstrap interval uses resampling from the observed data to represent some of that sampling variability.

For a binary outcome, each observation is either a success or a failure, where “success” means the outcome of interest and does not necessarily mean a desirable result. The population proportion is the probability of success in the population or sampling process. The sample proportion is the observed fraction of successes.

The method assumes the observed units can reasonably be treated as independent, representative observations from the target population or sampling process. The bootstrap does not correct a biased sample, account automatically for dependence, or turn a convenience sample into a random one. These design and data-quality questions matter before resampling begins.

How a Percentile Bootstrap Interval Is Formed

In each bootstrap resample, calculate the resampled proportion \(\hat{p}^{*}\). Repeating this many times produces a bootstrap distribution of proportions, which approximates the variation in the statistic under repeated sampling from the empirical distribution of the observed data. A percentile interval uses the lower and upper quantiles of that distribution.

A quantile convention is needed because the bootstrap distribution for a proportion is discrete. Here, a quantile at probability \(q\) is the smallest value whose cumulative probability is at least \(q\). With a finite number \(B\) of simulated resamples, software estimates these quantiles from the simulated values; results can vary slightly with the resampling run. When convenient, the exact conditional bootstrap distribution can instead be calculated.

Conditional on the observed data, each bootstrap draw is a success with probability \(\hat{p}\), because the resampling pool contains a fraction \(\hat{p}\) successes. The \(n\) draws are independent, since each is made with replacement. Thus the number of bootstrap successes \(K^{*}\) has a binomial distribution, and the resampled proportion is \(K^{*}/n\).

This result gives a direct way to understand the bootstrap distribution for binary data. It also makes clear that its possible values are only \(0,1/n,2/n,\ldots,1\). The bootstrap is resampling the outcomes actually observed; it is not generating new kinds of observations absent from the sample.

What the Interval Does—and Does Not—Say

A confidence interval is a procedure for estimating an unknown parameter. In repeated use under the relevant sampling assumptions, a 95% confidence procedure is designed to cover the fixed parameter about 95% of the time. A bootstrap percentile interval approximates such a procedure using the empirical distribution, but its actual coverage can differ from 95%, especially with small samples or discrete, highly skewed bootstrap distributions.

The interval's endpoints are constrained by the resampling data and by the sample size. For instance, with \(n=10\), bootstrap proportions change in steps of 0.1. With few observations, the discreteness can make the interval jump noticeably as the sample changes. A reported percentile interval should therefore be interpreted as an estimate of uncertainty, not as a guarantee that the true proportion falls between its endpoints.

Other bootstrap constructions, such as basic, studentized, and BCa intervals, use the bootstrap distribution differently. The binary-data structure still applies: ordinary resampling cannot produce a success if none was observed, or a failure if none was observed. A more elaborate interval formula cannot recover information that the resampling pool does not contain.

Boundary Data: When Every Resample Is Identical

A boundary problem occurs when the observed sample proportion is zero or one. If \(x=0\), every observed outcome is a failure. Resampling those outcomes with replacement still gives only failures. If \(x=n\), every observed outcome is a success, so every resample still consists only of successes. The bootstrap distribution then has no variation at all.

Consequently, the ordinary percentile interval is \([0,0]\) when no successes are observed and \([1,1]\) when all observations are successes. Basic or BCa procedures based on the same degenerate bootstrap information can also encounter serious problems. These zero-width intervals should not be taken as evidence that the population proportion is known with certainty. They reflect the limitation of resampling an empirical distribution that contains only one outcome category.

A Short Justification of the Bootstrap Distribution

The binomial form follows directly from the resampling rule. Label each draw as a success indicator, equal to 1 for a success and 0 for a failure. Given the observed data, a draw selects one of the \(n\) observed units uniformly. Exactly \(x\) of those units are successes, so the conditional probability that the draw is a success is \(x/n=\hat{p}\). Since draws are independent under sampling with replacement, the total number of successes across \(n\) draws is the sum of \(n\) independent success indicators with common success probability \(\hat{p}\). By the definition of the binomial distribution, that total has distribution \(\operatorname{Binomial}(n,\hat{p})\). Dividing the total by \(n\) gives the bootstrap proportion.

This justification also explains the boundary failure. If \(\hat{p}=0\), each indicator is zero with probability one, so their sum is zero with probability one. If \(\hat{p}=1\), each indicator is one with probability one, so their sum is \(n\) with probability one. No amount of repeating the same ordinary resampling scheme changes those facts.

Practical Checks and Common Mistakes

Before reporting a bootstrap interval for a proportion, state what counts as a success, identify the number of observations and successes, and check the sample design. Then confirm that bootstrap resampling reproduces the original sampling units and dependence structure. For a simple independent binary sample, resampling individual outcomes is appropriate; for clustered or paired data, the resampling unit must reflect that design.

Common mistake: “Every bootstrap resample had zero successes, so there is no uncertainty and the population proportion must be zero.” The first part describes the resampling output; the conclusion does not follow. Every resample had zero successes because the original empirical pool had none. The population can still have a positive success probability, and a method designed to express uncertainty for boundary samples is needed.

A second mistake is to treat a bootstrap interval as a probability statement about the fixed parameter. Instead, describe the interval as the result of a procedure intended to have a specified long-run coverage under its assumptions. For bootstrap intervals, also acknowledge that the procedure is an approximation and can behave poorly when data are sparse or at a boundary.