Normal and Basic Bootstrap Intervals

Why Bootstrap Intervals Use Replicate Estimates

A point estimate summarizes a sample, but it does not show how much the estimate might vary across samples. A confidence interval adds an uncertainty range. When the estimator’s sampling distribution is difficult to derive directly, bootstrap resampling provides an empirical approximation: repeatedly resample the observed data with replacement, calculate the estimate for each resample, and examine the resulting replicate estimates.

This tutorial focuses on two intervals built from those replicates. The normal-approximation interval uses the bootstrap standard error as a measure of spread and assumes an approximately normal sampling distribution for the estimator. The basic bootstrap interval uses bootstrap quantiles and reflects them around the original estimate. The two approaches use the same resampling output but make different calculations and distributional assumptions.

Definition: Suppose the observed sample produces an estimate $\hat{\theta}$. In bootstrap replicate $b$, resample the observed data with replacement, using the original sample size, and recalculate the estimate to obtain $\hat{\theta}^{*(b)}$. The resulting values $\hat{\theta}^{*(1)},\ldots,\hat{\theta}^{*(B)}$ are the bootstrap replicate estimates. Their estimated standard error is $$ \widehat{\operatorname{SE}}_{\!*}(\hat{\theta})= \sqrt{\frac{1}{B-1}\sum_{b=1}^{B} \left(\hat{\theta}^{*(b)}-\bar{\hat{\theta}}^*\right)^2}, \qquad \bar{\hat{\theta}}^*=\frac{1}{B}\sum_{b=1}^{B}\hat{\theta}^{*(b)}. $$ For a probability level $p$, let $q_p^*$ denote the $p$-quantile of the bootstrap replicate estimates, not of the bootstrap errors.

The bootstrap standard error measures the spread of the replicate estimates around their own mean. It is not generally the same as the difference between the original estimate and the mean of the replicates. A noticeable difference between those two centers can signal bias in the bootstrap distribution; the basic interval’s reflection calculation accounts for the location of the original estimate relative to the replicate quantiles.

Normal-Approximation Confidence Interval

For a two-sided confidence level $1-\alpha$, the normal-approximation interval is

\left[ \hat{\theta}-z_{1-\alpha/2}\widehat{\operatorname{SE}}_{\!*}(\hat{\theta}), \; \hat{\theta}+z_{1-\alpha/2}\widehat{\operatorname{SE}}_{\!*}(\hat{\theta}) \right],

where $z_{1-\alpha/2}$ is the corresponding standard normal quantile. For example, $z_{0.975}\approx1.96$ for a 95% interval. This construction is appropriate when the sampling distribution of $\hat{\theta}$ is reasonably approximated by a normal distribution and the bootstrap standard error estimates its standard deviation. Normality of the individual observations is not itself required; the relevant approximation concerns the estimator’s sampling distribution.

Justification: If $(\hat{\theta}-\theta)/\operatorname{SE}(\hat{\theta})$ is approximately standard normal, then it lies between $-z_{1-\alpha/2}$ and $z_{1-\alpha/2}$ with probability approximately $1-\alpha$. Replacing the unknown standard error with its bootstrap estimate gives $$ P\!\left( -z_{1-\alpha/2}\leq \frac{\hat{\theta}-\theta}{\widehat{\operatorname{SE}}_{\!*}(\hat{\theta})} \leq z_{1-\alpha/2} \right)\approx1-\alpha. $$ Multiplying by the positive estimated standard error and rearranging for $\theta$ gives the stated endpoints.

Worked Example: A Normal Interval for a Sample Mean

A hypothetical analyst observes 64 independent annual measurements of a portfolio’s administrative cost, in thousands of dollars. The sample mean is $\hat{\theta}=12.4$. The analyst draws $B=1{,}000$ bootstrap samples of size 64, each sampled with replacement from the 64 observed values, and obtains a bootstrap standard error of 0.8 thousand dollars. Construct a 95% normal-approximation interval. Assume the estimator’s sampling distribution is adequately approximated by a normal distribution.

Step 1: Identify the critical value. For a two-sided 95% interval, $\alpha=0.05$, so $1-\alpha/2=0.975$ and $z_{0.975}\approx1.96$.

Step 2: Calculate the margin of error. $1.96(0.8)=1.568$ thousand dollars.

Step 3: Calculate the endpoints. The lower endpoint is $12.4-1.568=10.832$, and the upper endpoint is $12.4+1.568=13.968$, both in thousands of dollars.

Answer and check. Rounding only at the final step to two decimal places, the interval is $[10.83,13.97]$ thousand dollars. It is centered at 12.4, since $(10.832+13.968)/2=12.4$, and its half-width is 1.568, as required. The method is the normal-approximation interval; it avoids treating the observed data themselves as normally distributed when the stated assumption concerns the estimator.

Basic Bootstrap Confidence Interval

The basic interval starts with quantiles of the bootstrap replicate estimates. For a two-sided interval, take $q_{\alpha/2}^*$ and $q_{1-\alpha/2}^*$. These estimate the lower and upper quantiles of the bootstrap distribution. The interval does not use those quantiles directly as its endpoints; instead, it reflects them around $\hat{\theta}$.

Derivation: The bootstrap distribution of $\hat{\theta}^*-\hat{\theta}$ approximates the sampling distribution of $\hat{\theta}-\theta$. Since the replicate-estimate quantiles are $q_{\alpha/2}^*$ and $q_{1-\alpha/2}^*$, the corresponding quantiles of the bootstrap errors are $q_{\alpha/2}^*-\hat{\theta}$ and $q_{1-\alpha/2}^*-\hat{\theta}$. Thus, $$ P\!\left( q_{\alpha/2}^*-\hat{\theta} \leq \hat{\theta}-\theta \leq q_{1-\alpha/2}^*-\hat{\theta} \right)\approx1-\alpha. $$ From the left inequality, $q_{\alpha/2}^*-\hat{\theta}\leq\hat{\theta}-\theta$, so $\theta\leq2\hat{\theta}-q_{\alpha/2}^*$. From the right inequality, $\hat{\theta}-\theta\leq q_{1-\alpha/2}^*-\hat{\theta}$, so $\theta\geq2\hat{\theta}-q_{1-\alpha/2}^*$. Therefore the basic interval is $$ \left[ 2\hat{\theta}-q_{1-\alpha/2}^*, \; 2\hat{\theta}-q_{\alpha/2}^* \right]. $$

The reversal of the quantiles in the endpoint formula is essential: the upper bootstrap quantile gives the lower basic endpoint, and the lower bootstrap quantile gives the upper endpoint. By contrast, the percentile interval uses $[q_{\alpha/2}^*,q_{1-\alpha/2}^*]$ directly. The basic and percentile intervals are not interchangeable.

Worked Example: Reflecting Bootstrap Quantiles

A hypothetical analyst estimates the median time, in days, for a small group of policyholders to complete a secure online form. The observed-sample estimate is $\hat{\theta}=22.5$ days. From $B=2{,}000$ bootstrap samples, each of the original sample size and drawn with replacement, the 2.5th and 97.5th percentiles of the replicate medians are 19 and 27 days. Construct the 95% basic bootstrap interval.

Step 1: Set the quantiles. Here, $\alpha=0.05$, so $q_{0.025}^*=19$ and $q_{0.975}^*=27$. These are quantiles of replicate medians.

Step 2: Reflect the upper quantile to get the lower endpoint. $2(22.5)-27=45-27=18$ days.

Step 3: Reflect the lower quantile to get the upper endpoint. $2(22.5)-19=45-19=26$ days.

Answer and check. The 95% basic interval is $[18,26]$ days. Its endpoints are ordered, and the original estimate 22.5 lies inside. The percentile interval would instead be $[19,27]$; reporting that as the basic interval would use the wrong endpoints.

Comparing the Two Methods

Both intervals depend on the quality of the bootstrap approximation, but the normal interval compresses the replicate distribution’s spread into one standard error and imposes symmetric endpoints around $\hat{\theta}$. The basic interval uses two quantiles and reflects them, so its endpoints need not be symmetric around the estimate. If the bootstrap distribution is nearly symmetric and centered near $\hat{\theta}$, the intervals may be similar. Material differences can indicate skewness, bias, or a poor normal approximation; the difference alone does not establish which interval is reliable.

Feature	Normal-approximation interval	Basic bootstrap interval
Bootstrap information used	Bootstrap standard error	Two bootstrap replicate quantiles
Endpoint calculation	$\hat{\theta}$ plus or minus a critical value times the standard error	Reflect each replicate quantile around $\hat{\theta}$
Shape implied	Symmetric around $\hat{\theta}$	Can be asymmetric around $\hat{\theta}$
Main distributional requirement	Estimator’s sampling distribution is approximately normal	Bootstrap error distribution approximates estimator sampling error

Worked Example: Comparing Normal and Basic Endpoints

In a hypothetical study, an analyst estimates a regression coefficient measuring the change in a model’s score per unit increase in a predictor. The estimate is $\hat{\theta}=6.4$ score units. A bootstrap using $B=1{,}500$ resamples of the original sample size gives a standard error of 1.1, a 2.5th replicate quantile of 4.3, and a 97.5th replicate quantile of 8.9. Calculate both 95% intervals and compare them.

Normal-approximation interval. Use $z_{0.975}=1.96$. The margin is $1.96(1.1)=2.156$. Therefore the endpoints are $6.4-2.156=4.244$ and $6.4+2.156=8.556$. Rounded to three decimal places, this interval is $[4.244,8.556]$.

Basic bootstrap interval. Reflect the upper replicate quantile for the lower endpoint: $2(6.4)-8.9=12.8-8.9=3.9$. Reflect the lower replicate quantile for the upper endpoint: $2(6.4)-4.3=12.8-4.3=8.5$. The basic interval is $[3.9,8.5]$.

Answer and check. The normal interval has half-width 2.156 and is centered at 6.4. The basic interval is wider on the lower side of the estimate (6.4−3.9=2.5) than on the upper side (8.5−6.4=2.1), reflecting the bootstrap quantiles’ unequal distances from the estimate. Both intervals contain 6.4. Their endpoints differ because one uses a symmetric normal approximation and the other reflects the observed bootstrap quantiles. The calculation avoids assuming that the basic interval is simply the percentile interval $[4.3,8.9]$.

Conditions, Interpretation, and Common Traps

Ordinary bootstrap resampling assumes the observed cases are suitable as independent, identically distributed representatives of the population sampling units. It resamples cases with replacement and keeps the original sample size. The interval also requires the bootstrap distribution to provide a useful approximation to the estimator’s sampling uncertainty. Small samples, estimators near a boundary, highly discrete statistics, and strongly irregular sampling distributions can undermine that approximation. A normal interval additionally requires a reasonable normal approximation for the estimator; a basic interval does not impose that same symmetric normal shape, but it is not automatically valid when the bootstrap itself fails.

A confidence interval is a procedure with a long-run coverage interpretation under its assumptions. A 95% interval does not mean there is a 95% probability that a fixed parameter lies in this particular computed interval. The parameter is fixed in the frequentist framework; the interval varies from sample to sample.

Using the wrong center for the standard error. The bootstrap standard error is the sample standard deviation of the replicate estimates around their replicate mean. It is not the bootstrap bias.
Confusing basic and percentile endpoints. The percentile interval uses the replicate quantiles directly; the basic interval reflects them around $\hat{\theta}$.
Failing to reverse the quantile roles. In the basic interval, $q_{1-\alpha/2}^*$ determines the lower endpoint, while $q_{\alpha/2}^*$ determines the upper endpoint.
Assuming every bootstrap interval is symmetric. Only the normal-approximation formula is necessarily symmetric around $\hat{\theta}$; the basic interval generally is not.
Ignoring the resampling unit. If the observed cases do not represent independent sampling units, resampling individual cases may not reproduce the relevant sampling uncertainty.

Key takeaway: For a normal-approximation interval, calculate $\hat{\theta}\pm z_{1-\alpha/2}\widehat{\operatorname{SE}}_{\!*}(\hat{\theta})$. For a basic interval, calculate $[2\hat{\theta}-q_{1-\alpha/2}^*,\,2\hat{\theta}-q_{\alpha/2}^*]$. Always identify whether given quantiles describe replicate estimates or bootstrap errors before using them.

Check Your Understanding

For each problem, identify the interval method and keep the quantile and endpoint roles distinct.

A statistic has estimate 8.2 and bootstrap standard error 0.6. Assuming an adequate normal approximation, calculate a 90% normal interval using $z_{0.95}=1.645$. (Answer: $[7.213,9.187]$, or $[7.21,9.19]$ to two decimals.)
An estimate is 31. The bootstrap 2.5th and 97.5th percentiles are 26 and 38. Calculate the 95% basic interval and state the percentile interval for comparison. (Answer: basic $[24,36]$; percentile $[26,38]$.)
A hypothetical estimated rate is 0.15, with bootstrap standard error 0.04. Using $z_{0.975}=1.96$, calculate a 95% normal interval. (Answer: $[0.0716,0.2284]$.)
An estimate is 14, and the bootstrap 2.5th and 97.5th percentiles are 11 and 18. Calculate the 95% basic interval. Is it centered at the estimate? (Answer: $[10,17]$; no, its midpoint is 13.5.)
A model coefficient estimate is 2.8, its bootstrap standard error is 0.35, and its bootstrap 2.5th and 97.5th percentiles are 2.1 and 3.6. Calculate the 95% normal interval using $z_{0.975}=1.96$ and the 95% basic interval. (Answer: normal $[2.114,3.486]$; basic $[2.0,3.5]$.)

What You'll Learn