Bootstrap Intervals for Means and Medians

Why the Statistic Matters

A confidence interval describes uncertainty about a particular population quantity. For quantitative data, two common targets are the population mean, which represents an arithmetic average, and the population median, which represents a central point in the population distribution. These targets are not interchangeable. In a skewed distribution, the mean may be pulled toward a long tail, while the median often remains closer to the middle observation.

Bootstrap methods estimate uncertainty by repeatedly resampling the observed data. For each resample, we calculate the statistic of interest. The resulting collection of estimates approximates the statistic’s sampling distribution. A bootstrap interval for the mean therefore describes uncertainty in the mean; an interval for the median describes uncertainty in the median. The two intervals can differ in width and shape because the statistics respond differently to the data.

Definition: In the ordinary nonparametric bootstrap, a bootstrap sample of size \(n\) is formed by drawing \(n\) observations independently with replacement from the original sample. A bootstrap estimate is the statistic calculated from one such resample. Repeating this process produces a bootstrap distribution of the statistic.

This method treats the observed sample as an estimate of the population distribution. Its usual justification assumes the observations are independent and that the sample reasonably represents the population being studied. The resampling unit must match the independent unit in the design. For this tutorial, the data are single, independent quantitative observations; other designs require a resampling scheme that respects their structure.

Constructing an Interval for a Mean or Median

Let \(T\) be the statistic chosen for the target: either the sample mean \(\bar{x}\) or the sample median. Draw \(B\) bootstrap samples, each of size \(n\), and calculate \(T^*_1,\ldots,T^*_B\). These values form the estimated bootstrap distribution. Its spread estimates the sampling uncertainty of \(T\), while its quantiles can be used to form a percentile interval.

Percentile bootstrap interval: For a two-sided confidence level of \(1-\alpha\), take the \(\alpha/2\) and \(1-\alpha/2\) quantiles of the bootstrap estimates. The interval is \( \left[Q^*_{\alpha/2},Q^*_{1-\alpha/2}\right] \), where \(Q^*_q\) is the \(q\) quantile of the bootstrap distribution for the chosen statistic. For a 95% interval, the endpoints are the 2.5th and 97.5th percentiles.

The percentile method uses the bootstrap estimates themselves as endpoints. With a finite number of resamples, quantile algorithms can produce slightly different endpoints, and Monte Carlo variation means results can change a little between runs. A large \(B\), such as 10,000, generally gives more stable quantile estimates than a very small number of resamples. The interval’s actual reliability still depends on whether the sampling design and sample support the bootstrap approximation.

The bootstrap standard error is commonly estimated by the sample standard deviation of the \(B\) bootstrap estimates:

SE_{\text{boot}}(T) = \sqrt{\frac{\sum_{b=1}^{B}(T_b^*-\bar{T}^*)^2}{B-1}}, \qquad \bar{T}^*=\frac{1}{B}\sum_{b=1}^{B}T_b^*.

This standard error summarizes the spread of the bootstrap distribution; it is not itself a confidence interval. A percentile interval also uses the distribution’s shape and quantiles. If that distribution is skewed, its two endpoints need not be equally far from the original sample estimate.

Why the Bootstrap Standard Error of the Mean Has a Simple Form

A short derivation helps explain why the mean’s bootstrap uncertainty can be calculated directly from the observed values. Conditional on the data, one bootstrap draw has probability \(1/n\) of being any one of the observed values. Its mean is \(\bar{x}\), and its variance is the empirical variance with denominator \(n\):

\operatorname{Var}(X^*\mid\text{data}) = \frac{1}{n}\sum_{i=1}^{n}(x_i-\bar{x})^2.

A bootstrap sample mean is the average of \(n\) independent draws from this empirical distribution. The variance of an average of \(n\) independent, equally distributed draws is the variance of one draw divided by \(n\). Therefore:

\operatorname{Var}(\bar{X}^*\mid\text{data}) = \frac{1}{n^2}\sum_{i=1}^{n}(x_i-\bar{x})^2, \qquad SE_{\text{boot}}(\bar{X}) = \sqrt{\frac{1}{n^2}\sum_{i=1}^{n}(x_i-\bar{x})^2}.

This is the spread over all possible bootstrap samples; a finite set of \(B\) resamples estimates it. The formula also shows why an extreme observation can substantially increase uncertainty for the mean: its squared distance from \(\bar{x}\) contributes to the sum.

Worked Example: A Skewed Sample and Two Targets

Suppose a hypothetical sample of eight measurements, in minutes, is \(1,2,2,3,4,5,8,15\). The data have a long upper tail. Find the sample mean and median, calculate the bootstrap standard error of the mean, and interpret a hypothetical pair of 95% percentile intervals.

The mean is \( \bar{x}=(1+2+2+3+4+5+8+15)/8=40/8=5 \) minutes. The two middle ordered values are 3 and 4, so the median is \( (3+4)/2=3.5 \) minutes.

For the mean’s bootstrap standard error, the squared deviations from 5 are \(16,9,9,4,1,0,9,100\), which sum to 148. Thus:

SE_{\text{boot}}(\bar{X}) = \sqrt{\frac{148}{8^2}} = \sqrt{2.3125} \approx 1.521\text{ minutes}.

Now suppose 10,000 bootstrap resamples give a hypothetical 2.5th and 97.5th percentile of 2.75 and 8.25 minutes for the mean, and 2 and 8.5 minutes for the median. The resulting intervals are 2.75 to 8.25 minutes for the population mean and 2 to 8.5 minutes for the population median. These endpoints are illustrative bootstrap output, not values determined by the displayed standard error alone.

The two intervals refer to different population quantities. In this sample, the mean exceeds the median because the high value of 15 pulls the arithmetic average upward. That observation also contributes 100 to the squared-deviation sum used for the mean’s standard error. A percentile interval reflects the full bootstrap distribution, so its width cannot be inferred from the standard error alone.

What Changes When an Outlier Is Present?

The mean uses every numerical value directly, so an extreme observation can move its estimate and affect its uncertainty. The median depends on the ordered middle of the sample, which usually makes it less sensitive to the magnitude of an extreme value. “Less sensitive” does not mean unaffected: the median can change if enough observations are extreme, and its bootstrap distribution can be discrete or irregular in small samples.

Worked Example: Replacing a High Value With an Outlier

Consider a second hypothetical sample in which the last value is 60 rather than 15: \(1,2,2,3,4,5,8,60\) minutes. Compare its mean, median, and bootstrap standard error for the mean with those in the preceding example.

The new mean is \( (1+2+2+3+4+5+8+60)/8=85/8=10.625 \) minutes. Its median is still \( (3+4)/2=3.5 \) minutes. The extreme value has increased the mean by 5.625 minutes but has not changed the sample median.

The squared deviations from 10.625 sum to 2,819.875. Therefore:

SE_{\text{boot}}(\bar{X}) = \sqrt{\frac{2819.875}{64}} = \sqrt{44.060546875} \approx 6.638\text{ minutes}.

This is much larger than the 1.521-minute bootstrap standard error for the first sample. For illustration, suppose 10,000 resamples from this second sample give percentile intervals of 3.75 to 27.5 minutes for the mean and 2 to 5.5 minutes for the median. The mean interval is much wider because resamples that include the outlier can have substantially larger means. The median interval is less affected by one extreme value, although it is not guaranteed to be narrower in every sample or to give a better answer for every research question.

If the scientific target is the population arithmetic average, the mean remains the relevant statistic even when the data are skewed. If the target is a central value that is less sensitive to extremes, the median may be more suitable. The choice is about the population quantity the question asks for, not about choosing whichever interval looks more favorable.

Calculating Bootstrap Spread From a Few Resamples

In practice, a confidence interval should use enough resamples for its tail quantiles to be estimated reasonably well. A very small number of resamples is useful only for seeing how the calculations work; it cannot provide stable 2.5th and 97.5th percentiles.

Worked Example: The Standard Error From Four Resamples

Take the hypothetical data \(1,2,4\). Its sample mean is \(7/3\), and its median is 2. Suppose four bootstrap samples happen to be \((1,1,2)\), \((1,4,4)\), \((2,2,4)\), and \((1,2,4)\). Their bootstrap means are \(4/3,3,8/3,7/3\), and their medians are \(1,4,2,2\).

For the means, the average of the four bootstrap estimates is \( \bar{T}^*=(4/3+3+8/3+7/3)/4=7/3 \). The sum of their squared deviations from \(7/3\) is \(1+4/9+1/9+0=14/9\). The estimated bootstrap standard error is therefore:

SE_{\text{boot}}(\bar{X}) = \sqrt{\frac{14/9}{4-1}} = \sqrt{\frac{14}{27}} \approx 0.7201.

For the medians, their average is \( (1+4+2+2)/4=2.25 \). The squared deviations from 2.25 sum to \(1.5625+3.0625+0.0625+0.0625=4.75\), giving:

SE_{\text{boot}}(\text{median}) = \sqrt{\frac{4.75}{4-1}} = \sqrt{1.5833} \approx 1.258.

These four resamples illustrate that the two statistics can have different bootstrap spreads. The estimates are not dependable standard errors or intervals: more resamples are needed to reduce simulation noise, and four resamples cannot support a useful 95% percentile interval.

Interpreting and Checking the Result

A 95% bootstrap confidence interval is an interval procedure intended to capture the specified population parameter in about 95% of repeated samples under its assumptions. Once a particular interval has been calculated, it is not correct to say there is a 95% probability that the fixed parameter lies inside it. Nor does an interval for the mean provide an interval for the median, or vice versa.

Before reporting an interval, check that the statistic matches the question, the observations were resampled at the correct independent unit, and the original sample contains enough information to represent the population. Bootstrap resamples can only reuse observed values: they do not create evidence about population values never represented in the sample. Extreme skew, a very small sample, or a sparse set of distinct values can make an ordinary bootstrap approximation questionable. These concerns are reasons to examine the data and design carefully, not reasons to assume that one statistic or interval method is always best.

Mean: describes an arithmetic average and is directly influenced by every value.
Median: describes the central value in the ordered distribution and is generally less influenced by a single extreme magnitude.
Bootstrap standard error: summarizes the resampling spread of the chosen statistic.
Percentile interval: uses quantiles of the bootstrap distribution and may be asymmetric.

Common mistake: “The median is robust, so its bootstrap interval must always be narrower and more trustworthy.” Robustness to an extreme value does not guarantee a narrow interval or a sound approximation. The median’s uncertainty depends on the sample distribution near its center, sample size, ties, and the adequacy of the resampling assumptions. Choose the statistic for the target, then assess the interval for that statistic.

Key takeaway: Bootstrap resampling estimates uncertainty for the statistic you calculate on each resample. Means and medians can respond very differently to skew and outliers, so state the target explicitly and interpret each interval only for that target.

Check Your Understanding

Answer each question using the definitions and reasoning in this tutorial.

Why must a bootstrap sample draw observations with replacement, and why does its size ordinarily equal the original sample size?
A bootstrap distribution of sample means has standard deviation 2.4 units. What does this number estimate, and what does it not represent?
For a skewed sample, explain why an interval for the mean might differ from an interval for the median.
Why can a single very large observation affect the mean’s bootstrap standard error more than the median’s?
What does a 95% bootstrap confidence interval claim about a repeated-sampling procedure, and what does it not claim about one calculated interval?