Why Resample Clusters?
When observations are grouped, records within a group may share conditions or influences. Students at one school, for example, may resemble one another because they share teachers, policies, and resources. If those students are resampled as if each were independent, a bootstrap sample can break apart the very groups that create the dependence. The resulting coefficient distribution may then understate uncertainty.
A cluster bootstrap instead treats each cluster as the resampling unit. It draws clusters with replacement and carries every record in a selected cluster into the resample. This preserves the observed arrangement of records within each cluster while allowing the sample of clusters to vary. The method is useful when the independent information comes primarily from the clusters, not from treating every record as independent.
Model and Conditions for the Procedure
For clarity, consider a regression of an outcome \(Y_{ig}\) on a predictor \(X_{ig}\), with an intercept:
Here \(\beta_1\) is the slope of interest. The model describes the conditional mean of the outcome as linear in the predictor; the errors \(\varepsilon_{ig}\) need not be independent within a cluster. The cluster bootstrap does not require resampled records within a cluster to become independent. Its key sampling premise is instead that the observed clusters can reasonably be treated as independent draws from the population of clusters relevant to the analysis.
The method is most defensible when clusters are representative of that population, the number of independent clusters is adequate for the intended approximation, and no single cluster dominates the coefficient estimate. The same regression specification and estimation rules must be used in every resample. A bootstrap describes sampling variability under those conditions; it does not repair a poor model, remove confounding, or make a nonrepresentative set of clusters representative.
The examples below use clusters of equal size to make the arithmetic transparent. In practice, include all records in each selected cluster, even if cluster sizes differ. The resulting bootstrap samples may then contain different total numbers of records.
How to Construct the Bootstrap Distribution
Start by fitting the chosen regression to the original data and recording the estimate of the coefficient of interest, such as \(\hat{\beta}_1\). Then repeat the following operation many times. In repetition \(b\), draw \(G\) cluster labels independently with replacement from the \(G\) observed labels, assemble the corresponding full clusters, and refit the regression. Record the result as \(\hat{\beta}_1^{*(b)}\). The collection of these values is the cluster-bootstrap distribution.
Record the coefficient estimate and specify the regression, variables, and any estimation choices that will be repeated.
Draw \(G\) cluster labels with replacement. Include every record from each selected cluster; do not select records separately.
Fit the same regression to the resampled records and save the coefficient. Give repeated copies of a selected cluster distinct bootstrap identifiers if the fitting method uses cluster labels.
Repeat for \(B\) resamples, then use the \(B\) estimates to summarize uncertainty, for example with a bootstrap standard error or percentile interval.
The choice of \(B\) is a computational choice: more repetitions generally make the estimated distribution summaries less noisy. It does not create more independent information than the \(G\) observed clusters provide. A thousand resamples from a study with very few clusters are still based on very few independent clusters.
Worked Example: Building One Resample
Suppose four hypothetical clinics each contribute two patients. The predictor \(X\) is a treatment indicator, with one untreated record (\(X=0\)) and one treated record (\(X=1\)) in each clinic. The paired outcomes are:
| Clinic | Outcome at \(X=0\) | Outcome at \(X=1\) | Within-clinic difference |
|---|---|---|---|
| A | 10 | 12 | 2 |
| B | 11 | 15 | 4 |
| C | 9 | 12 | 3 |
| D | 13 | 20 | 7 |
In this balanced setup, the fitted slope is the average of the four within-clinic differences. Thus the original estimate is
Now suppose one bootstrap draw selects clinics A, C, C, and D. Clinic C appears twice, so both copies contribute their two records. The resampled slope is
This is one bootstrap estimate, not a new original-data estimate. Repeating the draw-and-refit operation produces the distribution used to assess uncertainty. The duplicated clinic must be treated as a separate copy in that resample; it must not be collapsed into one clinic.
Bootstrap Standard Errors and Intervals
Let \(\hat{\beta}_1^{*(1)},\ldots,\hat{\beta}_1^{*(B)}\) be the coefficient estimates from \(B\) cluster-bootstrap samples, and let \(\bar{\beta}_1^*\) be their mean. The bootstrap standard error is the sample standard deviation of these estimates:
It estimates how much the fitted coefficient would vary across repeated samples of clusters, under the sampling assumptions. It is measured in the same units as the coefficient. A percentile confidence interval uses the appropriate quantiles of the bootstrap estimates. For a 95% percentile interval, take the 2.5th and 97.5th percentiles. The interval is an approximate procedure with a repeated-sampling interpretation; it does not mean there is a 95% probability that a fixed population coefficient lies in this particular computed interval.
Worked Example: An Exact Small Bootstrap Distribution
For a small demonstration, suppose there are just two hypothetical clusters, with within-cluster differences of 2 and 4 outcome points. As in the preceding example, each cluster has one record at \(X=0\) and one at \(X=1\), so the regression slope is the mean of the sampled cluster differences. A bootstrap sample draws two clusters with replacement. There are four equally likely ordered draws:
- First cluster, then first cluster: slope \((2+2)/2=2\).
- First cluster, then second cluster: slope \((2+4)/2=3\).
- Second cluster, then first cluster: slope \((4+2)/2=3\).
- Second cluster, then second cluster: slope \((4+4)/2=4\).
Thus the bootstrap distribution assigns probability \(1/4\) to 2, probability \(1/2\) to 3, and probability \(1/4\) to 4. Its mean is
The standard deviation of this exact bootstrap distribution is
This tiny example demonstrates how whole-cluster resampling generates a coefficient distribution. It is not a recommendation to rely on a two-cluster analysis: so few independent clusters cannot support dependable general-purpose inference. The example’s exact distribution is useful for understanding the mechanics, not for establishing that the approximation is adequate.
Why the Whole Cluster Must Stay Together
A cluster bootstrap approximates repeated sampling by using the observed clusters as the units that can vary from sample to sample. It preserves the observed within-cluster pattern because each selected unit is a complete cluster. In contrast, resampling individual records can put one member of a cluster into a bootstrap sample without its cluster-mates, or combine records as though their shared context were irrelevant.
A simple derivation makes the distinction visible in the paired setup above. Each cluster supplies a difference \(D_g\), and the fitted slope is the mean of those cluster differences:
A cluster-bootstrap draw selects \(G\) values from the observed list \(D_1,\ldots,D_G\), with replacement. If the selected values are \(D_1^*,\ldots,D_G^*\), the resampled slope is
Each draw therefore changes which clusters contribute while keeping each cluster’s paired information intact. A record-level bootstrap does not resample these cluster differences as units and can destroy their pairing. In a general regression, the coefficient is not usually a simple average of cluster-specific differences, so the procedure refits the original regression rather than averaging separate cluster estimates. The same principle still applies: change the sample of clusters, preserve the records within every selected cluster, and recompute the target coefficient.
Worked Example: Summarizing a Regression Bootstrap
Suppose a hypothetical analysis relates weekly study hours to a test score in points, with students grouped in \(G=30\) independent study groups. The original fitted slope is 1.40 points per additional study hour. After drawing \(B=2{,}000\) whole-group bootstrap samples and refitting the same regression each time, suppose the mean of the bootstrap slopes is 1.42 points per hour and the sum of squared deviations from that mean is 7.996.
The estimated bootstrap standard error is
Suppose also that the 2.5th and 97.5th percentiles of the 2,000 bootstrap slopes are 1.29 and 1.54 points per hour. The resulting 95% percentile interval is 1.29 to 1.54 points per hour. Its repeated-sampling interpretation is that, under the assumptions supporting this procedure, intervals formed in this way would cover the target slope about 95% of the time across repeated samples of clusters. The computed interval does not assign a probability to the fixed slope.
The estimate and interval describe an association between study hours and test score under the fitted model. They do not, by themselves, show that increasing a student's study time would cause the score to change by the estimated amount.
Common Mistakes and Practical Checks
The main implementation error is to resample rows rather than clusters. Another is to sample only \(G\) individual records, or to select \(G\) clusters but keep just one record from each. Both change the resampling unit. A correct implementation samples \(G\) cluster labels with replacement and includes all records for every draw.
Before interpreting the result, check that each bootstrap regression can be fitted and that the coefficient is defined in every resample. If some resamples do not permit estimation, that is a warning about the data or model and should not be hidden by silently discarding failures. Also check whether the number of independent clusters is large enough for the intended inference. Repeating a bootstrap many times reduces simulation noise in the summaries, but does not overcome a shortage of independent clusters.
- Wrong: “There are hundreds of records, so resampling rows gives hundreds of independent units.” Correction: records sharing a cluster may be dependent; resample the independent clusters when those are the sampling units.
- Wrong: “A bootstrap interval fixes an incorrect regression model.” Correction: resampling quantifies variability for the chosen estimator under its sampling assumptions; it does not correct bias or misspecification.
- Wrong: “More bootstrap repetitions make a small number of clusters adequate.” Correction: more repetitions stabilize computation, not the information available from the original clusters.
Check Your Understanding
Answer each question using the cluster-bootstrap procedure and its assumptions.
- A data set contains 18 schools, each with several students. How many cluster labels are drawn in one ordinary cluster-bootstrap sample, and what is included when a school is selected?
- Explain why a selected school that appears twice in a bootstrap draw should contribute two copies of its records.
- In a regression, the bootstrap slopes have a standard deviation of 0.12 minutes per unit of a predictor. What does this standard deviation estimate, and what are its units?
- Why does increasing the number of bootstrap repetitions not compensate for having very few independent clusters?
- State one reason a cluster-bootstrap interval might be misleading even if the resampling code correctly preserves whole clusters.