Why the Bootstrap Must Repeat Both Stages
Residual-inclusion models use an estimated first-stage residual as a control in a structural equation. That residual is not observed data: it changes when the first-stage coefficients change. A bootstrap that resamples or simulates data but then keeps the original residuals fixed does not reproduce the estimator’s full sampling variation.
Consider a linear first stage for an endogenous regressor \(d_i\), with exogenous regressors \(x_i\) and instruments \(z_i\):
Let \(\hat v_i\) be the residual from estimating this equation. A linear residual-inclusion structural equation is
Here, \(\theta\) is the structural coefficient of interest, and \(\lambda\) is the coefficient on the estimated control. The control-function interpretation relies on assumptions about how the first-stage disturbance relates to the structural disturbance and on suitable specification and identification. Bootstrap resampling does not establish those assumptions; it estimates uncertainty conditional on the model and resampling scheme being appropriate.
What Is Repeated in a Replication?
For each replication, use one resampled or simulated data set containing the outcome, endogenous regressor, exogenous regressors, and instruments. Estimate the first stage on that data set, calculate its residuals, and use them in the structural regression. Store the coefficient or other statistic of interest. The fitted values and residuals from the original sample are not substituted for their replication-specific counterparts.
Resample complete observational units, resample whole clusters when appropriate, or simulate from a specified parametric model.
Estimate the first-stage equation using the bootstrap instruments and regressors, and calculate a new residual for every bootstrap observation.
Regress the bootstrap outcome on the structural regressors and the newly estimated first-stage residuals. Store \(\hat\theta^{*}\), or the test statistic or interval endpoint being studied.
Use the empirical distribution of the stored statistics to estimate uncertainty, subject to the validity of the model and resampling design.
With \(B\) replications, the bootstrap standard error of \(\hat\theta\) is the sample standard deviation of the \(B\) estimates:
A percentile confidence interval uses the empirical quantiles of the bootstrap estimates. For a nominal 95% interval, its endpoints are the 2.5th and 97.5th percentiles. This interval is a procedure with approximate repeated-sampling coverage under suitable conditions; it does not mean there is a 95% probability that a fixed parameter lies in the realized interval.
Worked Example: A Percentile Interval From Full-Procedure Replications
Suppose a hypothetical study estimates the effect of an endogenous treatment \(d\), measured in units, on an outcome \(y\), measured in points. The original residual-inclusion estimate is \(\hat\theta=0.80\) points per treatment unit. The analyst draws \(B=1{,}000\) pairs-bootstrap samples and, in every sample, refits both stages and stores the new structural coefficient. The bootstrap standard deviation is \(0.18\), and the empirical 2.5th and 97.5th percentiles are \(0.45\) and \(1.15\).
The estimated bootstrap standard error is the reported standard deviation of the replication estimates:
The percentile interval is
Under the maintained model and the validity of the pairs resampling scheme, this procedure gives an approximate 95% confidence interval for the structural effect. Its endpoints suggest effects from 0.45 to 1.15 outcome points per treatment unit. The interval is based on refitted bootstrap estimates; it would not represent the same procedure if the original first-stage residuals had been held fixed.
Pairs and Parametric Bootstrap Schemes
The pairs bootstrap samples observational units with replacement. For independent, identically distributed observations, each sampled unit is a complete row containing all variables needed for both stages. A sampled row carries its \(y_i\), \(d_i\), \(x_i\), and \(z_i\) together. Repeated rows are allowed, and some original rows may not appear in a given replication. This scheme does not require specifying a full parametric distribution for the data, but it does rely on the observational units being appropriate independent sampling units and on the sample representing the relevant population.
A parametric bootstrap instead specifies a probability model, estimates its parameters, and simulates new data from the fitted model. For example, conditional on observed \(x_i,z_i\), an analyst might simulate first-stage disturbances \(v_i^{*}\), generate \(d_i^{*}\), and then simulate the structural disturbance and \(y_i^{*}\) under a fitted joint model. The simulated data must reproduce the dependence between the first-stage and structural disturbances that the residual-inclusion model is intended to represent, as well as any modeled heteroskedasticity or other relevant features. Both stages are then re-estimated in each simulated sample.
The choice is substantive. Pairs resampling reflects empirical variation in complete observed rows; parametric resampling reflects the assumed data-generating model. A parametric bootstrap can be useful when the model is credible or the design is fixed, but it may be sensitive to distributional misspecification. Pairs resampling is less dependent on a fully specified distribution, but it is not automatically valid under dependence, an inappropriate sampling design, or severe lack of support.
Worked Example: One Pairs-Bootstrap Replication
For transparent arithmetic, consider a hypothetical first stage with an intercept and one instrument \(z\). A particular pairs-bootstrap draw selects four rows, including a repeated row:
| Resampled row | \(z\) | \(d\) |
|---|---|---|
| 1 | -1 | 1 |
| 2 | 0 | 2 |
| 3 | 0 | 2 |
| 4 | 2 | 5 |
The first-stage sample means are \(\bar z=0.25\) and \(\bar d=2.5\). The estimated slope and intercept are
The fitted values are approximately \(0.7895, 2.1579, 2.1579,\) and \(4.8947\), so the new first-stage residuals are \(0.2105,-0.1579,-0.1579,\) and \(0.1053\). Suppose the corresponding hypothetical outcomes are \(3.6316,4.5263,4.5263,\) and \(11.3158\). These values satisfy \(y_i=1+2d_i+3\hat v_i^{*}\), up to rounding. Therefore the structural regression on an intercept, \(d_i\), and the newly calculated residual has coefficients 1, 2, and 3, respectively, and the replication’s estimate of \(\theta\) is \(2\).
This small calculation illustrates the required sequence: the duplicated row is included as sampled, the first stage is refitted on these four rows, and the residuals are recalculated before the structural regression. In an actual bootstrap, the other resamples would generally produce different first-stage fits, residuals, and structural coefficients. The example is a computational illustration, not evidence that such a small sample supports reliable inference.
Respect the Sampling Unit
The row is not always the independent sampling unit. If observations are clustered—for example, patients within clinics or students within schools—outcomes within a cluster may share unobserved conditions. Resampling individual rows independently would break that dependence and can produce misleading uncertainty estimates. A cluster pairs bootstrap samples clusters with replacement and includes all relevant observations from each selected cluster. Both stages are then refitted on the resulting cluster-resampled data.
Other designs require resampling at the unit that was independently sampled. Repeated observations on a person, for example, should not be treated as independent rows if the person was the sampling unit and within-person dependence is part of the design. In each case, the resampled data must preserve the joint information needed to estimate both stages. If the number of independent clusters is small, a cluster bootstrap may itself have poor finite-sample performance; resampling clusters does not create additional independent information.
Worked Example: Cluster Resampling and an Interval
Suppose a hypothetical study has patients nested within clinics, and clinics—not individual patients—are treated as independent sampling units. The analyst draws 500 cluster-bootstrap replications. Each draw selects clinics with replacement, carries all patients in each selected clinic into the bootstrap data, and refits both the first-stage and structural equations. The coefficient of interest is a change in outcome points per treatment unit. Suppose the resulting empirical 2.5th and 97.5th percentiles of the stored coefficients are \(-0.12\) and \(0.48\).
The percentile interval is
The interval includes zero, so these bootstrap results do not exclude a zero structural effect at the corresponding approximate two-sided 5% level under the maintained assumptions. The calculation is meaningful only if the clinic-level resampling scheme is appropriate. Sampling patients independently instead would treat within-clinic observations as independent and would fail to reproduce the specified sampling structure.
Why Re-estimating the First Stage Matters
The reason for refitting can be seen by writing the estimator as a function of both the structural data and the estimated first-stage parameters. Let \(\hat\eta\) denote the collection of first-stage coefficients, and let \(\hat\theta=g(\text{data},\hat\eta)\) denote the resulting structural estimate, where the residuals used in the structural equation depend on \(\hat\eta\). A first-order expansion around the population first-stage parameter \(\eta\) gives the schematic decomposition
The second term reflects variation from estimating the first stage and constructing its residuals. A bootstrap that fixes the original residuals suppresses this term: its structural regression is not responding to the replication-specific first-stage fit. Refitting both stages reproduces the estimator’s dependence on the estimated control, allowing the bootstrap distribution to reflect both sources of variation and their dependence.
This justification is not a guarantee that every bootstrap interval is accurate. Validity still requires suitable regularity conditions, a correctly represented sampling dependence structure, and an estimator that behaves regularly in the relevant region. Weak identification, a misspecified control-function relationship, invalid instruments, or a very small number of independent clusters can make conventional bootstrap inference unreliable. The bootstrap quantifies uncertainty under a procedure; it does not repair that procedure’s identifying assumptions.
Common Pitfalls
- Holding the original residuals fixed. This omits variation from estimating the first stage. Refit the first-stage equation and recalculate residuals in every replication.
- Resampling variables separately. Independently resampling \(y\), \(d\), and \(z\) destroys their observed joint relationships. For a pairs bootstrap, resample complete rows.
- Resampling patients when clinics are the independent units. This breaks the within-clinic dependence. Resample whole clusters and retain their observations together.
- Calling a parametric bootstrap assumption-free. Simulated results depend on the specified and fitted distribution, including the modeled dependence between disturbances.
- Treating bootstrap replication count as a cure for model problems. More replications reduce Monte Carlo noise in estimated bootstrap summaries, but do not fix weak identification, misspecification, or an unsuitable resampling unit.
- Interpreting an interval as a probability statement about a fixed parameter. A confidence interval describes the long-run performance of a procedure under its assumptions, not the probability that the parameter is inside the observed interval.
Check Your Understanding
Answer each question by identifying the estimation step or sampling assumption involved.
- In a residual-inclusion pairs bootstrap, which variables should be resampled together, and what must be refitted after resampling?
- Why does keeping the original first-stage residuals fixed generally omit part of the estimator’s sampling variation?
- State one difference between pairs and parametric bootstrap schemes for a residual-inclusion model.
- If clinics are independent but patients within clinics may be dependent, what should be resampled in a cluster bootstrap?
- A bootstrap interval includes zero. What does that imply about the evidence for a nonzero effect, and what does it not imply about the probability that the null is true?