Why Bootstrap 2SLS?
A 2SLS estimate is calculated using both a first stage, which relates endogenous regressors to instruments and exogenous covariates, and a second stage, which estimates the outcome equation using the first-stage fitted values. Its sampling uncertainty reflects the whole procedure. A bootstrap for 2SLS must therefore reproduce the data structure and re-estimate both stages in every replicate. Reusing fitted values from the original sample does not reproduce the variation in the 2SLS estimator.
Bootstrap inference can be useful when an analyst wants standard errors or confidence intervals obtained by repeatedly applying the estimator to resampled data. It does not remove the need for valid instruments, identification, or a suitable sampling design. Under strong identification and regularity conditions, resampling provides an approximation to the sampling distribution of 2SLS; it is not a general remedy for weak instruments or invalid instruments.
Setup and Assumptions
Let \(Y_i\) be an outcome, \(D_i\) a vector of endogenous regressors, \(X_i\) a vector of exogenous regressors (including an intercept), and \(Z_i\) a vector of excluded instruments. Write \(W_i=(X_i,D_i)\) for the regressors in the structural equation and \(Q_i=(X_i,Z_i)\) for the full instrument set. In each stage, the exogenous regressors \(X_i\) are included; the excluded instruments \(Z_i\) contribute variation used to predict \(D_i\).
For the ordinary observation-level pairs bootstrap, assume the observational units are independent and identically distributed draws from the target population, or that the sampling design otherwise justifies this resampling scheme. The 2SLS model must also be identified: the instruments must satisfy the model's exogeneity and exclusion requirements, and there must be enough relevant instrument variation to identify the coefficients of interest. For standard bootstrap approximations, identification should be sufficiently strong and the estimator should be regular, with the relevant matrices having full rank and finite moments. These conditions are substantive, not consequences of resampling.
If observations may be dependent within groups but the groups themselves are independent, resample whole clusters rather than individual rows. This allows arbitrary dependence within a sampled cluster while retaining the independent-cluster sampling structure. The number of independent clusters must be adequate for the approximation used; a large number of rows within only a few clusters does not provide the same information as many independent clusters. If the data come from a stratified or otherwise structured sample, the resampling scheme should preserve that design rather than treating all rows as exchangeable.
Re-estimating 2SLS in Each Replicate
With one endogenous regressor, the first stage regresses \(D\) on \(X\) and \(Z\), producing fitted values \(\widehat D\). The second stage regresses \(Y\) on \(X\) and \(\widehat D\). With multiple endogenous regressors, each is fitted using the same instrument set, and the second stage uses all their fitted values along with \(X\). In matrix notation, the 2SLS estimator can be written as
This expression makes clear that the estimate depends on the sampled values of \(Y\), \(W\), and \(Q\). A bootstrap draw changes those values and, consequently, the projection onto the instrument space. The operational rule is therefore simple: in a replicate, reconstruct the instrument set, run the first stage, form that replicate's fitted values, and run the second stage. Do not merely resample the outcome equation while retaining the original first-stage fit.
For independent observations, draw \(n\) complete rows with replacement. For clustered data, draw clusters with replacement and retain every row in each selected cluster.
Using only the replicate's data, regress each endogenous regressor on the included exogenous variables and instruments. Save the new fitted values.
Regress the replicate outcome on the exogenous regressors and the newly generated fitted endogenous regressors. Store the coefficient or other statistic of interest.
Repeat for \(b=1,\ldots,B\), inspect failures and the resulting distribution, and calculate the chosen bootstrap standard error or interval.
Worked Example: One Bootstrap Replicate With a Binary Instrument
Suppose a hypothetical sample has four observations. The outcome \(Y\), endogenous treatment \(D\), and binary instrument \(Z\) are as follows. There are no other covariates besides the intercept.
| Observation | \(Z\) | \(D\) | \(Y\) |
|---|---|---|---|
| 1 | 0 | 1 | 2 |
| 2 | 0 | 2 | 3 |
| 3 | 1 | 3 | 8 |
| 4 | 1 | 4 | 10 |
Consider a bootstrap draw that selects observation 1 once, observation 3 twice, and observation 4 once. The resampled data therefore contain four rows: one with \(Z=0\) and three with \(Z=1\). The first-stage fitted value is the mean \(D\) within each instrument group. For the one resampled row with \(Z=0\), the fitted value is \(D=1\). For the three resampled rows with \(Z=1\), the fitted value is
The corresponding outcome means are \(2\) for \(Z=0\) and \((8+8+10)/3=26/3\) for \(Z=1\). With an intercept and a single binary instrument, the second-stage slope is the difference in group outcome means divided by the difference in group fitted-treatment means:
This is the 2SLS coefficient for this particular resample. The replicate retained one row in the \(Z=0\) group and three in the \(Z=1\) group, so both group means and the calculation use the actual draw. In a full bootstrap, the same two stages would be run for many draws; this one coefficient alone is not an estimate of a standard error.
Bootstrap Standard Errors and Confidence Intervals
Let \(\widehat\theta^{*(b)}\) denote the estimate of a scalar coefficient or other statistic from bootstrap replicate \(b\), for \(b=1,\ldots,B\). The bootstrap standard error is the sample standard deviation of those replicate estimates:
This estimates the sampling variability of the estimator using the empirical distribution represented by the observed sample. It is not the standard deviation of the individual observations, nor is it obtained by taking an ordinary least-squares standard error from the second stage while treating fitted values as observed.
A percentile interval uses quantiles of the bootstrap coefficient estimates. For a nominal 95% interval, it takes the 2.5th and 97.5th percentiles of the replicate estimates. Its interpretation is about the repeated-sampling procedure: under the method's assumptions and with adequate resampling accuracy, the procedure aims for 95% coverage of the fixed target parameter. It does not mean that, after seeing the data, there is a 95% probability that the parameter lies in this particular interval. Other bootstrap intervals, including basic or studentized intervals, use different constructions and may have different accuracy properties.
Worked Example: Computing a Bootstrap Standard Error
Suppose six illustrative bootstrap replicates for a hypothetical 2SLS coefficient, measured in outcome units per treatment unit, are \(2.5,\ 2.2,\ 3.0,\ 2.2,\ 2.5,\) and \(3.0\). These few replicates are used only to demonstrate the arithmetic; a real analysis would ordinarily use many more.
Their mean is
The sum of squared deviations from the unrounded mean \(2.5667\) is approximately \(0.6533\). Thus the bootstrap standard error is
outcome units per treatment unit. The estimate describes the spread of the replicate coefficients around their bootstrap mean. With only six replicates, this standard error is itself very unstable; the calculation illustrates the formula, not an adequate final analysis.
Worked Example: Reading a Percentile Interval
Suppose a hypothetical analysis uses \(B=2{,}000\) valid observation-level bootstrap replicates and obtains the 2.5th and 97.5th percentiles \(1.10\) and \(3.05\). The point estimate is \(2.00\) outcome units per treatment unit. The 95% percentile bootstrap interval is therefore \([1.10,\ 3.05]\) outcome units per treatment unit.
The interval is constructed by sorting the 2,000 re-estimated coefficients and taking the specified lower and upper quantiles. In context, the procedure gives a range of values compatible with the sampling uncertainty represented by the bootstrap, under the validity and regularity assumptions. Since both endpoints are positive, this interval does not include zero. That fact is not proof that the instrument is valid, that identification is strong, or that the interval has its nominal coverage in this application.
Cluster Resampling and Diagnostics
When dependence is possible within clusters, resampling individual rows breaks the observed dependence structure and can give misleading uncertainty. In the cluster bootstrap, each selected cluster is copied as a whole; if a cluster is selected more than once, each copy appears as a separate bootstrap cluster. The analyst then re-estimates both stages on all rows in the resampled clusters. The number of rows may vary across replicates when cluster sizes differ.
Worked Example: Resampling Independent Clusters
Consider a hypothetical study with four independent clinics, each contributing one observation for simplicity. Their data are: clinic A has \((Z,D,Y)=(0,1,2)\), clinic B has \((0,2,3)\), clinic C has \((1,3,7)\), and clinic D has \((1,4,8)\). A cluster-bootstrap draw selects A once, C twice, and D once. Because each cluster contains one row here, this draw has the same rows as the individual-level draw A, C, C, D.
The \(Z=0\) group has \(\bar D_0=1\) and \(\bar Y_0=2\). The \(Z=1\) group has \(\bar D_1=(3+3+4)/3=10/3\) and \(\bar Y_1=(7+7+8)/3=22/3\). The replicate 2SLS coefficient is
In a study where clinics contain multiple observations, the cluster draw would instead include all rows from each selected clinic, including repeated copies of C's entire data. The stages would be re-estimated on that expanded cluster sample; it would not be correct to retain only one row per clinic unless the data actually had one row per clinic.
Useful diagnostics include checking that each replicate has adequate variation in the instruments, that the first-stage and second-stage design matrices have full required rank, and that the coefficient distribution is not dominated by extreme values. Record the number and reason for failed replicates. If a draw cannot identify the requested coefficient, silently dropping it changes the bootstrap distribution; frequent failures signal a problem with the design, sample size, instrument variation, or estimation procedure. Also examine whether a small number of clusters drives the result and whether the bootstrap distribution is highly skewed or heavy-tailed.
What the Bootstrap Does Not Fix
The pairs bootstrap resamples the observed joint records, preserving the empirical association among \(Y\), \(D\), \(X\), and \(Z\). It does not make an instrument exogenous, establish the exclusion restriction, or create identifying variation that the data lack. With weak instruments, 2SLS estimates can have highly non-normal and unstable sampling behavior; ordinary bootstrap standard errors and percentile intervals may not provide reliable inference. A large number of bootstrap draws reduces simulation noise in the estimated bootstrap distribution, but does not correct weak identification or model misspecification.
The main alternatives answer different needs. Analytic homoskedastic, heteroskedasticity-robust, cluster-robust, or HAC 2SLS standard errors use different variance estimators and dependence assumptions. The bootstrap approximates sampling variability by resampling and rerunning the estimator, while analytic methods estimate uncertainty from a variance formula. Neither approach is automatically superior in every setting. Residual resampling is not the same as resampling complete observation records and requires additional model assumptions; it should not be substituted casually for the pairs bootstrap. In all cases, the chosen method must reflect the sampling and dependence structure.
Common Mistakes
- Holding the original first-stage fitted values fixed. This omits the variation in the estimated first stage. Refit the first stage in every resample and use its new fitted values in that replicate's second stage.
- Resampling variables separately. This destroys the observed joint relationships. Sample complete rows, or complete clusters when clusters are the independent sampling units.
- Resampling rows despite within-cluster dependence. Individual resampling treats dependent observations as independent. Resample whole independent clusters instead.
- Treating a large number of bootstrap draws as a cure for weak instruments. Increasing \(B\) improves Monte Carlo precision, not identification or the validity of the bootstrap approximation.
- Calling the interval a probability statement about the fixed parameter. A confidence interval describes the long-run coverage of a procedure under assumptions; it does not assign a probability to the parameter after the data are observed.
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions.
- Why must the first stage be re-estimated in every bootstrap replicate?
- In a study with patients nested in independent hospitals, which units should be resampled if within-hospital dependence is expected?
- A bootstrap draw has 500 rows, but many replicates fail because the instrument matrix is rank deficient. What should the analyst do?
- Does increasing the number of bootstrap replicates repair weak identification? Explain.
- A 95% percentile bootstrap interval for a coefficient is \([0.4, 1.7]\) outcome units per treatment unit. State an appropriate interpretation and one conclusion it does not justify.
Solutions. (1) The fitted values depend on the instruments and covariates in the resampled data; refitting reproduces first-stage estimation variability. (2) Resample hospitals as whole clusters, including all patients in each selected hospital. (3) Investigate and report the failures rather than silently discarding them; frequent rank failures indicate that the design or available variation does not support stable estimation under this resampling scheme. (4) No. More replicates reduce simulation error in the bootstrap summaries but do not create instrument strength or guarantee reliable weak-instrument inference. (5) Under the assumptions and bootstrap procedure, the interval is a 95% confidence interval for the coefficient, in the stated units. Its exclusion of zero does not establish instrument validity or prove a causal interpretation.