Why Bootstrap the Anderson–Rubin Test?
The Anderson–Rubin (AR) test is useful when the instrument may be weak because its validity does not depend on estimating the treatment effect precisely in a first stage. But its usual reference distribution may still be inaccurate in finite samples, especially with heteroskedastic errors or clustered observations. A bootstrap can improve calibration by approximating the null distribution of the AR statistic while retaining the study’s variance or dependence structure.
The bootstrap does not make an invalid instrument valid, and it does not guarantee accurate inference in every small sample. Its central task is narrower: generate repeated data under the tested null, using a resampling scheme compatible with the sampling design, and compare the observed AR statistic with the resulting statistics.
The Null Hypothesis and the AR Statistic
Let \(Y_i\) be an outcome, \(D_i\) an endogenous treatment, \(X_i\) a vector of included exogenous covariates, and \(Z_i\) a vector of \(q\) excluded instruments. For a candidate treatment effect \(\beta_0\), define \(W_i(\beta_0)=Y_i-\beta_0D_i\). The structural model and instrument validity condition are \(Y_i=\beta D_i+X_i'\gamma+u_i\) and \(E[Z_i u_i]=0\). The test \(H_0:\beta=\beta_0\) is conducted by regressing \(W_i(\beta_0)\) on \(X_i\) and \(Z_i\), then testing whether the coefficients on all elements of \(Z_i\) are zero.
The covariance estimator must reflect the error structure: use a heteroskedasticity-robust estimator for independent observations with unequal variances, and a cluster-robust estimator when errors may be dependent within clusters. Under suitable regularity conditions, the statistic has an asymptotic \(\chi^2_q\) reference distribution. The bootstrap replaces that reference distribution with an empirical distribution generated under the null.
The structural model must be appropriately specified, the instruments must satisfy the relevant exclusion and exogeneity conditions, and observations or clusters must be independent across the units treated as independent by the resampling scheme. The bootstrap calibration additionally relies on the restricted residuals being informative about the null errors and on having enough independent sampling units for the chosen approximation. Weak first-stage relevance does not itself invalidate AR inference; it is not a condition that the bootstrap must repair.
Null-Imposing Bootstrap Logic
For each proposed \(\beta_0\), form \(W_i=Y_i-\beta_0D_i\) and fit the restricted regression of \(W_i\) on \(X_i\), excluding \(Z_i\). Under \(H_0\), the excluded instruments should have no explanatory role once \(X_i\) is included. The restricted fit therefore supplies fitted values and residuals for generating bootstrap outcomes under the null. In each replicate, construct \(W_i^*\), regress it on \(X_i\) and \(Z_i\), and calculate the same robust AR statistic as in the observed sample.
Here \(\hat{\gamma}_0\) is estimated in the restricted regression, \(\tilde e_i\) is a centered or leverage-adjusted restricted residual, and \(v_i\) is a random multiplier. For example, a Rademacher multiplier is \(+1\) or \(-1\), each with probability one-half. The observed \(D_i\), \(Z_i\), and \(X_i\) remain fixed in this conditional wild-bootstrap construction; the generated variable is the null-adjusted outcome \(W_i^*\). Use the same covariance type for the bootstrap statistic as for the observed statistic.
This construction imposes the tested null: the bootstrap outcome’s fitted part contains \(X_i\), not \(Z_i\). Resampling residuals from the unrestricted regression would generally preserve the instrument relationship being tested, so it would not generate the desired null distribution. The method can be summarized as follows.
Choose \(\beta_0\), calculate \(W=Y-\beta_0D\), and fit the restricted regression of \(W\) on \(X\).
Multiply restricted residuals by multipliers appropriate to the independent sampling units and add them to the restricted fitted values.
Regress the generated outcome on \(X\) and \(Z\), calculate the appropriate robust AR statistic, and repeat for \(B\) replicates.
Compare the observed statistic with the replicate statistics to obtain a bootstrap p-value or a bootstrap critical value.
Resampling Must Follow the Sampling Design
With independent observations and heteroskedastic errors, an observation-level wild bootstrap multiplies each restricted residual by its own random multiplier. This preserves the observation’s residual magnitude while varying its sign or scale, rather than imposing equal error variance through ordinary residual resampling.
With clustered data, observations within a cluster may share shocks. Give every observation in cluster \(g\) the same multiplier \(v_g\), multiplying the whole cluster’s restricted residual vector. This retains the observed within-cluster residual pattern while varying it across bootstrap replicates. The AR statistic must use a cluster-robust covariance estimator, and the effective number of independent units is the number of clusters, not the number of individuals.
| Sampling structure | Null-resampling scheme | Key requirement |
|---|---|---|
| Independent, heteroskedastic observations | Observation-level wild multipliers | Observations are independent across individuals |
| Clustered observations | One wild multiplier per cluster | Clusters are independent; within-cluster dependence is retained |
| Independent, identically distributed observations | Residual or pairs bootstrap may be considered under its assumptions | Resampling units match the independent sampling units |
A pairs bootstrap resamples whole independent sampling units and can be reasonable when the design is an independent random sample. For clustered data, resampling individuals separately breaks the cluster structure; if a pairs bootstrap is used, it must resample whole clusters. Wild methods are often attractive for robust testing because they keep the observed covariates and instruments fixed and impose the null directly. No scheme is automatically reliable with very few independent units: small-cluster corrections or specialized methods may be needed, and bootstrap output should not be treated as a cure for sparse cluster information.
Worked Example: Heteroskedastic Wild Bootstrap p-Value
Suppose a hypothetical study has independent observations, one instrument, and a heteroskedasticity-robust AR statistic of \(4.84\) for \(H_0:\beta=0.30\) years of outcome change per treatment unit. The restricted regression is \(W=Y-0.30D\) on \(X\). The analyst uses 999 observation-level Rademacher wild-bootstrap replicates, recomputing the heteroskedasticity-robust AR statistic each time. In 83 replicates, the statistic is at least \(4.84\).
The finite-replicate p-value estimate, counting the observed statistic in the numerator and denominator, is
At \(\alpha=0.05\), this procedure does not reject \(H_0:\beta=0.30\). The conclusion is not that the null is true; the bootstrap evidence is insufficient to reject it at this significance level. The design supports observation-level multipliers because observations are stipulated to be independent, and the robust statistic allows unequal variances. The p-value is an estimate based on the finite set of replicates, not an exact probability.
Worked Example: Cluster-Wild Bootstrap With Multiple Instruments
Consider a hypothetical study of 24 independent clinics, each contributing several patients. There are two excluded instruments, and the candidate effect is \(\beta_0=1.2\) outcome points per treatment unit. The observed cluster-robust AR statistic is \(8.10\). The analyst fits the restricted regression of \(Y-1.2D\) on \(X\), then uses one Rademacher multiplier per clinic, shared by every patient in that clinic. In \(B=1999\) replicates, 17 statistics are at least \(8.10\).
The bootstrap p-value is
Since \(0.009<0.05\), reject the joint AR null at the 5% level. This is evidence against the candidate effect of 1.2 points under the maintained IV assumptions. The shared clinic multiplier retains each clinic’s residual pattern; independent patient-level multipliers would fail to reflect within-clinic dependence. The 24 clinics, not the larger patient count, determine the number of independent multiplier draws, so the modest cluster count remains a caution about finite-sample calibration.
Inverting the Bootstrap Test
A confidence set for \(\beta\) is obtained by testing candidate values and retaining those not rejected. For a level-\(\alpha\) test, the set is \(C_{1-\alpha}=\{\beta_0:\hat p_{\mathrm{boot}}(\beta_0)>\alpha\}\), with the boundary convention chosen consistently with the test. Each candidate requires its own null-adjusted outcome \(Y-\beta_0D\), restricted fit, and bootstrap distribution. A weak instrument can produce an unbounded or disconnected confidence set; do not force the result into a single bounded interval.
Worked Example: A Bootstrap Confidence-Set Grid
In a hypothetical analysis, the outcome is measured in days and \(D\) is a one-unit treatment indicator, so candidate effects are in days per treatment unit. The analyst uses the same 1,000-replicate heteroskedastic wild-bootstrap procedure at each candidate. At \(\beta_0=2\), 214 replicate statistics equal or exceed the observed statistic; at \(\beta_0=5\), 43 do; and at \(\beta_0=8\), 176 do.
The estimated p-values are
At the 5% level, the grid accepts 2 and 8 days per treatment unit and rejects 5 days per treatment unit. This pattern warns that a single interval may not describe the accepted values. Three grid points do not establish the exact boundaries or prove that the full confidence set is disconnected; a denser search and careful boundary determination are required. Any reported set must be based on the full inversion, not just these illustrative candidate points.
Diagnostics, Limitations, and a Common Mistake
Before interpreting the result, verify that the resampling units match the sampling design, that the restricted regression excludes the instruments, and that the bootstrap and observed statistics use the same instrument restrictions and covariance type. Inspect the number of independent observations or clusters, the distribution of restricted residuals, and whether a small number of high-leverage units dominates the fit. Centering residuals and using leverage adjustments, such as dividing by \(\sqrt{1-h_i}\) for an observation with leverage \(h_i\), can improve the bootstrap approximation, but they do not guarantee accuracy.
A common mistake is to resample each patient independently in a clustered study because the data contain many patients. That treats dependent patients as independent, usually understating uncertainty. The correction is to preserve the cluster as the resampling unit, for example by applying one multiplier to all restricted residuals in a cluster and using cluster-robust covariance estimation. Another mistake is to assume that bootstrapping makes a weak-IV analysis automatically reliable. AR testing is robust to weak relevance under its conditions, but neither the test nor the bootstrap repairs invalid exclusion restrictions, misspecified structural assumptions, or too few independent clusters.
Compared with the asymptotic robust AR test, bootstrap calibration changes the reference distribution, not the tested null or the AR construction. Compared with a conventional Wald test, AR inference does not require a strong first stage for its weak-instrument robustness. Bootstrap computation is more demanding and remains approximation-based; it should complement, not replace, scrutiny of instrument validity and the sampling design.
Check Your Understanding
Answer each question, then compare with the brief solutions.
- For a candidate \(\beta_0\), which regression supplies residuals for a null-imposing AR bootstrap?
- A heteroskedastic bootstrap uses 1,499 replicates, of which 59 statistics are at least as large as the observed statistic. Calculate the bootstrap p-value and state the 5% decision.
- In a study with 18 independent schools and students nested within schools, should wild multipliers vary by student or school? Why?
- Why is an unrestricted-regression residual bootstrap generally unsuitable for testing the AR null?
- At a 5% level, a dense inversion accepts candidate values in two separated ranges. Should these be replaced by the interval spanning both ranges?
Solutions. (1) Regress \(Y-\beta_0D\) on \(X\) only; the excluded instruments are omitted under the null. (2) \(\hat p=(1+59)/(1499+1)=60/1500=0.040\), so reject at 5%. (3) Use one multiplier per school, because students within a school may have dependent errors and the school is the independent sampling unit. (4) The unrestricted fit retains the instrument relationship that the null says should be absent, so it does not generate the intended null distribution. (5) No. Report the two accepted ranges as a possibly disconnected confidence set; filling the gap would add rejected values and misstate the inversion result.