Inverting Anderson-Rubin Tests for Confidence Sets

From a Test to a Set of Plausible Effects

An Anderson–Rubin (AR) test asks whether a particular proposed treatment effect is compatible with the observed relationship between the instrument and the outcome. Inverting that test means repeating the test for candidate treatment effects and retaining those not rejected. The result is a confidence set for the structural treatment effect. It need not have the familiar form of a short interval centered on an estimate; its shape reflects the information in the tests.

This approach matters when the instrument may be weak. A conventional IV Wald interval relies on an estimate whose behavior can be unreliable when the first stage is weak. AR inversion instead uses a test whose size does not require strong instrument relevance. The introductory material on Anderson–Rubin tests establishes the candidate-value testing idea; here the focus is how its non-rejection decisions produce a confidence set.

Definition: Let $\beta$ be the scalar structural treatment effect, and let $p_{AR}(\beta_0)$ be the p-value from an AR test of $H_0:\beta=\beta_0$. At significance level $\alpha$, the inverted AR confidence set is $$ C_{1-\alpha}=\{\beta_0:p_{AR}(\beta_0)>\alpha\}. $$ Equivalently, it contains every candidate value whose AR test does not reject at level $\alpha$. A candidate with $p_{AR}(\beta_0)=\alpha$ is rejected under the usual rule that rejects when $p\leq\alpha$.

The Candidate-Value Test and Its Assumptions

Consider the linear structural model

Y_i=\beta D_i+X_i'\gamma+u_i,

where $Y_i$ is the outcome, $D_i$ is the endogenous treatment, $X_i$ is a vector of included covariates, and $u_i$ is the structural error. Let $Z_i$ be a vector of $q$ excluded instruments. For a proposed value $\beta_0$, form the transformed outcome $Y_i-\beta_0D_i$. Under $H_0:\beta=\beta_0$, this becomes

Y_i-\beta_0D_i=X_i'\gamma+u_i.

The AR test asks whether the excluded instruments help explain this transformed outcome after accounting for $X_i$. In the classical homoskedastic test, this is the usual partial $F$-test of the joint null that the coefficients on all $q$ excluded instruments are zero. A heteroskedasticity-robust or cluster-robust version replaces the classical variance calculation with the corresponding robust test from the earlier material on those AR tests.

The key assumptions are that the structural equation and included covariates are correctly specified for the parameter of interest, and that the excluded instruments are exogenous: conditional on $X_i$, they are uncorrelated with $u_i$. For a causal interpretation, the instruments must also satisfy the relevant exclusion restriction and the treatment-effect model must represent the causal effect being targeted. The classical finite-sample $F$ test additionally requires independent, homoskedastic normal errors and a full-rank regression design. With robust tests, the required sampling and variance-estimation conditions are those for the chosen robust procedure, such as independent sampling units or appropriately handled clusters.

Instrument relevance is not needed to control the size of the AR test under the null. This is why the procedure remains useful with weak instruments. Relevance still matters for how informative the resulting set is: weak or absent first-stage information can leave many candidate values unexcluded.

Why Inversion Gives Confidence Coverage

Suppose the AR test has size at most $\alpha$ for every candidate null value under the stated assumptions. In particular, when the true value is $\beta$, the test of $H_0:\beta=\beta$ rejects with probability at most $\alpha$. The true value is outside the inverted set exactly when this true-value test rejects. Therefore,

P(\beta\in C_{1-\alpha}) =1-P(\text{reject }H_0:\beta=\beta) \geq 1-\alpha.

If the test has exact size $\alpha$, the coverage is $1-\alpha$; if it is conservative, coverage is at least $1-\alpha$. For asymptotic robust tests, this is an asymptotic coverage statement under the conditions that make the test’s size valid. This argument concerns rejection at the true parameter, not simultaneous control of errors at every candidate value. No extra multiple-testing adjustment is needed to obtain this coverage result.

Worked Example: Inverting a One-Instrument Wald AR Test

Suppose a hypothetical study reports reduced-form coefficient estimates $\hat{\pi}_Y=0.80$ and $\hat{\pi}_D=0.40$, with estimated variances $0.04$ and $0.01$, and covariance $0.01$. For each candidate $\beta_0$, the reduced-form contrast is $\hat{\pi}_Y-\beta_0\hat{\pi}_D$, with estimated variance $0.04+\beta_0^2(0.01)-2\beta_0(0.01)$. Use a two-sided 5% Wald AR test, whose critical value is $1.96$.

The non-rejection condition is that the squared test statistic is no larger than $1.96^2=3.8416$:

\frac{(0.80-0.40\beta_0)^2}{0.04+0.01\beta_0^2-0.02\beta_0}\leq 3.8416.

The denominator is positive for every $\beta_0$, since it equals $0.01[(\beta_0-1)^2+3]$. Multiplying through and collecting terms gives

0.121584\beta_0^2-0.563168\beta_0+0.486336\leq 0.

The two roots are approximately $1.148$ and $3.484$. Since the quadratic opens upward, it is nonpositive between the roots. Thus, for this hypothetical Wald calculation, the inverted 95% confidence set is approximately $[1.148,3.484]$ treatment-effect units. Each candidate inside that range is not rejected at 5%; values outside it are rejected. The calculation relies on the stated Wald approximation and covariance estimates, rather than claiming that the interval is an exact finite-sample result.

Evaluating Candidates in Practice

For multiple instruments, the candidate-value null is joint: all excluded-instrument coefficients in the transformed-outcome regression must be zero. With a robust Wald statistic, write $\hat r(\beta_0)$ for the vector of estimated excluded-instrument coefficients and $\widehat V(\beta_0)$ for its estimated covariance matrix. The statistic is

W(\beta_0)=\hat r(\beta_0)'\widehat V(\beta_0)^{-1}\hat r(\beta_0).

Under the null and appropriate large-sample conditions, it is compared with a chi-square distribution with $q$ degrees of freedom. The p-value, or equivalently the critical-value comparison, determines whether $\beta_0$ belongs in the set. With a classical partial $F$-test, use its relevant numerator and residual degrees of freedom instead.

Worked Example: Testing Several Candidate Values With Two Instruments

In hypothetical reduced-form output with two instruments, suppose the estimated coefficient vectors are $\hat\pi_Y=(0.60,0.20)'$ and $\hat\pi_D=(0.30,0.10)'$. Suppose their estimated covariance matrices are $\widehat{\mathrm{Var}}(\hat\pi_Y)=0.04I$, $\widehat{\mathrm{Var}}(\hat\pi_D)=0.01I$, and $\widehat{\mathrm{Cov}}(\hat\pi_Y,\hat\pi_D)=0.01I$, where $I$ is the two-dimensional identity matrix. These imply $\widehat V(\beta_0)=(0.04+0.01\beta_0^2-0.02\beta_0)I$ for the contrast $\hat\pi_Y-\beta_0\hat\pi_D$. Use the 5% chi-square critical value $5.991$ with $q=2$ degrees of freedom.

At $\beta_0=0$, the contrast is $(0.60,0.20)'$, and its covariance is $0.04I$. Therefore,

W(0)=\frac{0.60^2+0.20^2}{0.04} =\frac{0.40}{0.04}=10.

Because $10>5.991$, reject this candidate at 5% (its chi-square p-value is approximately $0.0067$). At $\beta_0=1$, the contrast is $(0.30,0.10)'$, and the covariance is $0.03I$, so

W(1)=\frac{0.30^2+0.10^2}{0.03} =\frac{0.10}{0.03}=3.333.

Since $3.333<5.991$, do not reject $\beta_0=1$; its p-value is approximately $0.189$. At $\beta_0=2$, the contrast is $(0,0)'$, so $W(2)=0$, which is also not rejected. These evaluations show how candidate decisions are made; the full confidence set requires evaluating all real candidate values, not just these three.

Worked Example: A Candidate Test With Included Covariates

Suppose hypothetical data contain $n=40$ observations, an intercept and two included covariates, one excluded instrument, and a scalar treatment. For each candidate $\beta_0$, regress $Y-\beta_0D$ on the included covariates and the instrument. The classical partial test has numerator degrees of freedom $1$ and residual degrees of freedom $40-4=36$.

Suppose the instrument coefficient estimate at $\beta_0=0$ is $0.70$, with standard error $0.15$. With one restriction, $F=t^2$, so

F(0)=\left(\frac{0.70}{0.15}\right)^2=21.78.

The 5% critical value for $F_{1,36}$ is approximately $4.11$, so this candidate is rejected. At $\beta_0=2$, suppose the instrument coefficient is $0.30$, with the same standard error. Then

F(2)=\left(\frac{0.30}{0.15}\right)^2=4.00<4.11.

This candidate is not rejected at 5%. The conclusion applies only under the classical test assumptions stated above; if heteroskedasticity is a concern, the corresponding robust AR test should be used instead.

Computing, Diagnosing, and Interpreting the Set

In computation, the test should be run on $Y-\beta_0D$ for every candidate, with the included covariates and instrument set held fixed. One can evaluate a fine grid, retain candidates with $p_{AR}(\beta_0)>\alpha$, and refine near changes in the decision. A grid is a numerical approximation: it can miss a narrow retained region or make a boundary appear more precise than the calculations support. For reported results, state the test version, significance level, instrument set, and numerical method used to determine the set.

Several checks help identify errors. Confirm that the transformation uses $Y-\beta_0D$, not $Y-\hat\beta D$; that the null tests all excluded instruments jointly; and that included covariates are treated consistently. Check that the reference distribution and degrees of freedom match the chosen classical or robust test. For a scalar treatment, the resulting set should contain a candidate exactly when its test decision is non-rejection, apart from numerical boundary tolerance. If a candidate’s reported p-value and inclusion flag disagree, the inversion has been implemented incorrectly.

A common mistake is to calculate a single AR test and call its p-value a confidence interval. A test answers whether one specified candidate is rejected; a confidence set requires the collection of decisions over candidate values. Another mistake is to take a conventional IV estimate and attach a Wald interval because it looks simpler. That procedure is not AR inversion and can have poor coverage with weak instruments. Conversely, a wide or irregular AR set is not automatically evidence of a coding failure; it can reflect weak information. Detailed classifications of set shapes belong to a separate analysis, but any unusual output should first prompt checks of the candidate regression, covariance estimate, and numerical search.

Compared with a conventional IV Wald interval, the inverted AR set can preserve validity without strong instrument relevance, at the cost of being less compact or less familiar. The conventional interval is often more concise when its approximations are trustworthy. AR inversion does not repair invalid instruments, misspecification, poor clustering choices, or an inappropriate causal model. Its coverage is only as credible as the candidate tests and their assumptions.

Key takeaway: An AR confidence set is the set of candidate treatment effects that survive the specified AR test. Its coverage follows from the test’s size at the true value, and that validity does not require a strong first stage.

Check Your Understanding

Answer each question, then compare your reasoning with the brief solutions.

At level $\alpha=0.05$, an AR test of $\beta_0=1.5$ has p-value $0.03$. Is $1.5$ in the inverted set?
Why does inverting a valid level-$\alpha$ test give at least $1-\alpha$ coverage?
A two-instrument robust Wald statistic is $6.20$. At 5%, the chi-square critical value with two degrees of freedom is $5.991$. Is that candidate retained?
Does AR test validity require a strong instrument? What can weak relevance still affect?
Why is testing only the estimated IV coefficient insufficient to construct an AR confidence set?

Solutions. (1) No. Since $0.03\leq0.05$, the candidate is rejected. (2) The true value is excluded exactly when its own null test rejects, an event with probability at most $\alpha$; hence coverage is at least $1-\alpha$. (3) No. Since $6.20>5.991$, reject the candidate. (4) No strong relevance is required for test size, but weak relevance can make the retained set less informative. (5) A set requires testing candidate values throughout the parameter space; a test at one value, including the point estimate, does not provide those decisions.