Why Validity Does Not Guarantee Power
A weak first stage creates a familiar difficulty in instrumental-variables analysis: conventional IV Wald tests can have unreliable size. The Anderson–Rubin (AR) test addresses that problem by testing a proposed structural coefficient through a regression involving the outcome after subtracting that proposed effect times the endogenous regressor. Under the null, the test’s distribution does not require a strong first stage. That is a statement about size, not about the chance of detecting a false null.
Power depends on how strongly the tested null changes the distribution of the outcome-adjusted regression. If the instruments predict little of the endogenous regressor, a false coefficient can produce only a small shift in that regression. The AR test may then reject infrequently, even though its null rejection rate is controlled. The distinction matters in practice: a non-rejection can reflect limited information rather than evidence that the proposed coefficient is correct.
Setup and the Source of Power
Consider a linear structural equation \(y=d\beta+X\gamma+u\), where \(y\) is the outcome, \(d\) is an endogenous regressor, \(X\) contains included covariates, and \(Z\) is a vector of \(L\) excluded instruments. The first stage can be written \(d=Z\pi+X\delta+v\). The AR test evaluates \(H_0:\beta=\beta_0\) by forming \(q(\beta_0)=y-\beta_0d\), regressing \(q(\beta_0)\) on \(X\) and \(Z\), and testing whether the coefficients on all \(L\) instruments are zero.
The interpretation requires that the instruments be excluded from the structural outcome equation and exogenous: conditional on \(X\), they have zero association with the structural error \(u\). The instruments must also be appropriately specified, and the test’s covariance estimator must reflect the sampling design. For the classical finite-sample result below, assume fixed regressors, independent normally distributed errors with constant variance, and a full-rank design. With heteroskedasticity or clustering, use the corresponding robust covariance estimator; the usual justification is then asymptotic, under its regularity conditions, rather than the classical exact \(F\) result.
Under the null, \(q(\beta_0)=X\gamma+u\). Thus, the tested instrument coefficients are zero, regardless of whether \(\pi\) is large, small, or zero. Under a false null, writing \(\Delta=\beta-\beta_0\), we have \(q(\beta_0)=\Delta d+X\gamma+u\). The first-stage component \(Z\pi\) then induces an instrument coefficient approximately equal to \(\Delta\pi\). The greater the instrument-predicted variation in \(d\), and the farther \(\beta_0\) is from \(\beta\), the more distinguishable the false null tends to be.
Why the Null Distribution Is Robust to Weak Instruments
The size property follows directly from what is being tested. At the true null value, subtracting \(\beta_0d\) removes the structural contribution of \(d\), leaving \(X\gamma+u\). The first-stage coefficient \(\pi\) does not appear in this equation. Consequently, the null distribution of the joint test of the \(Z\) coefficients is not made invalid merely because the instruments weakly predict \(d\).
In the classical case, residualize the transformed outcome and the instruments with respect to \(X\). Under the null, the residualized outcome is the residualized structural error. Conditional on the regressors, normality and constant variance give the usual \(F\) test for whether the coefficients on the residualized instruments are jointly zero. Its reference distribution is \(F_{L,n-K-L}\), whether the first-stage coefficients are large or small. The weak-first-stage advantage is therefore a consequence of testing a restriction whose null distribution does not use an approximation based on a well-estimated IV coefficient.
For a power approximation under a false null, suppose the residualized first stage is \(d=Z\pi+v\), and use the classical homoskedastic normal setup. The instrument coefficient in the transformed-outcome regression is \(\theta=\Delta\pi\). Let \(M_X\) denote residualization with respect to \(X\), and let \(\sigma_q^2\) be the variance of the error in that regression under the alternative. Its noncentrality parameter is
The classical \(F\) statistic then has a noncentral \(F\) distribution with numerator degrees of freedom \(L\), denominator degrees of freedom \(n-K-L\), and noncentrality \(\lambda\). Power at level \(\alpha\) is the probability that this statistic exceeds the central \(F\) critical value. In the large-sample robust formulation, the analogous approximation is a noncentral \(\chi^2_L(\lambda)\) statistic exceeding the central \(\chi^2_L\) critical value. The formula makes the main trade-off visible: with other features fixed, weaker instrument-predicted variation in \(d\) reduces \(\lambda\), and a smaller \(|\Delta|\) also reduces it.
Worked Power Calculations
Worked Example: Weak Relevance and Low Power
Suppose a hypothetical large-sample, robust AR test uses two instruments and has significance level \(\alpha=0.05\). Its central \(\chi^2_2\) critical value is 5.991. At a particular false null, suppose the noncentrality parameter is \(\lambda=0.25\). What is the approximate power?
For a noncentral chi-square variable, the distribution is a Poisson-weighted mixture of central chi-square distributions. If \(J\sim\operatorname{Poisson}(\lambda/2)\), then the mixture components have \(2+2J\) degrees of freedom. Here \(J\) has mean \(0.125\), so most of the mixture is the central \(\chi^2_2\) distribution. The upper-tail probabilities at 5.991 for the first few components are approximately 0.0500 for \(\chi^2_2\), 0.1998 for \(\chi^2_4\), and 0.4241 for \(\chi^2_6\). The mixture weights begin \(P(J=0)=0.8825\), \(P(J=1)=0.1103\), and \(P(J=2)=0.0069\). Weighting the component tails, including the small remaining mixture terms, gives power of approximately 0.069.
Thus, although the test is calibrated at a 5% null rejection rate, it rejects this false null only about 6.9% of the time under the approximation. This is low power: a typical sample has little chance of distinguishing the stated alternative from the null when the first-stage signal is weak.
Worked Example: A Larger Signal Raises Power
Keep the same hypothetical two-instrument test, significance level, and critical value 5.991, but suppose a more relevant first stage and the same effect difference produce \(\lambda=4\). Calculate the approximate power using the same mixture representation.
Now \(J\sim\operatorname{Poisson}(2)\), with initial weights \(0.1353,\ 0.2707,\ 0.2707,\ 0.1804,\ 0.0902,\ldots\) for \(J=0,1,2,3,4,\ldots\). At 5.991, the corresponding central chi-square upper-tail probabilities begin \(0.0500,\ 0.1998,\ 0.4241,\ 0.6479,\ 0.8156,\ldots\). Multiplying each component tail by its Poisson weight and summing the mixture gives
The approximate power is therefore 41.5%, substantially higher than in the preceding example but still below one half. Under this alternative, repeated samples would reject the false null in about 41.5% of cases. The calculation illustrates that size robustness alone does not make a test powerful.
Worked Example: Strong Relevance and the Same Test
Suppose a third hypothetical setting retains two instruments and the 5% critical value 5.991, but the first stage is much more informative, producing \(\lambda=25\) for the same coefficient difference. The noncentral mixture now has \(J\sim\operatorname{Poisson}(12.5)\). Most of its weight is on components with many degrees of freedom, whose upper-tail probabilities at 5.991 are close to one. Summing the mixture gives power of approximately 0.996.
This is high power for the specified alternative: the test rejects in nearly all repeated samples. The conclusion depends on the chosen alternative and the assumed model, not merely on the label “strong instrument.” A smaller coefficient difference would lower \(\lambda\) and could reduce power even with the same first stage.
Instrument Count, Conservatism, and Diagnostics
The number and quality of instruments both affect power. In the classical approximation, adding instruments changes the numerator degrees of freedom and can also change the noncentrality parameter. A noisy or weak additional instrument may add little signal while increasing the critical value. For example, at a 5% level the \(\chi^2_1\) critical value is 3.841, whereas the \(\chi^2_2\) critical value is 5.991. If the noncentrality were held fixed at 4, the respective powers would be approximately 0.516 and 0.415. This comparison isolates the cost of an additional tested restriction; in real applications, adding an instrument can also change the signal and covariance estimate, so the net effect is not determined by degrees of freedom alone.
“Conservative” needs careful interpretation. Under the classical assumptions, the finite-sample \(F\) test is calibrated to its stated level; it is not inherently conservative simply because instruments are weak. In finite samples, a robust asymptotic test can be conservative or liberal if its approximation or covariance estimate is poor. High-dimensional instrument sets, limited clusters, leverage, or a very small effective sample can all complicate calibration. A conservative test rejects less often than its nominal level under the null; this does not mean it has greater power.
Useful checks include inspecting first-stage strength with diagnostics appropriate to the design, examining instrument leverage and the size of the instrument set, and verifying that the robust covariance estimator matches the heteroskedastic or clustered sampling structure. Those checks do not establish instrument validity. Exclusion and exogeneity remain substantive assumptions; a large first-stage statistic cannot verify them. Conversely, weak first-stage diagnostics warn about limited power but do not invalidate the AR test’s null justification under its assumptions.
The main alternative, conventional IV Wald inference, can be much more powerful when the first stage is strong and its approximations are adequate. But its usual reference distribution can be badly misleading with weak instruments. AR is attractive when reliable weak-instrument-robust testing is a priority, at the cost of potentially low power and, with multiple instruments, a joint test that may be less informative about which instrument contributes signal. Instrument selection should be guided by design and validity, not by searching for a set that happens to produce rejection.
Common Pitfall
Correction: Weak-instrument robustness refers to null size control, not guaranteed power. If the first stage is weak, the test may rarely reject even when the null is false. Report the test in context, consider its power for scientifically meaningful alternatives, and distinguish failure to reject from evidence of equivalence.
State \(H_0:\beta=\beta_0\), the instrument exogeneity and exclusion conditions, and the covariance approach justified by the design.
Examine first-stage relevance, instrument count, and diagnostics for leverage or dependence that could undermine the approximation.
Explain why the null test is robust under its assumptions, then assess whether the available signal gives useful power against relevant alternatives.
Check Your Understanding
Answer each question, then compare with the solutions below.
- Why does the classical AR null distribution not depend on the first-stage coefficient \(\pi\)?
- For a two-instrument robust AR test at the 5% level with observed statistic 4.2, use the \(\chi^2_2\) reference distribution to decide whether to reject.
- If the instrument-predicted variation and residual variance stay fixed, what happens to the noncentrality parameter when the absolute difference \(|\beta-\beta_0|\) is halved?
- Does a correctly sized AR test necessarily have high power? Explain briefly.
Solutions. (1) Under the null, \(y-\beta_0d=X\gamma+u\), so the restriction being tested concerns the coefficients on \(Z\) in a regression whose null error is the structural error; \(\pi\) does not enter that null equation. (2) The upper-tail probability is \(P(\chi^2_2>4.2)=\exp(-4.2/2)\approx0.1225\). Since \(0.1225>0.05\), do not reject \(H_0\) at the 5% level. This is not evidence that the null is true. (3) Since \(\lambda\) is proportional to \((\beta-\beta_0)^2\), halving the absolute difference reduces \(\lambda\) to one quarter, all else equal. (4) No. Size describes rejection under the null; power describes rejection under a specified alternative. Weak relevance can yield low power while null size remains controlled.