Three Questions, Three Different Diagnostics
Instrument diagnostics are often compressed into one question—“Are the instruments strong?”—but a careful analysis separates three questions. Do the instruments predict the endogenous regressors? Is the model identified at all, in the sense that the instrument–regressor relationships have sufficient rank? And can the effect of interest be tested or estimated reliably even if instrument strength is limited? Different procedures answer these different questions. A statistic designed for one should not be treated as an answer to the others.
The appropriate choice also depends on the number of endogenous regressors and on the error structure. In particular, a conventional first-stage F statistic and its familiar reference values rely on assumptions that may not hold with heteroskedastic or clustered errors. The earlier material on Anderson–Rubin tests, conditional likelihood ratio tests, and weak-IV-robust score tests provides procedures for inference; the task here is to select diagnostics and inference methods suited to the design.
In practice, these procedures are based on estimated relationships and sampling assumptions. The structural interpretation additionally requires that instruments be valid: in particular, they must not be associated with the structural error after conditioning on \(X\), and they must affect \(Y\) through the endogenous regressors rather than through a direct pathway. A strong but invalid instrument does not produce a valid causal estimate. Diagnostics of relevance and rank do not test these substantive assumptions.
Match the Diagnostic to the Question
| Question | Typical diagnostic | What the result does not establish |
|---|---|---|
| Do excluded instruments predict an endogenous regressor? | First-stage test; with multiple endogenous regressors, conditional first-stage measures such as the Sanderson–Windmeijer conditional F statistics | That the instruments are valid, or that weak-instrument distortions are negligible |
| Does the instrument–regressor system have enough rank for identification? | An underidentification test, such as a heteroskedasticity-robust Kleibergen–Paap rk LM test | That the identified relationships are strong enough for conventional inference |
| Can a specified structural effect be tested despite possible weakness? | A weak-IV-robust test, such as an Anderson–Rubin test with a covariance estimator appropriate to the errors | That the instrument assumptions hold, or that the test will be highly informative |
When \(K=1\), the first stage can be summarized by a joint test that the \(L\) excluded-instrument coefficients are zero. Under independent, homoskedastic errors, the conventional first-stage F statistic is often reported, along with the partial \(R^2\), which describes the fraction of residual variation in \(D\) explained by \(Z\) after accounting for \(X\). Neither measure automatically answers whether conventional IV inference is reliable: interpreting strength requires a reference framework whose assumptions match the design.
For example, Stock–Yogo critical values are tied to particular homoskedastic settings and criteria such as maximum bias or test-size distortion. They are not a universal cutoff that can be applied to every first-stage statistic. In particular, the common rule “F greater than 10 means strong” is not a general guarantee. Under heteroskedasticity, a heteroskedasticity-robust first-stage Wald test or a Kleibergen–Paap rk Wald F statistic may be appropriate, but its value should not be compared mechanically with critical values derived for a different statistic or error model.
With \(K>1\), a single joint first-stage statistic can conceal the fact that one endogenous regressor is weakly predicted conditional on the others. Sanderson–Windmeijer conditional F statistics assess the excluded instruments’ predictive contribution for each endogenous regressor, conditional on the other endogenous regressors and included covariates. They help locate a weak link in the system, but their interpretation and reference values still depend on the setting and assumptions. An overall test that the instrument coefficient matrix has full rank addresses a different question from the conditional strength of each first stage.
Worked Example: One Endogenous Regressor With Heteroskedastic Errors
Suppose a hypothetical study has one endogenous treatment \(D\), two excluded instruments, and included covariates \(X\). The analyst expects heteroskedastic errors and reports a heteroskedasticity-robust first-stage Wald statistic of \(\chi^2(2)=36.8\). For this joint test, the equivalent Wald F scaling is
This result is evidence that the excluded instruments jointly predict \(D\), conditional on \(X\). It is not a test of instrument validity, and the number 18.4 should not be declared sufficient by invoking a universal threshold. Because the errors are expected to be heteroskedastic, the conventional homoskedastic first-stage F reference and critical values are not automatically appropriate. The analyst should use a robust diagnostic suited to the model and, for inference about \(\beta\), consider a weak-IV-robust procedure with a compatible covariance treatment.
Underidentification Is Not Weak Identification
An underidentification test asks whether the instrument–regressor relationships have insufficient rank to identify all \(K\) endogenous-regressor coefficients. The null is rank deficiency; rejection is evidence against underidentification. For heteroskedastic settings, the Kleibergen–Paap rk LM test is a commonly used rank test. Under homoskedastic assumptions, the Cragg–Donald statistic and related reference frameworks may be used. The error assumptions matter: a test derived under homoskedasticity should not be treated as automatically valid under heteroskedasticity or clustering.
Rejection of underidentification does not establish that the instruments are strong. A relationship can be nonzero and therefore satisfy the rank condition while still being weak enough to cause substantial finite-sample problems for conventional IV estimates and tests. Conversely, a failure to reject underidentification is not proof that the model is underidentified; it may reflect limited information in the sample. Thus rank testing and strength assessment should be reported as distinct steps.
Worked Example: Several Endogenous Regressors
Consider a hypothetical model with two endogenous regressors, \(D_1\) and \(D_2\), three excluded instruments, and included covariates. Suppose heteroskedasticity-robust output gives Sanderson–Windmeijer conditional F statistics of 14.2 for \(D_1\) and 6.10 for \(D_2\). A heteroskedasticity-robust underidentification test reports \(\chi^2(2)=10.9\), with \(p=0.0043\).
The underidentification test rejects its rank-deficiency null at the 5% level because \(0.0043<0.05\). This supports the conclusion that the system is not underidentified according to that test. It does not show that both endogenous regressors are strongly predicted. The conditional first-stage results flag \(D_2\) as the more concerning relationship: its conditional F statistic, 6.10, is notably smaller than the value for \(D_1\). It would be misleading to summarize the design only by the rejected rank test or by an overall first-stage statistic.
The appropriate next step is to choose inference that does not depend on treating both first stages as sufficiently strong. For a specified hypothesis about the structural coefficients, the analyst can use a weak-IV-robust test that is appropriate for the error structure. The diagnostics alone do not establish validity, and the values given do not justify applying a cutoff from an incompatible reference framework.
Choose Inference for the Error Structure and Goal
A weak-IV-robust procedure is chosen because it controls inference without requiring the instruments to be strong in the way conventional 2SLS approximations do. For a proposed value \(\beta_0\), the Anderson–Rubin logic tests whether the excluded instruments predict \(Y-D\beta_0\), conditional on \(X\). Under \(H_0:\beta=\beta_0\), the structural equation implies \(Y-D\beta_0=X\gamma+u\). If the instruments are valid, their coefficients in this transformed equation should be zero. This gives the core justification for testing the excluded-instrument coefficients rather than relying on a well-behaved 2SLS estimate.
With homoskedastic normal errors, an appropriately constructed Anderson–Rubin test has an exact finite-sample reference distribution under the null. With heteroskedastic errors, use a heteroskedasticity-robust covariance estimator; with clustered errors, use a cluster-robust version that respects the sampling design and the number of independent clusters. These robust procedures generally rely on suitable large-sample conditions, so the choice of covariance estimator does not remove every concern, especially with few clusters.
The inferential goal matters too. If the goal is to test one specified value of a scalar coefficient, a robust Anderson–Rubin test is a direct option. Inverting such tests over candidate values produces a confidence set, which need not be a single bounded interval. Conditional likelihood ratio methods can offer greater power in settings where their assumptions and implementation are appropriate, but they are not a universal replacement for robust procedures under every error structure or with every number of endogenous regressors. For a joint hypothesis involving several coefficients, the test must be formulated for that joint parameter restriction; a diagnostic for one first stage is not a substitute.
Worked Example: Clustered Errors and a Coefficient Test
Suppose a hypothetical policy analysis has one endogenous treatment, two excluded instruments, and observations grouped in 24 clinics. The target is a test of \(H_0:\beta=0\), and outcomes may be correlated within clinics. An Anderson–Rubin regression for the null yields a clinic-cluster-robust statistic \(F(2,23)=1.46\), with \(p=0.253\). A conventional 2SLS report also gives a coefficient \(0.42\) outcome units per treatment unit and a nominal \(p=0.042\), but that conventional p-value is not the weak-IV-robust test of the stated null.
For the cluster-robust Anderson–Rubin result, \(p=0.253>0.05\), so at the 5% level the data do not reject \(H_0:\beta=0\) using this weak-IV-robust procedure. This is not evidence that the null is true. Rather, the test does not provide sufficient evidence against it under the specified procedure. The clinic clustering is reflected in the covariance calculation and the stated \(23\) denominator degrees of freedom; treating all observations as independent would not match the assumed error structure.
The conventional 2SLS p-value and the Anderson–Rubin p-value answer different inferential questions under different robustness properties. The conventional result cannot replace the cluster-robust weak-IV-robust test when the objective is inference that remains reliable under weak instruments and within-clinic dependence. The diagnostic choice also does not resolve whether the instruments satisfy the exclusion restriction.
Practical Decision Sequence and Failure Modes
For one regressor, inspect the excluded-instrument first stage. For multiple regressors, examine conditional first-stage diagnostics as well as system-level rank evidence.
Decide whether the design supports homoskedastic, heteroskedastic-robust, or cluster-robust inference. Use diagnostics and tests with covariance estimates appropriate to that structure.
Use an underidentification test to assess rank deficiency, and first-stage strength measures to assess predictive relationships. Neither result establishes instrument validity.
For a coefficient hypothesis or confidence set under possible weakness, use a weak-IV-robust procedure suited to the error structure and parameter restriction.
Several recurring mistakes follow from skipping one of these steps. Treating a significant underidentification test as proof of strong instruments confuses rank with strength. Applying Stock–Yogo values to a heteroskedasticity-robust statistic confuses statistics with different assumptions and reference distributions. Reporting only an overall first-stage statistic with multiple endogenous regressors can obscure a weak conditional first stage. Finally, a weak-instrument diagnostic cannot validate exclusion or independence: those require substantive design arguments and, where possible, separate sensitivity analysis.
Diagnostics are useful for describing where an IV design may be fragile, but they do not turn a borderline instrument into a reliable one. A clear report identifies the statistic, the null it tests, its error assumptions, and the inferential procedure used for the effect. If the diagnostic framework and the final inference use incompatible assumptions, their numbers should not be combined as though they formed one coherent decision rule.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- What is the difference between an underidentification test and a first-stage strength diagnostic?
- A model has three endogenous regressors. Why can one overall first-stage statistic be inadequate as a summary?
- Why should a conventional homoskedastic first-stage F statistic not automatically be used with clustered errors?
- For a specified coefficient value, what does an Anderson–Rubin test examine?
- A rank test rejects underidentification, but one conditional first-stage statistic is small. What conclusion is justified?
Solutions. An underidentification test assesses whether the instrument–regressor relationships have sufficient rank; a strength diagnostic assesses how predictive those relationships are. With three endogenous regressors, the overall statistic can hide a weak conditional relationship for one regressor, so conditional diagnostics are informative. A homoskedastic F procedure relies on an error structure that clustered errors violate, so cluster-appropriate covariance estimation and inference are needed. For \(H_0:\beta=\beta_0\), Anderson–Rubin tests whether the excluded instruments predict \(Y-D\beta_0\), conditional on included covariates. Rejection of underidentification supports adequate rank according to that test, but the small conditional statistic still signals a potentially weak first stage; neither result establishes instrument validity.