Why Overidentification Tests Need Careful Interpretation
Instrumental-variable analysis can provide more instruments than are needed to identify the endogenous regressors. Those extra instruments impose testable restrictions: if the model is correctly specified and the instruments are valid, they should be jointly consistent with the same structural relationship. Sargan and Hansen tests assess these restrictions.
A complication arises when the instruments are weakly related to the endogenous regressors. The usual reference distribution for an overidentification statistic relies on strong identification. Under weak identification, a conventional p-value may not have its advertised error rate. The test is still about the instrument moment restrictions, not about instrument strength; weak identification instead affects how confidently its statistic can be calibrated and interpreted.
Setup and Hypotheses
Consider a linear instrumental-variables model with \(k\) endogenous regressors and \(q\) excluded instruments, after accounting for included exogenous regressors. Suppose \(q>k\), so there are more excluded instruments than endogenous regressors. The difference \(q-k\) is the number of overidentifying restrictions. Write \(u_i(\beta)\) for the structural error for observation \(i\) at parameter value \(\beta\), and \(z_i\) for the vector of excluded instruments. The population moment restrictions are \(E[z_i u_i(\beta_0)]=0\), where \(\beta_0\) is the true parameter.
This test concerns the joint compatibility of the instruments with the model’s moment restrictions. A rejection does not identify which instrument is invalid, and it does not by itself establish why a restriction fails. For example, a misspecified structural equation can also make the moments incompatible.
Sargan and Hansen Statistics
The Sargan test is the classical homoskedastic version. In its common linear-IV implementation, estimate the model by two-stage least squares, regress the estimated structural residuals on the full instrument set, and let \(R^2\) be the resulting coefficient of determination. With \(n\) observations, the statistic is \(nR^2\). Under valid instruments, homoskedastic errors, a correctly specified model, and strong identification, it has an asymptotic chi-square distribution with \(q-k\) degrees of freedom.
The Hansen \(J\) test uses the sample moment vector and a covariance estimate that can accommodate heteroskedasticity. Define \(\bar g(\hat\beta)=n^{-1}\sum_{i=1}^{n}z_i u_i(\hat\beta)\), and let \(\hat\Omega\) estimate the covariance matrix of \(\sqrt{n}\bar g(\beta_0)\). For the efficient weighting choice, the statistic is \(J=n\bar g(\hat\beta)'\hat\Omega^{-1}\bar g(\hat\beta)\). Under the standard null and strong identification, it too is compared with a chi-square distribution with \(q-k\) degrees of freedom. The robust covariance makes Hansen’s test suitable for heteroskedasticity under its regularity conditions; it does not make the test robust to weak identification.
For both tests, the conventional chi-square calibration requires more than valid instruments. It also relies on a correctly specified model, appropriate moment and covariance assumptions, and enough instrument relevance to identify the parameters strongly in the asymptotic approximation. In finite samples, the chi-square approximation may itself be imperfect even when those conditions hold.
Worked Example: Calculating a Sargan Statistic
Suppose a hypothetical sample has \(n=240\), one endogenous regressor, and three excluded instruments. The Sargan residual regression gives \(R^2=0.018\). For this calculation, assume homoskedastic errors, a correctly specified equation, valid instruments under the null, and strong identification.
There are \(q-k=3-1=2\) overidentifying restrictions. The statistic is
A chi-square random variable with 2 degrees of freedom has upper-tail probability \(P(\chi^2_2\geq 4.32)=e^{-4.32/2}\approx 0.115\). At significance level \(\alpha=0.05\), this conventional test does not reject \(H_0\). Under the stated assumptions, the sample does not provide sufficient evidence against the joint moment restrictions. It does not establish that every instrument is valid, nor does it show that the instruments are strong.
Worked Example: A Hansen Test with a Robust Covariance
Consider a hypothetical sample with \(n=200\), one endogenous regressor, and two excluded instruments. Suppose an efficient GMM fit yields \(\bar g(\hat\beta)=(0.04,-0.02)'\), and the heteroskedasticity-robust covariance estimate for the scaled moments is
The inverse covariance matrix is diagonal with entries \(25\) and \(11.111\). Substitution into the Hansen statistic gives
Here \(q-k=2-1=1\). Under the standard strong-identification conditions, the upper-tail probability from a chi-square distribution with 1 degree of freedom is about \(0.0029\), so the conventional test rejects at the 5% level. That result is evidence against the joint moment restrictions only to the extent that the reference distribution is justified. If the instruments are weak, the conventional p-value need not be correctly calibrated.
Why Strong Identification Enters the Reference Distribution
The degrees of freedom and the weak-identification caveat can be understood from the local behavior of the moment vector. Under strong identification, the moments vary in \(k\) independent directions as the \(k\) endogenous-regressor coefficients change. Estimation uses those directions to fit the moments; the remaining \(q-k\) directions supply the overidentifying restrictions.
More formally, under the null and standard strong-identification regularity conditions, the scaled sample moments at the true parameter are asymptotically normal with covariance \(\Omega\). Let \(G\) be the \(q\)-by-\(k\) derivative matrix of the population moments with respect to the parameters, and suppose it has full column rank. Whiten the moments by multiplying by \(\Omega^{-1/2}\). The estimator adjusts the whitened moment vector along the column space of \(\Omega^{-1/2}G\). The minimized quadratic criterion is therefore the squared length of the component orthogonal to that column space.
Under the null, the whitened limiting moment vector has a standard multivariate normal distribution. Its orthogonal projection onto a fixed \(q-k\) dimensional subspace has \(q-k\) independent standard-normal coordinates. The sum of their squares has a chi-square distribution with \(q-k\) degrees of freedom. This is the justification for the conventional reference distribution.
With weak instruments, the relevant derivatives can be close to zero rather than converging to a fixed, full-rank matrix. The estimator’s adjustment to the moments is then not well approximated by projection onto a fixed \(k\)-dimensional space. The residual quadratic form need not have the conventional chi-square distribution. Its behavior can depend on the strength of the first stage and other features of the data-generating process. Consequently, there is no general rule that weak identification must make the test overreject or underreject: conventional size and power can both be unreliable.
Use first-stage diagnostics and weak-instrument-robust inference to assess instrument strength. These answer questions different from the overidentification test.
Use the Sargan form under homoskedasticity, or a suitable Hansen form with a robust moment covariance when heteroskedasticity is allowed.
Compare with the usual chi-square distribution only when the assumptions supporting its calibration are credible, especially strong identification.
Report what the test says about the moment restrictions, and report instrument-strength evidence separately. Neither conclusion substitutes for the other.
Weak Instruments: What the Test Does and Does Not Say
An overidentification test is not a diagnostic for weak instruments. A first-stage statistic addresses relevance; the Sargan or Hansen statistic addresses joint compatibility of the overidentifying moments with the structural model. A large overidentification statistic does not establish weak relevance. A small statistic does not show strong relevance.
Weak identification also changes how a test result should be read. Under strong identification, rejection at level \(\alpha\) is calibrated to limit the long-run probability of rejecting valid restrictions to approximately \(\alpha\). If identification is weak, that calibration may fail, so a conventional p-value alone may not support a reliable size-\(\alpha\) conclusion. Nonrejection is not proof of validity: even with strong identification, a test can lack power against some violations, and weak identification can further complicate interpretation.
Worked Example: A Small Statistic Does Not Diagnose Strength
In a hypothetical sample with \(n=180\), one endogenous regressor, and three excluded instruments, suppose the residual regression for the Sargan test gives \(R^2=0.006\). A first-stage analysis separately reports a partial \(F\)-statistic of \(3.2\). Assume the homoskedastic conditions needed to compute the Sargan statistic, while keeping in mind that its usual calibration also depends on identification strength.
The Sargan calculation is
The conventional chi-square upper-tail probability is \(e^{-1.08/2}=e^{-0.54}\approx 0.583\), so this test does not reject at the 5% level under the usual reference distribution. The first-stage statistic, however, is a separate indication that relevance may be weak; the value \(1.08\) does not measure instrument strength. Nor does the nonrejection establish that the instruments are valid. With weak identification, the conventional Sargan p-value itself may not be reliably calibrated.
Common Pitfall
Wrong reasoning: “The overidentification test did not reject, so the instruments are valid and strong.” This treats nonrejection as proof, and it incorrectly makes one statistic answer two different questions.
Correction: Nonrejection means only that the test did not find sufficient evidence against the joint moment restrictions at its stated level, subject to the test’s assumptions and calibration. Instrument strength must be assessed separately. If identification may be weak, state that the conventional chi-square p-value may be unreliable rather than treating it as definitive evidence for or against validity.
Conversely, rejection is not a stand-alone diagnosis of weak instruments. It can reflect invalid moments, structural misspecification, or failure of assumptions used for the test. When instruments are weak, the usual rejection rate may also differ from its nominal level. A careful analysis therefore reports the test statistic, degrees of freedom, reference distribution, and strength diagnostics together, and avoids interpreting the overidentification result as a causal verdict on any single instrument.
Check Your Understanding
Answer each question by distinguishing the test’s null from the assumptions behind its calibration.
- A model has two endogenous regressors and five excluded instruments. How many overidentifying restrictions are tested under the standard setup?
- In the Sargan test, what is calculated from the residual-regression \(R^2\), and what error assumption is central to its classical use?
- Why does a heteroskedasticity-robust Hansen covariance estimate not, by itself, make the conventional \(J\)-test valid under weak identification?
- A Hansen test does not reject. What can be concluded, and what cannot be concluded about instrument validity and strength?
- If an overidentification test rejects when instruments may be weak, name two reasons not to interpret the conventional p-value as definitive evidence against the moment restrictions.