Workshop: Testing Endogeneity In A Linear Structural Equation

Why Test for Endogeneity in a Linear Structural Equation?

A linear regression slope has a structural interpretation only under assumptions that make the regressor appropriately unrelated to the structural error. When an explanatory variable is endogenous, ordinary least squares generally does not identify that slope. A residual-inclusion test gives a direct, model-based way to assess endogeneity using excluded instruments: estimate the endogenous regressor's first stage, include its residual in the outcome equation, and test the residual's coefficient.

The test is informative only in the setting that supports it. In particular, the instrument must be relevant, the first-stage specification must be adequate, and the control-function model for the outcome error must be appropriate. A robust standard error can address heteroskedasticity in the specified estimating equations; it cannot repair a weak or invalid instrument, or a misspecified model.

Model and Test

Definition: Let $y_i$ be an outcome, $x_i$ an endogenous regressor, $W_i$ a vector of included exogenous controls, and $Z_i$ a vector of excluded instruments. A linear first stage and residual-inclusion outcome equation are $$ x_i = W_i'\kappa + Z_i'\pi + v_i, $$ $$ y_i = W_i'\delta + \beta x_i + \gamma v_i + \varepsilon_i. $$ The first-stage residual $v_i$ is unobserved and is estimated by $\hat v_i=x_i-W_i'\hat\kappa-Z_i'\hat\pi$. The endogeneity test is $H_0:\gamma=0$ against $H_a:\gamma\ne 0$. Under the maintained control-function assumptions, a nonzero $\gamma$ indicates that the component of $x_i$ not explained by $W_i$ and $Z_i$ is related to the structural outcome error.

The relevant assumptions include a correctly specified first-stage conditional mean, adequate instrument relevance after controlling for $W_i$, and instrument exogeneity and exclusion: the excluded instruments affect the outcome through $x_i$, not directly, and are uncorrelated with the structural disturbance. The control-function outcome equation must also be appropriate; for example, its conditional mean must be adequately represented by a linear term in $v_i$. The residual test is not a general test that establishes all of these assumptions.

The first-stage residual is estimated, not observed. Consequently, uncertainty for $\hat\gamma$ must account for its construction. A suitable robust sandwich covariance can be formed from the stacked first- and second-stage estimating equations; a bootstrap alternative must resample observations and re-estimate both stages in each resample. Treating $\hat v_i$ as fixed and applying an ordinary second-stage standard error is not automatically valid.

\widehat{\operatorname{Var}}(\hat\theta) = \frac{1}{n}\hat A^{-1}\hat B(\hat A^{-1})', \qquad \hat\theta=(\hat\kappa',\hat\pi',\hat\delta',\hat\beta,\hat\gamma)'.

Here $\hat A$ is the sample derivative of the stacked estimating equations with respect to their parameters, and $\hat B$ is the sample covariance “meat” formed from their observation-level contributions. The sandwich form permits heteroskedasticity when the estimating equations and assumptions are correctly specified. The test statistic is $\hat\gamma/SE(\hat\gamma)$; with a large sample, it is compared with the standard normal distribution. A Wald confidence interval is $\hat\gamma\pm1.96SE(\hat\gamma)$ for an approximate 95% interval.

Why the Residual Coefficient Is Relevant

The first-stage fitted value is a linear combination of the included controls and excluded instruments. Thus, the residual $v_i$ represents the part of $x_i$ left after those predictors are accounted for. If the structural error's conditional mean has a linear control-function representation, $E[u_i\mid W_i,Z_i,v_i]=\gamma v_i$, then $\gamma=0$ means that this component does not predict the structural error under that representation.

The connection to endogeneity can also be seen from a covariance argument. Suppose $E[u_i\mid W_i,Z_i,v_i]=\gamma v_i$, and the first-stage fitted component is orthogonal to $v_i$, as it is in population under the stated conditional-mean projection. Then $$ \operatorname{Cov}(x_i,u_i\mid W_i) = \operatorname{Cov}(W_i'\kappa+Z_i'\pi+v_i,u_i\mid W_i) = \gamma\operatorname{Var}(v_i\mid W_i), $$ where instrument exogeneity removes the covariance contribution from $Z_i$. If the residual has positive variance, $\gamma=0$ implies no such covariance in this model. This is a model-based justification for the test, not proof that the model or instruments are valid.

Worked Example: From First Stage to a Robust Endogeneity Test

Consider a hypothetical sample of 240 workers. The outcome $y_i$ is hourly wage in dollars, $x_i$ is years of job training, $W_i$ includes an intercept and measured background controls, and $Z_i$ is an excluded lottery offer for a training place. Assume the lottery is relevant, is unrelated to unobserved wage determinants conditional on the controls, and affects wages only through training. These are substantive assumptions for this hypothetical example, not facts established by the regression.

The estimated first stage gives a lottery-offer coefficient of $0.80$ training years with standard error $0.17$. Its test statistic is $$ t_{\text{first stage}}=\frac{0.80}{0.17}=4.706, \qquad t_{\text{first stage}}^2=22.15. $$ With one excluded instrument, the squared statistic is the corresponding one-restriction Wald statistic. This suggests that the offer predicts training conditional on the controls. It does not, by itself, establish instrument exogeneity or exclusion.

For one worker, suppose observed training is $1.4$ years and the fitted first-stage value is $1.1$ years. That worker's estimated control is $$ \hat v_i = 1.4-1.1=0.30\text{ training years}. $$ The same subtraction is performed for every worker. The outcome equation is then estimated with the controls, training, and all 240 estimated residuals.

Suppose the residual-inclusion estimate is $\hat\gamma=0.065$ dollars per hour per residual training year, with robust standard error $0.029$. This reported uncertainty uses a sandwich calculation that accounts for estimating the first stage. The large-sample test is $$ z=\frac{0.065}{0.029}=2.241. $$ The two-sided normal-reference p-value is approximately $0.025$. At $\alpha=0.05$, reject $H_0:\gamma=0$. Under the maintained assumptions, the result is evidence that training is endogenous in this structural equation; it is not evidence that endogeneity is present with probability 97.5%.

The approximate 95% confidence interval for the residual coefficient is $$ 0.065\pm1.96(0.029) = 0.065\pm0.05684 = [0.00816,\ 0.1218] $$ dollars per hour per residual training year. It excludes zero, in agreement with the test. This conclusion concerns the residual coefficient under the specified control-function model; it does not validate the lottery assumptions.

Reading the Test Carefully

A significant residual coefficient is evidence against the exogeneity restriction within the maintained model, provided the instrument and control-function assumptions hold. It does not identify which assumption would fail if the result is unexpected. A non-significant coefficient is also not proof that $x_i$ is exogenous: low precision, weak first-stage information, or a poor control-function approximation can limit the test's ability to detect endogeneity.

The test is distinct from an instrument-validity test. An endogeneity test asks whether the residual term contributes to the outcome equation under its assumptions. An overidentification test asks whether the data are inconsistent with the joint restrictions imposed by a set of overidentifying instruments. These questions are not interchangeable. In particular, an overidentification test is available only when there are more excluded instruments than endogenous regressors; it is unavailable in an exactly identified model.

Worked Example: A Non-Significant Residual Test

In a second hypothetical wage analysis, suppose the estimated residual coefficient is $\hat\gamma=0.018$ dollars per hour per residual training year, with a robust standard error of $0.024$ from a procedure that re-estimates the first stage. Then $$ z=\frac{0.018}{0.024}=0.750, $$ with a two-sided normal-reference p-value of approximately $0.453$. At the 5% level, do not reject $H_0:\gamma=0$.

The approximate 95% interval is $$ 0.018\pm1.96(0.024) = 0.018\pm0.04704 = [-0.0290,\ 0.0650] $$ dollars per hour per residual training year. The data do not provide strong evidence against a zero residual coefficient, but the interval includes effects in either direction. The defensible conclusion is “no statistically significant evidence of endogeneity from this test,” not “training is proven exogenous.”

Worked Example: What an Overidentification Test Can and Cannot Say

Suppose a hypothetical study has one endogenous regressor and two excluded instruments. There is one overidentifying restriction. A robust overidentification test reports $J=1.84$, compared with a chi-square reference distribution with $df=1$. The p-value is approximately $0.175$. At $\alpha=0.05$, the test does not reject the joint restrictions.

This is not evidence that both instruments are certainly valid. The test can have limited power, and a rejection would indicate that at least part of the joint instrument restrictions is inconsistent with the data and model; it would not identify which instrument is at fault. The test can also be affected by outcome-model misspecification. If the study instead had one endogenous regressor and only one excluded instrument, it would be exactly identified and there would be no overidentifying restriction to test. The residual-inclusion endogeneity conclusion must therefore be assessed separately from this validity diagnostic.

Common Pitfall: Treating the Second-Stage Residual as Ordinary Data

A common mistake is to generate $\hat v_i$, add it to the outcome regression, and report the usual second-stage standard error as if the added regressor had been observed from the start. That reasoning ignores that $\hat v_i$ depends on estimated first-stage coefficients. The correction is to use inference that accounts for first-stage estimation, such as a valid stacked-equation sandwich or a bootstrap that re-estimates both stages. Merely choosing a heteroskedasticity-robust option for the second regression does not automatically make the generated-regressor inference valid.

A second mistake is to treat a significant residual coefficient as proof that the instrument is valid. The test relies on instrument relevance, exogeneity, exclusion, and a suitable control-function specification. Robust uncertainty protects against a particular variance misspecification; it cannot establish those identifying assumptions.

Key takeaway: In a linear residual-inclusion test, the estimated first-stage residual is added to the outcome equation and its coefficient is tested against zero using uncertainty that accounts for first-stage estimation. Interpret the result conditional on the instrument and control-function assumptions, and do not confuse it with an overidentification test.

Check Your Understanding

Answer each question using the model and distinctions developed above.

What does the coefficient on the estimated first-stage residual represent in the linear control-function outcome equation?
A residual coefficient is $0.040$ with robust standard error $0.020$. Calculate the large-sample test statistic and state the two-sided conclusion at the 5% level.
Why is a usual second-stage standard error not automatically valid when the first-stage residual is estimated?
What assumptions support interpreting a significant residual coefficient as evidence of endogeneity?
Why can a non-rejection of an overidentification test not establish that every instrument is valid?