Why the Choice of Residuals Matters
A control-function outcome equation adjusts for endogeneity by adding information about the parts of endogenous regressors that their instruments and exogenous controls do not explain. With one endogenous regressor, this adjustment uses one first-stage residual. With several, the choice is not to select one residual as a general-purpose correction: construct a residual for each endogenous regressor, using the same complete set of first-stage regressors each time.
This distinction is important because a residual is defined relative to a projection. Leaving an excluded instrument out of that projection changes the residual, even if that instrument was included in another first stage or seems most closely associated with a particular endogenous regressor. The residuals must also correspond to the endogenous regressors in the structural equation, not to the outcome or to fitted values.
What Each First Stage Contains
For a linear first stage, write the population linear projection for each endogenous regressor as
Here \(x\) includes the constant and every exogenous control that belongs in the structural equation, \(z\) contains every excluded instrument available for the model, and \(v_j\) is the projection error. In a sample, estimate a separate regression for each \(d_j\) on the same complete list of regressors. If \(\widehat d_{ij}\) is the fitted value for observation \(i\), then its estimated first-stage residual is \(\widehat v_{ij}=d_{ij}-\widehat d_{ij}\).
The coefficients \(\pi_j\) and \(\delta_j\) can differ across first stages. The regressor list does not: each first stage includes the exogenous controls and all excluded instruments. Instruments are called excluded because they do not appear directly in the structural outcome equation; exclusion does not mean they should be left out of the first stages.
This definition uses a linear projection. In a sample, ordinary least squares residuals are orthogonal to the included first-stage regressors: their sample inner products with each included regressor are zero. That orthogonality is a property of the fitted projection; it does not imply that residuals are independent of the instruments, nor does it alone establish the control-function assumptions needed for an outcome equation.
Worked Example: A Single Endogenous Regressor
Suppose a first stage has an intercept, an exogenous control \(w\), and an excluded instrument \(z\). For four hypothetical observations, take \(w=(-1,1,-1,1)'\), \(z=(-1,-1,1,1)'\), and suppose \(d=11+2w+z+3r\), where \(r=(-1,1,1,-1)'\). These three centered vectors are mutually orthogonal, so the stated coefficients are the least-squares projection coefficients.
The fitted values are \( \widehat d=11+2w+z\), which gives \((8,12,10,14)'\). Subtracting fitted values from the observed values gives
The check is direct: \(1'\widehat v=0\), \(w'\widehat v=3-3-3+3=0\), and \(z'\widehat v=3-3+3-3=0\). The residual is therefore the component of \(d\) left after projecting on the intercept, \(w\), and \(z\). A control-function outcome equation for this single endogenous regressor would include this one residual, not the fitted value \(\widehat d\).
Why Include a Residual for Each Endogenous Regressor?
The structural equation might be written as
A linear control-function representation decomposes the structural error \(u\) into a component explained by the first-stage residuals and a remaining error. For example, if the appropriate representation is \(u=v'\rho+\varepsilon\), with \(E[\varepsilon\mid x,z]=0\), then substituting the decomposition into the structural equation gives
This motivates including \(v_1,\ldots,v_K\), one for each endogenous component of \(d\). A single residual generally cannot stand in for a vector of residuals: different endogenous regressors can contain different unexplained components, and the structural error can depend on those components in different ways.
The representation is an assumption about the conditional error structure, not a consequence of running first-stage regressions. In particular, ordinary least-squares residuals being orthogonal to \(x\) and \(z\) does not by itself show that the conditional mean of \(u\) is a linear function of those residuals. The control-function interpretation requires a suitable error decomposition. Instrument relevance and identification are also separate requirements: adding residuals does not repair an instrument set that fails to identify the structural coefficients.
Worked Example: Why an Instrument Must Not Be Omitted
Consider a hypothetical first stage for one endogenous regressor:
At an observation with \(w=1\), \(z_1=0\), \(z_2=2\), and observed \(d=4\), the fitted value using the stated full first stage is
Thus the residual is \(d-\widehat d=4-(-1)=5\). If one mistakenly drops \(z_2\) while calculating the fitted value, the calculation gives \(2+1+3(0)=3\), and the resulting difference is \(4-3=1\). That is not the residual from the specified full projection. The example does not claim that the reduced regression's fitted value is a valid alternative: it shows exactly how omitting an instrument changes the residual being constructed.
In practice, first-stage coefficients are estimated rather than known. The same principle applies: include all excluded instruments and all exogenous controls in each first-stage regression, then subtract that regression's fitted value from its own dependent variable.
Two Endogenous Regressors: A Complete Construction
Suppose \(d_1\) and \(d_2\) are both endogenous. Let \(w\) be an included exogenous control and \(z_1,z_2\) be the excluded instruments. Both first stages include \(w,z_1,z_2\), even if their coefficients differ. The outcome equation can then use both first-stage residuals:
The following hypothetical data are designed so that the first-stage projections can be checked directly. The columns \(W,Z_1,Z_2,R_1,R_2\) are mutually orthogonal, each has mean zero, and each has squared length 8. The endogenous regressors are constructed from them as
| Observation | W | Z1 | Z2 | R1 | R2 | D1 | D2 |
|---|---|---|---|---|---|---|---|
| 1 | 1 | 1 | 1 | 1 | 1 | 18 | 10 |
| 2 | 1 | 1 | -1 | 1 | -1 | 16 | 0 |
| 3 | 1 | -1 | 1 | -1 | 1 | 8 | 8 |
| 4 | 1 | -1 | -1 | -1 | -1 | 6 | -2 |
| 5 | -1 | 1 | 1 | -1 | -1 | 10 | 6 |
| 6 | -1 | 1 | -1 | -1 | 1 | 8 | 8 |
| 7 | -1 | -1 | 1 | 1 | -1 | 8 | 4 |
| 8 | -1 | -1 | -1 | 1 | 1 | 6 | 6 |
Worked Example: Constructing Both Residual Vectors
For \(D_1\), regress on the intercept, \(W,Z_1,Z_2\). Because the four regressors are mutually orthogonal, the intercept coefficient is the mean of \(D_1\), which is 10. The slope on each centered regressor is its inner product with \(D_1\) divided by its squared length. For example,
The same calculation gives coefficients 1 on \(Z_2\), so the fitted values are \(\widehat D_1=10+2W+3Z_1+Z_2\). Subtraction yields \(\widehat v_1=D_1-\widehat D_1=2R_1\), or \((2,2,-2,-2,-2,-2,2,2)'\).
For \(D_2\), the mean is 5. Its projection coefficients are \(-1\) on \(W\), 1 on \(Z_1\), and 2 on \(Z_2\). For instance, \(W'D_2=(-1)(W'W)=-8\), so the coefficient on \(W\) is \(-8/8=-1\). Thus \(\widehat D_2=5-W+Z_1+2Z_2\), and the residual vector is \(\widehat v_2=D_2-\widehat D_2=3R_2\), or \((3,-3,3,-3,-3,3,-3,3)'\).
Both residuals pass the projection check. Since \(R_1\) and \(R_2\) are orthogonal to the intercept, \(W,Z_1,Z_2\), we have \(1'\widehat v_j=W'\widehat v_j=Z_1'\widehat v_j=Z_2'\widehat v_j=0\) for \(j=1,2\). The outcome equation therefore includes both \(2R_1\) and \(3R_2\) as estimated control variables. Including only one would leave out the other endogenous regressor's first-stage residual.
Derivation: Why OLS Residuals Are Orthogonal to the Full First Stage
The sample projection minimizes the sum of squared residuals over its intercept and slope coefficients. Write the first-stage design matrix as \(Q\), containing the constant, exogenous controls, and all excluded instruments, and write the dependent-variable vector as \(d_j\). The least-squares coefficient vector is \(\widehat a_j=(Q'Q)^{-1}Q'd_j\), assuming \(Q\) has full column rank. The fitted values are \(Q\widehat a_j\), so
This establishes orthogonality to every column of the full first-stage design matrix. It also shows why checking only the intercept or one instrument is insufficient: the residual must be orthogonal to each included instrument and control. Conversely, this algebra does not imply orthogonality to variables omitted from \(Q\), nor does it prove the structural error decomposition needed for a control-function interpretation.
Common Mistakes and Practical Checks
- Using only the instrument that seems to match a regressor. Each endogenous regressor is projected on the full excluded-instrument set, not an individually assigned instrument. The coefficients can vary by first stage.
- Using one residual when there are multiple endogenous regressors. Include one residual per endogenous regressor under the vector control-function representation. Do not replace several residuals with an arbitrary average or a residual from just one first stage.
- Confusing residuals with fitted values. A first-stage residual is observed endogenous regressor minus its fitted value. A fitted value is not the residual control.
- Assuming orthogonality proves validity. Sample orthogonality is a mechanical OLS property. A valid control-function interpretation additionally depends on the structural error decomposition and the model's identification conditions.
- Checking arithmetic incompletely. Recompute each projection coefficient from its own cross-product and verify every residual's inner products with the intercept, controls, and all instruments.
Check Your Understanding
Answer each question by referring to the first-stage projection and its role in the control-function equation.
- A model has two endogenous regressors, three included exogenous controls, and four excluded instruments. How many first stages are required, and which regressors appear in each?
- Why is a residual computed from a first stage that omits one excluded instrument generally not the residual defined by the full projection?
- In the worked two-regressor construction, verify that the coefficient on \(Z_2\) in the \(D_2\) first stage is 2 using its cross-product.
- What does the OLS orthogonality condition establish about a sample first-stage residual, and what does it not establish about the structural error?
- Why is the fitted value for an endogenous regressor not interchangeable with its first-stage residual as a control variable?