Why Include a Residual?
When the outcome is binary, replacing treatment with its fitted value in an ordinary linear two-stage procedure does not generally recover the structural effect: the nonlinear outcome link means that this substitution changes the model. Two-stage residual inclusion (2SRI) instead uses the first-stage model to construct a control function, then includes that function in the binary-outcome model. The aim is to account for the dependence between treatment and unobserved determinants of the outcome.
The method is useful only when the control function is justified by a model of that dependence. A first-stage residual is not a general-purpose correction that makes an invalid instrument valid, or a misspecified outcome model correct. The assumptions and interpretation therefore matter as much as the two fitting steps.
Assumptions and the First Stage
A causal interpretation requires more than fitting two regressions. The instrument must be relevant for treatment after conditioning on \(X\); it must be independent of unmeasured causes of the outcome, conditional on \(X\); and it must affect the outcome only through treatment, conditional on \(X\). The last condition is the exclusion restriction. These conditions are accompanied by consistency and no interference: each person's observed outcome corresponds to the treatment received, and one person's treatment does not change another person's outcome.
For 2SRI specifically, the first-stage conditional distribution of \(D\) given \(Z,X\) must be correctly specified, and the included residual must adequately control the dependence between \(D\) and unobserved outcome determinants. The second-stage link and the functional form of the control-function term must also be appropriate. Instrument relevance does not by itself guarantee adequate identification: probabilities close to zero or one, or little instrument-driven variation in treatment, can make estimates unstable. Standard IV assumptions alone do not guarantee that an arbitrary residual-inclusion specification is correct.
A common first stage for binary treatment is a probit model:
where \(\Phi\) is the standard normal cumulative distribution function and the coefficients are estimated from a model containing the instrument and the covariates. Under a latent-index interpretation, \(D=1\) when \(\eta+V>0\), with \(V\) a standard normal disturbance. If \(V\) is the latent treatment disturbance, its conditional expectation given observed \(D,Z,X\) is the probit generalized residual:
Here \(\phi\) is the standard normal density. For treated observations, the latent disturbance is truncated below at \(-\eta\); for untreated observations, it is truncated above at \(-\eta\). The two expressions are the corresponding means of a truncated standard normal variable. This derivation explains why the generalized residual is not simply the observed treatment minus its fitted probability.
Worked Example: Calculating a Probit Generalized Residual
Suppose a hypothetical first-stage probit gives \(\hat\eta=0.5\) for a person. The fitted treatment probability is \(\Phi(0.5)=0.6915\). Use \(\phi(0.5)=0.3521\) and \(\Phi(-0.5)=0.3085\) to calculate the generalized residual for a treated person and for an otherwise identical untreated person.
For \(D=1\), substitute into the treated-observation formula:
For \(D=0\), the denominator is \(1-\Phi(0.5)=0.3085\):
For comparison, the raw residual \(D-\hat P(D=1\mid Z,X)\) is \(1-0.6915=0.3085\) when treated and \(0-0.6915=-0.6915\) when untreated. These are different quantities. Under the latent-probit control-function model, the generalized residual estimates the conditional mean of the latent treatment disturbance; it is the appropriate quantity for that particular derivation.
Another practical specification includes the raw first-stage residual \(D-\hat P(D=1\mid Z,X)\) as the control function. That choice is sometimes used as a 2SRI implementation, but it is not algebraically interchangeable with the probit generalized residual, nor is it automatically valid for every nonlinear first stage. The chosen residual and its functional form need a model-based justification. Do not calculate residuals from a model that omits the instrument or relevant exogenous covariates.
The Second-Stage Model and Predicted Contrasts
With a control function constructed, a binary outcome can be modeled by probit or logit. For example, a logit specification is
A probit specification uses \(G(t)=\Phi(t)\) instead. The coefficient \(\rho\) represents the modeled contribution of the control function; evidence that it differs from zero may be consistent with treatment endogeneity under the specification, but it does not prove that the instrument or the model is valid. In a nonlinear model, \(\beta_D\) is not itself a risk difference. To report a probability-scale contrast, predict each person's outcome risk twice, once setting \(D=1\) and once setting \(D=0\), while holding \(X\) and the estimated control function fixed. Average the paired differences over a stated target sample if an average contrast is desired.
Worked Example: An Average Logit Risk Contrast
Suppose a hypothetical second-stage logit has estimated linear predictor \(-1+0.8D+0.2X+0.5\hat r\). For four individuals, the covariate and control-function values are shown below. Calculate each predicted risk under \(D=0\) and \(D=1\), then average the four individual differences. Use \(G(t)=1/(1+\exp(-t))\).
| Individual | \(X\) | \(\hat r\) | Predictor if \(D=0\) | Predictor if \(D=1\) |
|---|---|---|---|---|
| 1 | 1 | -0.2 | -0.90 | -0.10 |
| 2 | 1 | -0.3 | -0.95 | -0.15 |
| 3 | 0 | 0 | -1.00 | -0.20 |
| 4 | 1 | 0.4 | -0.60 | 0.20 |
For individual 2, for example, the untreated predictor is \(-1+0.2(1)+0.5(-0.3)=-0.95\); the treated predictor adds \(0.8\), giving \(-0.15\). The corresponding risks and differences, calculated before rounding, are:
| Individual | Risk if \(D=0\) | Risk if \(D=1\) | Risk difference |
|---|---|---|---|
| 1 | 0.2891 | 0.4750 | 0.1860 |
| 2 | 0.2789 | 0.4626 | 0.1837 |
| 3 | 0.2689 | 0.4502 | 0.1812 |
| 4 | 0.3543 | 0.5498 | 0.1955 |
For individual 1, for instance, \(G(-0.10)-G(-0.90)\approx0.4750208-0.2890505=0.1859703\), which rounds to \(0.1860\). Averaging the unrounded individual differences gives
For this hypothetical sample, the model-based average predicted risk is about 18.66 percentage points higher under \(D=1\) than under \(D=0\). This is a contrast defined by the fitted structural model and the four covariate-control-function profiles; a causal interpretation requires the assumptions stated above. The calculation is not obtained by exponentiating \(0.8\): that coefficient is on the conditional log-odds scale, not the risk-difference scale.
Worked Example: A Probit Risk Contrast
Consider a hypothetical second-stage probit with linear predictor \(-0.6+0.7D+0.25X+0.4\hat r\). For a person with \(X=0\) and \(\hat r=0.5\), calculate the fitted risk under each treatment and their difference. Use \(\Phi(-0.4)=0.3446\) and \(\Phi(0.3)=0.6179\).
Holding \(X\) and \(\hat r\) fixed, the untreated predictor is \(-0.6+0.25(0)+0.4(0.5)=-0.4\). The treated predictor is \(-0.4+0.7=0.3\). Thus the predicted risks are \(\Phi(-0.4)=0.3446\) and \(\Phi(0.3)=0.6179\), and the risk difference is \(0.6179-0.3446=0.2733\), or 27.33 percentage points. This is an individual model-based contrast, not automatically a population-average causal effect.
Implementation, Inference, and Diagnostics
A defensible implementation makes the modeling choices visible. Fit the treatment model using \(Z\) and \(X\), calculate the specified residual for every observation, and fit the binary-outcome model using observed \(D\), \(X\), and that residual. Then compute predicted contrasts on the probability scale for a clearly defined target population. In particular, when predicting the two treatment states, keep the control-function value fixed at the value estimated for that observation; recomputing it as if treatment had changed would define a different contrast.
Because the second-stage regressor is estimated, ordinary second-stage standard errors that treat \(\hat r\) as known may understate uncertainty. A common approach is a nonparametric bootstrap that resamples independent observational units and repeats both stages, including residual construction and contrast calculation, in every replicate. If data are clustered, resample clusters or use an inference procedure appropriate to the sampling design. A standard error from a one-stage outcome regression that ignores the first stage is not a general solution.
Confirm that the first stage includes the excluded instrument and covariates, and inspect fitted treatment probabilities across instrument values. Weak separation or probabilities concentrated near zero and one can signal weak information or unstable generalized residuals.
Assess whether the first-stage link and covariates are plausible, and whether the outcome link and residual term adequately represent the conditional relationship. A single linear residual term may be too restrictive; adding nonlinear terms should be substantively justified and assessed with sensitivity analyses.
Use a suitable generated-regressor inference method, such as a bootstrap that repeats both stages, and report probability-scale contrasts with uncertainty intervals and units.
The main alternatives answer related but distinct modeling choices. Predictor substitution inserts a fitted treatment value into the outcome equation; in a nonlinear outcome model it generally does not reproduce the structural treatment effect. A joint bivariate probit specifies linked latent treatment and outcome equations and estimates their dependence jointly; it can be more efficient when its joint distributional assumptions are correct, but relies on that parametric structure. 2SRI is modular and permits a probit or logit outcome link, but its validity still depends on a suitable control-function specification and correct inference. Neither approach rescues an instrument that violates exclusion or independence.
Common Mistakes and Limits
- Calling any first-stage residual valid. \(D-\hat P(D=1\mid Z,X)\) and the probit generalized residual are not the same. State which is used and the assumptions supporting it.
- Substituting fitted treatment in stage two. 2SRI includes observed \(D\) and adds a control function; replacing \(D\) with its fitted value is predictor substitution, a different procedure.
- Interpreting a nonlinear coefficient as a risk difference. Compute predicted risks under both treatment states and subtract them on the probability scale.
- Ignoring the generated regressor. The residual is estimated, so uncertainty calculations should account for the first stage.
- Overstating the causal result. A significant residual coefficient, a strong first stage, or a plausible fitted model does not establish instrument validity. Exclusion and independence are substantive assumptions.
Finally, risk contrasts depend on the outcome model, the residual specification, and the population over which predictions are averaged. A 2SRI estimate is not automatically a local average treatment effect, nor does a risk contrast from a nonlinear model have the same interpretation as a conditional log-odds coefficient. Report the target population, model, residual construction, instrument assumptions, and probability-scale estimand together.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- In a 2SRI logit model, should the second stage use observed treatment or its first-stage fitted probability?
- For a probit first stage with \(\hat\eta=0.5\), what is the generalized residual for an untreated observation, given \(\phi(0.5)=0.3521\) and \(\Phi(-0.5)=0.3085\)?
- Why is a logit treatment coefficient not itself a predicted risk difference?
- Why should a bootstrap repeat the first stage as well as the outcome stage?
Solutions. (1) It uses observed \(D\) and includes the first-stage residual or justified generalized residual as an additional control function. (2) The value is \(-0.3521/0.3085\approx-1.141\). (3) The coefficient is on the log-odds scale; a risk difference requires computing \(G(\eta)\) under \(D=1\) and \(D=0\), then subtracting. (4) The residual is estimated, so repeating both stages carries first-stage estimation uncertainty into the distribution of the final estimate.