Why Use Two-Stage Least Squares?
A regression of an outcome on an exposure estimates an association, but that association need not equal the causal effect of the exposure. For example, factors that influence both a biomarker and a health outcome can confound their association. Mendelian randomization uses genetic variation as an instrument to help identify an exposure effect under specific assumptions. With individual-level data, two-stage least squares (2SLS) is a standard way to estimate that effect when the exposure and outcome are continuous.
The procedure has two stages. First, regress the exposure on the genetic instrument and any included covariates, then calculate a fitted exposure for each individual. Second, regress the outcome on that fitted exposure and the same covariates. The fitted exposure isolates the component predicted by the instrument and covariates; it is not a replacement measurement of each person’s actual exposure.
Model and Conditions for Interpretation
The two regressions describe the computation. A causal interpretation additionally requires assumptions about how the instrument relates to exposure and outcome. In a linear structural model, write \(Y_i=\beta X_i+C_i^\mathsf{T}\delta+u_i\), where \(\beta\) is the exposure effect per exposure unit, conditional on \(C_i\). The instrument must be relevant for \(X_i\) after adjustment for \(C_i\); it must be independent of unmeasured causes of the outcome, conditional on those covariates; and it must affect the outcome only through the exposure, conditional on the covariates. The last two conditions are often expressed using the requirement that \(G_i\) is uncorrelated with the structural error \(u_i\), conditional on \(C_i\).
These assumptions are substantive, not guaranteed by having a genetic variant. For example, adjustment may be needed if ancestry is related to both allele frequencies and the outcome. A causal interpretation also presumes that the specified linear effect is an appropriate target; when effects vary across individuals, the interpretation can require additional care. The instrument must have a nonzero association with exposure after covariate adjustment. This tutorial focuses on the estimator, rather than on diagnosing these conditions.
The coefficient’s units are outcome units per exposure unit. If \(X_i\) is measured in mmol/L and \(Y_i\) in risk-score points, a coefficient of 0.6 means an estimated 0.6 risk-score points per 1 mmol/L higher exposure under the model and instrument assumptions. It is not the outcome difference per additional allele.
Worked Example: A Binary Genetic Instrument Without Covariates
Consider hypothetical data on eight individuals. \(G\) indicates carrier status (0 or 1), \(X\) is a continuous exposure in units, and \(Y\) is a continuous outcome in points. The displayed values are individual observations.
| Individual | \(G\) | \(X\) | \(Y\) |
|---|---|---|---|
| 1 | 0 | 1 | 8 |
| 2 | 0 | 2 | 10 |
| 3 | 0 | 3 | 11 |
| 4 | 0 | 2 | 11 |
| 5 | 1 | 3 | 11 |
| 6 | 1 | 4 | 13 |
| 7 | 1 | 5 | 14 |
| 8 | 1 | 4 | 14 |
The mean exposure is 2 for noncarriers and 4 for carriers. With a binary \(G\), the first-stage slope is the difference in those means: \(4-2=2\) exposure units per carrier. The mean outcome is 10 points for noncarriers and 13 points for carriers, so the reduced-form slope is \(13-10=3\) points per carrier.
The fitted exposure from the first stage is \(\hat{X}_i=2+2G_i\): it is 2 for each noncarrier and 4 for each carrier. In the second stage, a one-unit increase in fitted exposure corresponds to a change of \(1/2\) in \(G\). The fitted outcome difference between carrier and noncarrier groups is 3 points across a fitted-exposure difference of 2 units. Thus,
Under the instrumental-variable assumptions, the estimate is a causal effect of 1.5 outcome points per exposure unit. The computation alone does not establish those assumptions.
Fitted Exposures With Covariate Adjustment
When covariates are included, they belong in both stages. The first-stage fitted exposure is obtained from the regression of \(X_i\) on \(G_i\) and \(C_i\); the second-stage regression includes \(\hat{X}_i\) and \(C_i\). Including covariates in only one stage changes the estimator and generally does not implement the intended adjusted 2SLS procedure.
The fitted exposure reflects both the genetic instrument and the covariates. Its purpose is not to predict exposure as accurately as possible for every individual. Rather, 2SLS uses the component of exposure variation explained by the included instruments, while controlling for the included covariates. With one genetic instrument, the coefficient is identified by the instrument’s exposure association after adjustment, not by all observed variation in \(X_i\).
Worked Example: Adjusting for a Covariate
Suppose a hypothetical sample has four individuals, with \(G=(0,1,2,3)\) and a centered covariate \(C=(1,-1,-1,1)\). The exposure and outcome observations are:
| Individual | \(G\) | \(C\) | \(X\) (units) | \(Y\) (points) |
|---|---|---|---|---|
| 1 | 0 | 1 | 7.5 | 18.25 |
| 2 | 1 | -1 | 8.5 | 14.75 |
| 3 | 2 | -1 | 10.5 | 23.25 |
| 4 | 3 | 1 | 13.5 | 23.75 |
To show the adjusted calculation, center \(G\) to obtain \(z=G-\bar{G}=(-1.5,-0.5,0.5,1.5)\), and adjust for the intercept and \(C\). Here \(z\) is orthogonal to \(C\), so residualizing \(G\) against the intercept and \(C\) leaves \(z\). From the table, the exposure values satisfy \(X=10+0.5C+2z\). After removing the intercept and covariate contribution, the residualized exposure is therefore \(x=2z\).
The outcome values satisfy \(Y=20+C+2.5z+e\), where \(e=(1,-3,3,-1)\). This residual component is orthogonal to both the intercept and \(C\), and also to \(z\), since
Thus, after adjustment for the intercept and \(C\), the residualized outcome is \(y=2.5z+e\). The adjusted first-stage slope is \(z^\mathsf{T}x/(z^\mathsf{T}z)=2\), because \(x=2z\). The adjusted reduced-form slope is \(z^\mathsf{T}y/(z^\mathsf{T}z)=2.5\), because \(z^\mathsf{T}e=0\). The resulting 2SLS estimate is \(2.5/2=1.25\) outcome points per exposure unit. This example shows why both regressions must use the same covariate adjustment.
Why 2SLS Equals an Instrumental-Variable Ratio
The equivalence can be shown by removing the covariates from the algebra. Let \(z\), \(x\), and \(y\) denote the vectors of \(G\), \(X\), and \(Y\) after each has been residualized on the intercept and included covariates. With a single instrument, the first-stage fitted vector for the residualized exposure is the projection of \(x\) onto \(z\):
The second-stage coefficient is the slope from regressing residualized \(y\) on \(\hat{x}\). Since \(\hat{x}\) is a scalar multiple of \(z\), its coefficient is
The final expression is the adjusted outcome association with the instrument divided by the adjusted exposure association with the instrument. In regression language, it is the reduced-form coefficient for \(G\) in the outcome regression divided by the first-stage coefficient for \(G\) in the exposure regression, with the same covariates in both. This is the individual-level single-instrument result. The nonzero first-stage association is necessary for the ratio to be defined.
This derivation establishes equality of point estimates, not equality of every inferential calculation. In particular, an ordinary least-squares standard error from the second-stage regression treats the fitted exposures as though they were observed fixed values. They were estimated from the first stage, so that standard error is generally not the correct 2SLS standard error. Use a 2SLS or instrumental-variable procedure that accounts for the estimation and instrument structure; the chosen variance estimator should also reflect the data’s error structure.
Worked Example: Interpreting Fitted Values and the Ratio
In another hypothetical analysis, the first-stage regression estimates \[ \hat{X}_i=4.2+0.8G_i+1.1C_i, \] where \(G_i\) is allele dosage and \(C_i\) is a centered covariate. The estimated coefficient for \(G_i\) in the outcome regression on \(G_i\) and \(C_i\) is 0.48 outcome points per allele. Both regressions use the same covariate.
For an individual with \(G_i=2\) and \(C_i=3\), the fitted exposure is
The fitted value is a model-based exposure prediction for this individual; it is not an estimate of the individual’s causal response. The adjusted first-stage effect is 0.8 exposure units per allele, while the reduced-form effect is 0.48 outcome points per allele. Their ratio gives
Under the instrument and model assumptions, the estimated outcome changes by 0.60 points per exposure unit. The allele-dosage coefficient itself is not the exposure effect; dividing by the first-stage effect converts it to the exposure scale.
Common Mistakes and Practical Meaning
- Wrong: “The second-stage coefficient is the effect of genotype on the outcome.” Correction: It is the fitted-exposure coefficient, expressed in outcome units per exposure unit. The genotype’s outcome association is the reduced-form effect.
- Wrong: “The fitted exposure is each person’s true exposure, so I can treat it as an ordinary measured predictor.” Correction: It is the component predicted by the instrument and covariates. Its role is to construct the IV estimate, not to replace \(X_i\).
- Wrong: “The second-stage OLS standard error is valid because I ran a regression.” Correction: Ordinary second-stage OLS does not generally account for estimation of fitted exposure. Use an IV/2SLS variance calculation.
- Wrong: “A genetic instrument is automatically valid because alleles are inherited.” Correction: Relevance, conditional independence, and exclusion are assumptions to justify in the scientific setting; genetic instruments can still be confounded or affect outcomes through other routes.
2SLS matters because it combines individual-level exposure and outcome data in a single instrumental-variable analysis and makes the source of identifying variation explicit. Its fitted-exposure construction is a computational device; causal interpretation rests on the instrument assumptions, not on the act of fitting two regressions.
Check Your Understanding
Use the model definitions and derivation above to answer the following.
- In the first worked example, what are the first-stage slope, reduced-form slope, and 2SLS estimate, with units?
- Why must the same covariates be included in the first-stage and second-stage regressions?
- State the three central instrument conditions needed for a causal interpretation of the 2SLS coefficient.
- In the projection derivation, why does the single-instrument fitted vector lie along \(z\)?
- Why is an ordinary OLS standard error from the second-stage regression generally not the appropriate 2SLS standard error?