Residualization and the Frisch–Waugh–Lovell Result

Why Residualize Both Sides?

A variant’s conditional regression coefficient describes its association with phenotype while holding specified covariates and other included variants fixed. The Frisch–Waugh–Lovell result gives an equivalent way to calculate that coefficient: first remove from the phenotype and target genotype everything linearly explained by the same nuisance variables, then regress one set of residuals on the other. This is useful for understanding what “conditional” means algebraically, and for seeing precisely which part of a genotype contributes to its coefficient.

The operation must be performed on both the phenotype and the target genotype. Residualizing only the phenotype, or only the genotype, generally does not recover the conditional coefficient. The residualization must also use the same observations and the same nuisance variables on both sides.

Model and Residualization

Suppose there are \(n\) observations. Let \(y\) be the \(n\)-vector of phenotype values, and let \(g_j\) be the \(n\)-vector of dosages for the target variant \(j\). Write \(C\) for the nuisance design matrix: its columns contain an intercept, any included covariates, and the genotype columns for the other variants in the conditional model. The model is

Here \(\beta_j\) is the coefficient of interest, and \(\gamma\) contains the nuisance coefficients. Assume the observations and columns are arranged consistently throughout, \(C\) has full column rank, and \(g_j\) is not an exact linear combination of the columns of \(C\). These conditions ensure that the model coefficients are identifiable. The algebraic result below does not require normally distributed errors; assumptions for conventional standard errors and tests are discussed separately.

The residual-maker is symmetric and idempotent: \(M_C^\mathsf{T}=M_C\) and \(M_C^2=M_C\). Define the residualized phenotype and genotype as \(y^*=M_Cy\) and \(g_j^*=M_Cg_j\). They contain the parts of \(y\) and \(g_j\) that cannot be linearly explained by \(C\). In particular, \(g_j^*\) is the variation in the target genotype that remains after accounting for the other included variables.

The Frisch–Waugh–Lovell Result

Thus \(\hat{\beta}_j\) is the slope from an ordinary least-squares regression of residualized phenotype on residualized genotype through the origin. When \(C\) includes an intercept, both residual vectors have mean zero, so the through-origin formulation is natural. This coefficient is a regression slope, not a correlation: it retains the units of phenotype per genotype-dosage unit.

A short derivation makes clear why the same \(C\) must be used for both variables. For a fixed candidate value \(\beta\), choose \(\gamma\) to minimize the residual sum of squares. The least-squares fit of \(y-g_j\beta\) on \(C\) leaves residual vector \(M_C(y-g_j\beta)\). Therefore the full model’s residual sum of squares, after minimizing over \(\gamma\), is

The second equality uses symmetry and idempotence of \(M_C\). Differentiating this quadratic with respect to \(\beta\) and setting the derivative to zero gives

Finally, symmetry and idempotence also give \(g_j^\mathsf{T}M_Cy=(M_Cg_j)^\mathsf{T}(M_Cy)\) and \(g_j^\mathsf{T}M_Cg_j=(M_Cg_j)^\mathsf{T}(M_Cg_j)\). Substituting \(g_j^*\) and \(y^*\) yields the residual-regression formula. The denominator is positive precisely when \(g_j\) has variation remaining after projection on \(C\).

What the Residual Slope Does—and Does Not—Say

The residual slope measures how phenotype residuals change with genotype residuals. It does not measure the association between the original phenotype and the original genotype, and it does not by itself establish a causal effect. In a genetic association model, its interpretation is conditional on the covariates and other included variants, the chosen phenotype scale, and the linear model specification.

A residual correlation is a different quantity. If \(r\) denotes the correlation between \(y^*\) and \(g_j^*\), then the slope can be written as \(r\,s_{y^*}/s_{g_j^*}\), where \(s_{y^*}\) and \(s_{g_j^*}\) are the sample standard deviations of the residuals. Correlation is unitless; the regression slope has phenotype units per dosage unit. A high residual correlation need not imply a large slope if the residual genotype variation is small.

Standard Errors and the Full Model

The coefficient identity is an algebraic property of least squares. It does not, by itself, determine a standard error or a p-value. For the conventional ordinary least-squares standard error, assume errors are independent with mean zero and common variance \(\sigma^2\). If errors are also normally distributed, the usual t reference distribution is exact under the null. Let \(q\) be the number of linearly independent columns of \(C\). The full model has \(q+1\) fitted coefficients, and its residual degrees of freedom are \(n-q-1\). With residual sum of squares \(SSE\) from the full model, the conventional estimated variance is \(\hat{\sigma}^2=SSE/(n-q-1)\), and

The residual-only regression reproduces the coefficient and the full model’s residuals, so it reproduces \(SSE\) as well. But its degrees of freedom must still account for the nuisance coefficients already estimated. Treating it as a standalone regression with only one predictor and using \(n-2\) degrees of freedom would generally give the wrong variance estimate. If the error structure is not independent and homoskedastic, the conventional formula may not apply; an appropriate variance estimator must reflect that structure.

Common Mistakes and Practical Checks

• Residualizing only one variable. Regressing \(y^*\) on the raw genotype, or the raw phenotype on \(g_j^*\), does not implement the theorem. Residualize both against the same \(C\).
• Changing the nuisance set. If the target genotype is residualized against one set of covariates but the phenotype against another, the displayed coefficient identity no longer follows.
• Using different observations. Missingness can change projections and coefficients. The full regression and both residualizations must use the same rows.
• Confusing slope with correlation. The slope has phenotype-per-dosage units; the correlation is unitless. Report and interpret the quantity actually calculated.
• Ignoring rank deficiency. If \(g_j\) is fully explained by \(C\), then \(g_j^*=0\), the denominator is zero, and its separate conditional coefficient cannot be identified from these data.
• Reading conditional association as causation. The theorem is a statement about equivalent least-squares calculations. It does not establish that changing genotype would cause the fitted phenotype change.