Residualized Moments in Partially Linear Regression

Why Residualize Before Estimating the Coefficient?

In a partially linear regression, the relationship between an outcome and covariates can be flexible, while the relationship of interest—a coefficient on a treatment or exposure—remains linear. The challenge is to separate that coefficient from the way covariates predict both treatment and outcome. Subtracting conditional means does this without requiring the covariate effects to be linear.

The resulting moment is also an instance of a Neyman-orthogonal estimating equation: at the true nuisance functions, small first-order errors in estimating those functions do not change the population moment to first order. This property concerns sensitivity of the moment, not whether nuisance estimates are accurate enough for reliable finite-sample inference.

Model, Conditional Means, and Identification

Let \(Y\) be an outcome, \(D\) a scalar treatment or exposure, and \(X\) a vector of observed covariates. Consider the partially linear model

Here \(\theta_0\) is the coefficient of interest, \(g_0\) is an otherwise unrestricted function of the covariates, and \(U\) is the outcome error. The conditional-mean restriction says that, given treatment and covariates, the error has mean zero. In particular, \(E[U\mid X]=0\). Define the treatment and outcome regression functions by

Taking a conditional expectation of the model given \(X\) gives \(\ell_0(X)=\theta_0m_0(X)+g_0(X)\). Define the residualized treatment \(V=D-m_0(X)\). It satisfies \(E[V\mid X]=0\). Subtracting the conditional outcome mean from the model gives

Thus, after removing the parts predicted by \(X\), the remaining outcome is related to the remaining treatment through \(\theta_0\), plus the outcome error. Identification requires residual treatment variation:

Assume the expectations used below exist, including finite second moments for the residuals. The conditional-mean model and this positive-variation condition identify the coefficient through the moment derived next. If treatment is fully determined by \(X\), then \(V=0\) almost surely, the denominator is zero, and residualization cannot identify \(\theta_0\).

Deriving the Residualized Moment

At the true nuisance functions, \(Y-\ell_0(X)=\theta_0V+U\). Multiplying this equation by \(V\) and taking expectations yields

The last term is zero. In fact, the stated conditional-mean restriction implies \(E[U\mid D,X]=0\), so \(E[VU]=E[V\,E[U\mid D,X]]=0\), because \(V\) is determined by \(D,X\). Therefore

Since \(E[V^2]>0\), this moment has a unique solution for \(\theta\) when the nuisance functions are true:

This is a population identity, not an assertion that \(g_0\) is linear or that ordinary regression of \(Y\) on \(D\) alone identifies the coefficient. The conditional means can be nonlinear functions of \(X\); they are removed before relating the residuals.

Why the Moment Is Neyman-Orthogonal

A moment is Neyman-orthogonal at the truth if its expected first derivative with respect to small perturbations of its nuisance functions is zero, holding the target parameter at its true value. Here the nuisance functions are \(\ell_0\) and \(m_0\). To verify the property, perturb them along arbitrary functions \(a(X)\) and \(b(X)\):

Write \(V=D-m_0(X)\). At \(\theta=\theta_0\), the expression inside the second factor of the moment at \(t=0\) is \(Y-\ell_0(X)-\theta_0V=U\). Differentiating the product at \(t=0\) gives

Taking expectations shows why the derivative vanishes. Since \(a(X)\) and \(b(X)\) depend only on \(X\), \(E[b(X)U]=E[b(X)E[U\mid X]]=0\) and \(E[V a(X)]=E[a(X)E[V\mid X]]=0\); the same argument gives \(E[Vb(X)]=0\). Therefore

This is a first-order statement at the true nuisance functions. It does not say the moment is unchanged under arbitrary large nuisance errors. It explains why small nuisance errors enter the expected moment through higher-order terms rather than through a first-order change.

Sample Calculation and Conditions to Check

With observations \((Y_i,D_i,X_i)\), \(i=1,\ldots,n\), nuisance estimates \(\widehat{\ell}\) and \(\widehat{m}\) produce residuals \(\widehat{R}_{Y,i}=Y_i-\widehat{\ell}(X_i)\) and \(\widehat{R}_{D,i}=D_i-\widehat{m}(X_i)\). Solving the sample analogue of the residualized moment gives

The numerator is the sample average product of treatment and outcome residuals; the denominator is the sample average squared treatment residual. This formula requires a nonzero residual-treatment denominator. Its population identification argument additionally relies on the model’s conditional-mean restrictions and positive residual treatment variation. In applications, conditional means are usually estimated rather than known, so the formula alone does not guarantee good statistical performance.

For a finite discrete \(X\), group-specific sample means provide a direct illustration of residualization: subtract each group’s mean treatment and mean outcome. For more complex \(X\), the conditional means may be estimated flexibly. The orthogonality calculation above explains the moment’s local robustness to nuisance-function perturbations, but does not by itself establish the conditions needed for a particular estimator’s inference.

Interpretation and a Common Pitfall

The residualized coefficient is identified by the way treatment varies among observations with the same covariates, as represented by \(D-m_0(X)\). The outcome’s predictable component \(\ell_0(X)\) is removed at the same time. If the model is only a conditional-mean model, \(\theta_0\) describes that model’s partial linear association. A causal interpretation requires additional substantive assumptions and design justification; residualization alone does not establish causality.

• Pitfall: treating residualization as proof of causality. The incorrect reasoning is, “I controlled for \(X\) by subtracting conditional means, so the coefficient is causal.” The correction is that residualization yields the stated moment under its model assumptions. Causal interpretation requires appropriate causal assumptions beyond this algebra.
• Pitfall: assuming the covariate effects must be linear. The incorrect reasoning is, “A partially linear model means \(g_0(X)\) must be a linear function.” The correction is that the target term is linear in \(D\), while \(g_0\), \(m_0\), and \(\ell_0\) may be nonlinear functions of \(X\).
• Pitfall: ignoring the denominator. Even if the numerator can be calculated, there is no identified residualized slope when \(E[(D-m_0(X))^2]=0\). The treatment must vary after conditioning on \(X\).