Neyman-Orthogonal Scores for Causal Parameters

Why Orthogonality Matters

In a partially linear treatment-effect model, the coefficient of interest is accompanied by nuisance functions that describe how the outcome and treatment vary with observed covariates. These functions may be high-dimensional or estimated flexibly. Even if the target coefficient is identified by a population moment, replacing the nuisance functions with estimates can disturb that moment. Neyman orthogonality is a design property of the moment: small first-order errors in the nuisance functions do not produce a first-order change in its expectation at the truth.

This property does not make nuisance estimates irrelevant, nor does it establish a causal interpretation by itself. It changes how nuisance-estimation errors affect the target moment. The distinction is especially useful when the nuisance functions are difficult to estimate accurately, as in high-dimensional settings.

The Partially Linear Model and Its Score

Let \(Y\) be an outcome, \(D\) a treatment, and \(X\) a vector of observed pretreatment covariates. A partially linear conditional-mean model is

where \(\theta_0\) is the scalar coefficient of interest, \(g_0(X)\) is an unknown outcome regression component, and \(m_0(X)\) is the conditional mean of treatment. The residuals satisfy \(E[U\mid D,X]=0\) and \(E[V\mid X]=0\). The first condition implies \(E[U\mid X]=0\) as well. The model allows the covariate effects \(g_0\) and \(m_0\) to be nonlinear.

Define \(l_0(X)=E[Y\mid X]\). Taking conditional expectations in the outcome equation gives \(l_0(X)=\theta_0m_0(X)+g_0(X)\). The score below is written using these two conditional means, so its nuisance argument is \(\eta=(l,m)\).

The score residualizes both treatment and outcome with respect to \(X\). Since \(D-m_0(X)=V\), and \(Y-l_0(X)=\theta_0V+U\), the score at the truth and at \(\theta_0\) is \(V U\). Its expectation is zero because \(E[U\mid D,X]=0\). If \(E[V^2]>0\), the population moment with the true nuisance functions also identifies \(\theta_0\): for any \(\theta\),

For a causal interpretation, further assumptions are needed. For example, consistency, no interference, conditional exchangeability of treatment and potential outcomes given \(X\), and a constant conditional treatment effect compatible with the partially linear mean model can support interpreting \(\theta_0\) as that effect. Without such assumptions, \(\theta_0\) is a parameter of a conditional-mean model, not automatically a causal effect. Positive residual treatment variation, \(E[V^2]>0\), is also needed to identify it.

Definition of Neyman Orthogonality

A nuisance perturbation is a small change to one or more nuisance functions. To define orthogonality precisely, consider a path \(\eta_r=\eta_0+r h\), where \(r\) is a real scalar near zero and \(h\) is a fixed, admissible direction of perturbation. For function-valued nuisances, write \(h=(h_l,h_m)\). The derivative is taken in the population moment, not in a particular sample.

For the score defined above, orthogonality must hold with respect to both \(l\) and \(m\). It is not enough that the score have mean zero at the true nuisance functions: a moment may be unbiased there while still changing to first order when those functions are perturbed.

Deriving and Verifying the Orthogonal Score

Write \(V=D-m_0(X)\). First perturb the outcome regression in direction \(h_l\), holding \(m=m_0\). Differentiating the score with respect to \(l\) gives

The last equality follows from \(E[V\mid X]=0\). Now perturb the treatment regression in direction \(h_m\), holding \(l=l_0\). Differentiating both appearances of \(m\) in the score gives

Here the score residual at the truth is \(U\). The expectation of \(h_m(X)U\) is zero because \(E[U\mid X]=0\); the expectation of \(Vh_m(X)\) is zero because \(E[V\mid X]=0\). The derivative with respect to a joint direction \((h_l,h_m)\) is the sum of these two zero derivatives, so it is also zero.

This calculation also explains why the particular form of the score matters. Residualizing treatment creates a factor with conditional mean zero, which eliminates the first-order effect of perturbing \(l\). The treatment-residual perturbation has two contributions, one from the leading factor and one from the bracketed residual; the model’s conditional-mean restrictions make their expected first-order contribution vanish.

Why the Nuisance Bias Is Second-Order

Orthogonality can be made more concrete by calculating the population moment when the nuisance functions are not exact. Let \(\delta_l(X)=l(X)-l_0(X)\) and \(\delta_m(X)=m(X)-m_0(X)\). At \(\theta_0\), the two factors in the score become

Multiplying these factors and taking expectations yields an exact identity. The terms \(E[VU]\), \(E[V\delta_l(X)]\), \(E[V\delta_m(X)]\), and \(E[\delta_m(X)U]\) are zero by the conditional-mean restrictions. The remaining terms give

Thus the population moment error is a product of nuisance errors, or a squared treatment-regression error, rather than a term linear in either error alone. This is the second-order bias protection supplied by orthogonality. It does not say that the error is always negligible: the functions must still be estimated accurately enough, and sampling variation and other sources of bias remain.

Using the Score in a Sample

With observations \(W_i=(Y_i,D_i,X_i)\), a sample version of the moment replaces the expectation with an average and the nuisance functions with specified estimates \(\hat l\) and \(\hat m\). A value of \(\theta\) that sets this average to zero is an estimator based on the score. When the nuisance estimates are held fixed, solving the equation gives a ratio of residual cross-products, provided the denominator is nonzero. Orthogonality concerns the population moment’s response to nuisance perturbations; it does not by itself guarantee that a finite-sample equation has a stable solution or valid uncertainty estimates.

When Orthogonality Helps—and What It Does Not Do

The practical value of orthogonality is that it reduces sensitivity of the target moment to small nuisance errors. In a high-dimensional problem, nuisance functions may be estimated with regularization or other flexible methods. A non-orthogonal moment can inherit a first-order bias from those estimates. An orthogonal moment instead requires attention to products of errors and other higher-order terms, making reliable target estimation possible under weaker nuisance accuracy than would otherwise be needed.

The protection is conditional on the score and model being appropriate. Orthogonality does not repair confounding from omitted covariates, misspecified causal assumptions, poor treatment overlap, or a failure of the partially linear structure. Nor does it mean that any nuisance estimates will work: errors must be controlled, and the residualized treatment must contain enough variation to identify the coefficient. It is also a population property; finite-sample performance depends on estimation and sampling details.

Common Mistakes

• Equating mean-zero with orthogonality. A score can satisfy \(M(\theta_0,\eta_0)=0\) but still have a nonzero derivative with respect to a nuisance. Orthogonality requires the directional derivative itself to vanish.
• Claiming nuisance estimates no longer matter. Orthogonality removes first-order sensitivity, not all nuisance-induced error. The exact remainder still contains products and squares of nuisance errors.
• Calling the coefficient causal without assumptions. The score defines a parameter of the partially linear conditional-mean model. A causal interpretation additionally requires appropriate treatment-assignment and potential-outcome assumptions.
• Ignoring treatment variation after residualization. If \(E[(D-m_0(X))^2]=0\), the score cannot distinguish treatment effects; the target coefficient is not identified by this moment.