Neyman-Orthogonal Estimating Equations

Why Orthogonality Matters

Many econometric parameters are defined by an estimating equation that depends on both a target parameter and one or more nuisance functions. The target may be a mean or treatment effect; nuisance functions may describe conditional means or treatment probabilities. In practice, those functions are estimated, so the resulting moment equation is evaluated using approximations rather than the unknown population functions.

A key question is how sensitive the population moment is to small errors in those nuisance functions. Neyman orthogonality means that, at the true target and true nuisance functions, the moment has zero first derivative in every admissible nuisance direction. It is a local condition: small nuisance perturbations do not change the population moment to first order. It does not say that nuisance estimates are exact, that a moment remains correct under arbitrary misspecification, or that an estimator is automatically valid.

Definition: Let $W$ denote observed data, $\theta_0$ a target parameter, and $\eta_0$ the true nuisance function (or collection of nuisance functions). An estimating equation $\psi(W;\theta,\eta)$ is Neyman-orthogonal at $(\theta_0,\eta_0)$ if, for every admissible direction $h$, the path $\eta_r=\eta_0+r h$ satisfies $$ \left.\frac{d}{dr} E\!\left[\psi(W;\theta_0,\eta_r)\right]\right|_{r=0}=0. $$ The expectation is under the true data distribution. With multiple nuisance functions, the condition must hold for every admissible joint direction, including directions that perturb one component at a time.

This definition concerns the population moment, not necessarily the score value for each observation. The derivative is a Gateaux derivative: it measures the first-order change in the expected moment along a specified path through the nuisance-function space. Its existence generally requires enough regularity to differentiate under the expectation. In calculations below, directions are assumed to preserve the model's restrictions locally, and the relevant expectations are finite.

Checking the Derivative

For a scalar nuisance function, substitute $\eta_r=\eta_0+r h$ into the population moment, differentiate with respect to $r$, and evaluate at $r=0$. If nuisance functions are conditional expectations, conditioning on covariates often makes the calculation transparent. For a vector of nuisance functions, perturb each component or write a joint direction and verify that the resulting derivative vanishes.

Orthogonality is distinct from identification of $\theta_0$. A useful estimating equation must also satisfy $E[\psi(W;\theta_0,\eta_0)]=0$; to identify a locally unique target, the moment must respond appropriately to changes in $\theta$, commonly through a nonzero derivative with respect to $\theta$. Orthogonality sets the nuisance derivative to zero, not the target-parameter derivative.

Worked Example: A Plug-In Mean and a Residual Correction

Suppose $Y$ is observed for everyone, $X$ is a covariate, and the target is $\theta_0=E[Y]$. Let $m_0(X)=E[Y\mid X]$ be a nuisance regression function. Compare a plug-in moment with a residual-corrected moment.

The plug-in moment is $\psi_{\mathrm{plug}}(W;\theta,m)=m(X)-\theta$. It has mean zero at the truth because $E[m_0(X)]=E[Y]=\theta_0$. But under $m_r=m_0+r h$, $$ \left.\frac{d}{dr}E[m_r(X)-\theta_0]\right|_{r=0} =E[h(X)], $$ which need not be zero. Thus the plug-in moment is not generally Neyman-orthogonal.

For a concrete hypothetical covariate distribution, let $X$ take values $a$ and $b$ with probability $1/2$ each, and consider a direction with $h(a)=2$ and $h(b)=0$, in the units of $Y$. Then $$ E[h(X)]=\tfrac12(2)+\tfrac12(0)=1. $$ A perturbation of size $r$ shifts the population plug-in moment by $r$, to first order; the derivative is not zero.

Now use the residual-corrected moment $\psi_{\mathrm{corr}}(W;\theta,m)=m(X)+Y-m(X)-\theta$. The nuisance terms cancel, so this moment is $Y-\theta$ for every $m$. Consequently, $$ E[\psi_{\mathrm{corr}}(W;\theta_0,m_r)] =E[Y-\theta_0]=0 $$ for every $r$, and its derivative is zero for every direction $h$. This example isolates the basic mechanism: a residual term can offset the first-order effect of plugging in an estimated conditional mean. In this fully observed mean problem the cancellation is exact, but in more complex problems it is typically the population derivative, rather than the whole score, that cancels.

Missing Outcomes: Verifying Two Nuisance Directions

Consider a hypothetical missing-outcome setting. The observed data are $W=(X,R,RY)$, where $R=1$ indicates that $Y$ is observed. Assume missing at random, $R$ is conditionally independent of $Y$ given $X$, and the observation probability is positive: $\pi_0(X)=P(R=1\mid X)>0$. Let $m_0(X)=E[Y\mid X]$, and let the target be $\theta_0=E[Y]$. For this example, assume the displayed expectations exist and the functions used as directions are integrable.

An augmented moment is $\psi(W;\theta,m,\pi)=m(X)+\frac{R}{\pi(X)}\{Y-m(X)\}-\theta$. At the true functions, its expectation is zero: conditionally on $X$, the residual term has expectation $E[R\{Y-m_0(X)\}/\pi_0(X)\mid X]=0$, while $E[m_0(X)]=E[Y]$.

Perturb both nuisance functions along $m_r=m_0+r h_m$ and $\pi_r=\pi_0+r h_\pi$. Differentiating the moment at $r=0$ gives $$ h_m(X)\left(1-\frac{R}{\pi_0(X)}\right) -\frac{R\,h_\pi(X)\{Y-m_0(X)\}}{\pi_0(X)^2}. $$ To check orthogonality, take its conditional expectation given $X$. The first term has conditional expectation $h_m(X)\{1-\pi_0(X)/\pi_0(X)\}=0$. For the second term, missing at random implies $E[R\{Y-m_0(X)\}\mid X]=\pi_0(X)E[Y-m_0(X)\mid X]=0$. Its conditional expectation is therefore also zero. Averaging over $X$ proves that the full derivative vanishes for arbitrary admissible $h_m$ and $h_\pi$.

Worked Example: A Discrete Missing-Outcome Check

Use hypothetical values with $X$ taking two equally likely values, $a$ and $b$. Let $\pi_0(a)=0.5$, $\pi_0(b)=0.8$, $m_0(a)=10$, and $m_0(b)=20$. Suppose the directions are $h_m(a)=2$, $h_m(b)=-1$, $h_\pi(a)=0.1$, and $h_\pi(b)=-0.2$. The outcome units are arbitrary, and the directions are small local perturbations, not estimated values.

For $X=a$, the conditional expectation of the derivative's mean-function term is $2\{1-0.5/0.5\}=0$. For $X=b$, it is $-1\{1-0.8/0.8\}=0$. The propensity-direction term has conditional expectation zero in each stratum because $E[Y-m_0(X)\mid X]=0$ and $R$ is conditionally independent of $Y$ given $X$. Thus the overall derivative is $\tfrac12(0)+\tfrac12(0)=0$.

This calculation checks both nuisance directions, not just one. The numerical probabilities also satisfy positivity, so the denominators used in the score and derivative are defined. The example verifies local orthogonality under the stated assumptions; it does not imply that the score is protected against violations of missing at random or against probabilities equal to zero.

Treatment Effects and a Nonorthogonal Contrast

A second illustration makes the role of augmentation explicit. Let $D$ be a binary treatment, $X$ covariates, $e_0(X)=P(D=1\mid X)$, and $m_{d,0}(X)=E[Y\mid D=d,X]$ for $d\in\{0,1\}$. Assume consistency, conditional exchangeability of treatment and potential outcomes given $X$, and overlap $0<e_0(X)<1$. Under these assumptions, the average treatment effect is $E[m_{1,0}(X)-m_{0,0}(X)]$.

The augmented moment for this target is $\psi_{\mathrm{aug}}=m_1(X)-m_0(X)+\frac{D}{e(X)}\{Y-m_1(X)\}-\frac{1-D}{1-e(X)}\{Y-m_0(X)\}-\theta$. At the true functions, its expectation is zero at the average treatment effect. Conditional on $X$, the residual terms have mean zero.

Here is the derivative check. Perturb $m_1,m_0,e$ by $h_1,h_0,h_e$, respectively. The derivative contributions for the outcome regression directions are $h_1(X)\{1-D/e_0(X)\}$ and $-h_0(X)\{1-(1-D)/(1-e_0(X))\}$. Their conditional expectations are zero because $E[D\mid X]=e_0(X)$ and $E[1-D\mid X]=1-e_0(X)$. The propensity-direction contribution is $-D h_e(X)\{Y-m_{1,0}(X)\}/e_0(X)^2 -(1-D)h_e(X)\{Y-m_{0,0}(X)\}/\{1-e_0(X)\}^2$. Each term has conditional expectation zero: within its treatment group, the outcome residual has conditional mean zero. Therefore the augmented moment is orthogonal to all three nuisance directions.

Worked Example: Why an Inverse-Probability Score Is Not Orthogonal

Consider a hypothetical population with no covariate variation, treatment probability $e_0=0.4$, treated mean outcome $m_{1,0}=12$, and control mean outcome $m_{0,0}=8$. These values are in the outcome's units. The inverse-probability score without outcome-regression augmentation is $\psi_{\mathrm{IPW}}=DY/e-(1-D)Y/(1-e)-\theta$.

Perturb $e_r=0.4+r h_e$, with direction $h_e=0.1$. At the truth, the derivative of the expected score is $$ -h_e\left(\frac{m_{1,0}}{e_0}+ \frac{m_{0,0}}{1-e_0}\right) =-0.1\left(\frac{12}{0.4}+\frac{8}{0.6}\right) =-0.1(30+13.3333) =-4.3333. $$ The derivative is nonzero, so this score is not Neyman-orthogonal with respect to the propensity function in this population. Its sign and magnitude here come from the specified direction and outcome means; they are not a probability or a treatment-effect estimate.

The augmented score cancels this propensity-direction derivative through the residual terms, as shown above. The comparison demonstrates why one must differentiate the particular population moment being used rather than assume that any score involving an estimated propensity is orthogonal.

Interpretation, Scope, and Common Pitfalls

If the population moment is differentiable, a zero nuisance derivative says that its leading, linear change along any allowed nuisance path is absent at the truth. It does not say that changes of larger size have no effect: second- and higher-order changes may remain. Nor does it by itself establish identification, consistency, a valid standard error, or robustness to every form of model error. Those conclusions require additional conditions on the estimator, the data, and the nuisance estimates.

The condition is often valuable because it reduces sensitivity to small nuisance estimation errors. But the exact benefit depends on the full estimation argument, including how the empirical moment behaves and how accurately the nuisance functions are estimated. Those are separate questions from checking the population derivative.

A common mistake is to verify only that the moment has mean zero at the true functions, and then call it orthogonal. These are different checks: $E[\psi(W;\theta_0,\eta_0)]=0$ checks the moment condition, while orthogonality checks the derivative of that expectation with respect to nuisance perturbations. Another mistake is to differentiate the individual score and demand that the derivative be zero for every observed $W$. The definition requires the expected derivative to vanish; conditional-mean-zero terms can satisfy this even when their individual values do not vanish.

Key takeaway: Write down the population moment, perturb each nuisance function along an arbitrary admissible direction, differentiate its expectation at the truth, and show that the result is zero. Check separately that the moment identifies the target and that the assumptions needed for the calculation hold.

Check Your Understanding

For each question, distinguish the population moment condition from the nuisance-derivative condition.

In the definition of Neyman orthogonality, what is varied along a path, and what quantity must have derivative zero?
Why does $E[\psi(W;\theta_0,\eta_0)]=0$ alone not establish orthogonality?
In the missing-outcome example, which conditional-mean property makes the propensity-direction derivative vanish?
For the inverse-probability score in the treatment example, use the displayed population values to explain why the propensity derivative is nonzero.
Does a zero first derivative imply that the population moment is unchanged for all finite nuisance errors? Explain.