Projection Calculations in Conditional Models

Why Conditional Projections Matter

In semiparametric models, a projection is a way to separate a function into components that can move along allowed model directions and components that are orthogonal to those directions. Conditional-mean restrictions make this calculation especially structured: conditional expectations remove the component explained by the covariates, while a second residualization can remove variation aligned with the outcome residual.

The calculations below take place in \(L_2(P)\), the space of square-integrable functions of the observation \(O\), under its probability distribution \(P\). Its inner product is \(\langle f,g\rangle=E[fg]\), and its squared norm is \(E[f^2]\). An \(L_2(P)\) projection of a function onto a closed linear subspace is the unique element of that subspace whose difference from the original function is orthogonal to every element of the subspace.

Setup: Let \(X\) be covariates and let \(Y\) be an outcome with conditional mean \(m(X)=E[Y\mid X]\). Define the centered outcome \(U=Y-m(X)\), so \(E[U\mid X]=0\). Assume the functions considered are square-integrable and \(E[U^2\mid X]>0\) almost surely. A conditional score \(s_c\) that preserves both the conditional distribution's total probability and its conditional mean satisfies \(E[s_c\mid X]=0\) and \(E[Us_c\mid X]=0\).

The first restriction says that a conditional density perturbation integrates to zero for each value of \(X\). The second says that, to first order, it does not change the conditional mean. The calculations apply to the \(L_2(P)\) closure of such score directions. If the conditional mean itself is allowed to vary with the parameter of interest, its score restrictions must instead reflect that variation; the fixed-mean calculation here should not be applied unchanged.

Removing the Component Explained by Covariates

The subspace of square-integrable functions of \(X\) is denoted \(L_2(P_X)\). The projection of any \(h\in L_2(P)\) onto this subspace is its conditional expectation \(E[h\mid X]\). The residual \(h-E[h\mid X]\) has conditional mean zero, and therefore is orthogonal to every square-integrable function of \(X\).

E\!\left[\{h-E[h\mid X]\}a(X)\right] =E\!\left[a(X)E\{h-E[h\mid X]\mid X\}\right]=0.

This identity is a basic projection calculation, not merely a convenient way to center data. It establishes that \(E[h\mid X]\) is the closest \(L_2(P)\) function of \(X\) to \(h\). In conditional score problems, the same operation separates covariate-only variation from variation within covariate values.

Worked Example: Projecting onto Functions of a Binary Covariate

Suppose \(X\) takes values 0 and 1 with equal probability. A square-integrable function \(h\) has conditional means \(E[h\mid X=0]=3\) and \(E[h\mid X=1]=1\). Find its projection onto functions of \(X\) and describe the residual.

The projected function is the conditional mean itself:

\widehat h(X)=E[h\mid X]= \begin{cases} 3,&X=0,\\ 1,&X=1. \end{cases}

Thus the residual is \(r=h-\widehat h(X)\), and \(E[r\mid X=0]=0\) and \(E[r\mid X=1]=0\). To verify orthogonality directly, any function \(a(X)\) takes values \(a(0)\) and \(a(1)\), so \(E[ra(X)]=\tfrac12 a(0)E[r\mid X=0]+\tfrac12 a(1)E[r\mid X=1]=0\). The calculation does not require the full conditional distributions of \(h\); the two conditional means are sufficient for this projection.

Projecting onto Conditional-Mean-Preserving Scores

At each value of \(X\), the restrictions on a conditional score make it orthogonal to the two functions \(1\) and \(U\). Because \(E[U\mid X]=0\), these functions are conditionally orthogonal. The projection of \(h\) onto their conditional span is therefore its conditional mean plus a multiple of \(U\). The multiple is obtained by matching the conditional inner product with \(U\).

Theorem: Define \(\mathcal C=\{s\in L_2(P):E[s\mid X]=0,\ E[Us\mid X]=0\}\). If \(E[U^2\mid X]>0\) almost surely, the projection of \(h\in L_2(P)\) onto \(\mathcal C\) is \(h-E[h\mid X]-\frac{E[Uh\mid X]}{E[U^2\mid X]}U\). The removed component, \(E[h\mid X]+\frac{E[Uh\mid X]}{E[U^2\mid X]}U\), lies in the conditional span of \(1\) and \(U\).

The conditional coefficient in the formula is allowed to vary with \(X\). A single constant coefficient would generally fail to make the residual orthogonal to \(U\) separately at every covariate value. The next derivation verifies that the formula meets both restrictions and is the closest such residual.

Remove the conditional mean.
Set \(a(X)=E[h\mid X]\). Then \(h-a(X)\) has conditional mean zero.

Remove the component aligned with \(U\).
Set \(b(X)=E[U h\mid X]/E[U^2\mid X]\). Since \(E[U\mid X]=0\), \(E[U\{h-a(X)-b(X)U\}\mid X]=E[Uh\mid X]-b(X)E[U^2\mid X]=0\).

Verify the projection property.
The residual \(r=h-a(X)-b(X)U\) belongs to \(\mathcal C\). The removed part \(a(X)+b(X)U\) is orthogonal to every \(s\in\mathcal C\), because \(E[s\mid X]=E[Us\mid X]=0\).

Conclude uniqueness.
For any other \(t\in\mathcal C\), \(h-t=(a+bU)+(r-t)\) is a sum of orthogonal components. Hence \(E[(h-t)^2]=E[(a+bU)^2]+E[(r-t)^2]\), which is minimized at \(t=r\).

Worked Example: Residualizing Against the Conditional Mean and Outcome

For each value of \(X\), suppose \(U\) takes the four values \(-2,-1,1,2\) with equal conditional probabilities. The table gives \(h\) at each value of \(U\). Calculate its projection onto conditional-mean-preserving scores.

\(X\)	\(U=-2\)	\(U=-1\)	\(U=1\)	\(U=2\)
0	0	1	4	3
1	1	0	2	1

In either stratum, \(E[U\mid X]=0\) and \(E[U^2\mid X]=(4+1+1+4)/4=2.5\). For \(X=0\), \(E[h\mid X=0]=(0+1+4+3)/4=2\) and \(E[Uh\mid X=0]=(0-1+4+6)/4=2.25\). Therefore \(b(0)=2.25/2.5=0.9\), and the projected score direction is \(r=h-2-0.9U\) in this stratum. Its four values are \(-0.2,-0.1,1.1,-0.8\). Their average is zero, and their products with the corresponding \(U\) values average to zero.

For \(X=1\), \(E[h\mid X=1]=(1+0+2+1)/4=1\) and \(E[Uh\mid X=1]=(-2+0+2+2)/4=0.5\). Thus \(b(1)=0.5/2.5=0.2\), giving \(r=h-1-0.2U\). Its four values are \(0.4,-0.8,0.8,-0.4\); again their average and their average product with \(U\) are both zero. The coefficient and conditional mean vary across strata, so the projection must be computed conditionally.

Orthogonal Directions When the Covariate Distribution Can Change

If the distribution of \(X\) is unrestricted while the conditional mean \(m(X)\) is held fixed, a joint score can include a covariate-distribution component \(a(X)\), with \(E[a(X)]=0\), as well as a conditional component \(s_c\in\mathcal C\). The resulting tangent space is the closure of sums \(a(X)+s_c\). A function orthogonal to every such score must be orthogonal both to all mean-zero functions of \(X\) and to all conditional-mean-preserving scores.

The first requirement removes any component that varies only with \(X\). The second leaves the conditional span of \(1\) and \(U\). Combining them in the mean-zero score space leaves functions of the form \(b(X)U\). Indeed, \(E[b(X)U\mid X]=0\), so this direction is orthogonal to \(a(X)\); and \(E[b(X)U s_c]=E[b(X)E[Us_c\mid X]]=0\). Thus conditional residualization identifies directions orthogonal to nuisance variation without requiring a full basis for the nuisance tangent space.

Worked Example: Extracting an Orthogonal Score Direction

Use the conditional distributions in the preceding example and treat the distribution of \(X\) as unrestricted. Project \(h\) onto the orthogonal complement of the joint nuisance tangent space.

An element of that orthogonal complement has the form \(b(X)U\). To project \(h\) onto this form, choose \(b(X)\) so the residual is orthogonal to \(U\) conditionally: \(E[U\{h-b(X)U\}\mid X]=0\). It follows that \(b(X)=E[Uh\mid X]/E[U^2\mid X]\). The calculations already made give \(b(0)=0.9\) and \(b(1)=0.2\). The projected direction is therefore \(q=0.9U\) when \(X=0\), and \(q=0.2U\) when \(X=1\).

For a direct check, \(E[q\mid X]=b(X)E[U\mid X]=0\), so \(q\) is orthogonal to every covariate-distribution score \(a(X)\). Also \(E[q s_c]=E[b(X)E[Us_c\mid X]]=0\) for every conditional-mean-preserving score \(s_c\). The conditional coefficient is not the same in both strata because the conditional association between \(h\) and \(U\) differs across them.

Common Mistakes and Practical Checks

Conditional projections are easy to miscalculate when unconditional and conditional orthogonality are treated as interchangeable. The requirements \(E[Us\mid X]=0\) and \(E[Us]=0\) are not equivalent: the second allows positive conditional products in some covariate strata to cancel negative products in others. A conditional-mean restriction must hold within covariate values, so the conditional equation is the relevant one.

Using a single coefficient for \(U\). The correct coefficient is \(E[Uh\mid X]/E[U^2\mid X]\), which may depend on \(X\). A constant is justified only under additional conditions.
Forgetting to center the outcome. The score restriction uses \(U=Y-E[Y\mid X]\), not generally \(Y\) itself. Centering ensures \(E[U\mid X]=0\) and makes the conditional basis components orthogonal.
Checking only one restriction. A valid conditional-mean-preserving residual must satisfy both \(E[r\mid X]=0\) and \(E[Ur\mid X]=0\), not just one of them.
Confusing the two projection targets. The projection onto \(\mathcal C\) is the residual \(h-E[h\mid X]-b(X)U\). When the covariate distribution is unrestricted as well, the component orthogonal to the whole nuisance tangent space is instead \(b(X)U\).

Key takeaway: Conditional expectation removes covariate-only variation. Under a fixed conditional mean, removing the additional conditional component aligned with \(U\) gives the projection onto mean-preserving conditional scores. If the covariate distribution is also unrestricted, the remaining orthogonal score directions are \(b(X)U\).

Check Your Understanding

Use the conditional restrictions and projection identities to answer the following.

Why is \(h-E[h\mid X]\) orthogonal to every square-integrable function of \(X\)?
State the two conditional restrictions that define a mean-preserving conditional score when \(U=Y-E[Y\mid X]\).
If \(E[Uh\mid X]=6\) and \(E[U^2\mid X]=3\) in a stratum, what is the coefficient on \(U\) in the conditional projection formula?
Explain why checking only \(E[Us]=0\) is not enough to establish conditional orthogonality to \(U\).
When the distribution of \(X\) is unrestricted but the conditional mean is fixed, what form do directions orthogonal to the nuisance tangent space take?