Linear Projection Versus Conditional Expectation

Introduction

A regression line can summarize how the mean of an outcome changes with its predictors, but it need not describe the outcome’s conditional mean at every predictor value. The distinction is between a best linear predictor, which is chosen from a restricted class of linear functions, and the conditional expectation, which gives the mean outcome at each covariate value.

This distinction matters because a linear projection is defined by an optimization criterion and moment restrictions. It does not, by itself, assert that the underlying conditional mean is linear. The two objects coincide when the conditional mean belongs to the set of functions allowed in the projection. When it does not, the projection retains particular averages and covariances while smoothing away features such as curvature.

Two Different Objects

Let \(Y\) be a scalar outcome and let \(X\) be a vector of covariates. Assume \(Y\) and the chosen regressors have finite second moments. The conditional expectation \(m(X)=E[Y\mid X]\) is the mean of \(Y\) conditional on the observed covariates. It can be a nonlinear function of \(X\).

Choose a vector of regressors \(Z\) formed from \(X\), including an intercept. For example, with one covariate, \(Z=(1,X)'\), so \(Z'\beta=\beta_0+\beta_1X\). The population best linear predictor is the linear combination of those regressors that minimizes expected squared prediction error.

The projection is well defined and unique when the expected squared errors are finite and \(E[ZZ']\) is nonsingular. Its coefficients then satisfy the population normal equations \(E[Z(Y-Z'\beta^{LP})]=0\). These equations say the projection residual has zero average product with each included regressor. They do not say that the residual has conditional mean zero at every value of \(X\).

For a single scalar regressor \(X\) with an intercept, the formula is particularly familiar. If \(0<\operatorname{Var}(X)<\infty\), then \(\beta_1^{LP}=\operatorname{Cov}(X,Y)/\operatorname{Var}(X)\) and \(\beta_0^{LP}=E[Y]-\beta_1^{LP}E[X]\). These are population quantities: the calculation uses the distribution of \((X,Y)\), not a finite sample.

When Do They Coincide?

Suppose the regressors \(Z\) are functions of \(X\), as when \(Z=(1,X')'\). If the conditional expectation is exactly a linear combination of those regressors, then the projection and conditional expectation are the same, except possibly on a set of covariate values with probability zero. More generally, equality holds precisely when \(m(X)\) lies in the span of the chosen regressors almost surely.

The key justification is a squared-error decomposition. For any candidate coefficient vector \(\beta\), write \(m(X)=E[Y\mid X]\). Since \(Z\) is determined by \(X\), conditioning on \(X\) gives \(E[(Y-Z'\beta)^2\mid X]=E[(Y-m(X))^2\mid X]+(m(X)-Z'\beta)^2\). The cross-product term vanishes because \(E[Y-m(X)\mid X]=0\). Taking expectations yields

The first term does not depend on \(\beta\). Therefore, minimizing prediction error for \(Y\) is equivalent to finding the closest linear combination of \(Z\) to \(m(X)\) in expected squared distance. If \(m(X)\) is already in the regressor span, the second term can be zero, and the projection equals the conditional expectation. Conversely, if the two are equal almost surely, the conditional expectation is that linear combination. This also explains why a projection can summarize a nonlinear conditional mean without reproducing its shape.

What a Projection Preserves When the Mean Is Nonlinear

When the conditional expectation is not in the chosen span, the projection generally differs from it pointwise. It still satisfies the normal equations: its residual has zero mean and zero covariance with each included regressor when an intercept is present. Those are aggregate moment statements, not a claim that the residual averages to zero within every covariate value.

The shape of the projection also depends on which regressors are included. If the chosen regressors include \(1\), \(X\), and \(X^2\), then a conditional mean of the form \(a+bX+cX^2\) is in their span and can be represented exactly. Calling a predictor “linear” refers to being linear in the included coefficients and regressors; it does not require that the covariate enter only as its first power.

Why the Distinction Matters

A population projection is useful as a compact summary of a relationship and as the target described by its own optimization and orthogonality conditions. But those properties alone do not license a pointwise interpretation as the conditional mean. Nor does a projection by itself establish a causal effect: it describes a population relationship under the specified regressors and distribution.

The distinction can be stated in terms of conditional residual means. If \(m(X)=E[Y\mid X]\) and the projection is \(Z'\beta^{LP}\), then \(E[Y-Z'\beta^{LP}\mid X]=m(X)-Z'\beta^{LP}\). This conditional residual mean is zero at every covariate value (almost surely) exactly when the conditional expectation is represented by the chosen regressors. The normal equations require only that the residual’s expected products with the regressors are zero.

Common Mistake

• Wrong reasoning: “The population linear projection has zero residual covariance with \(X\), so the conditional mean of \(Y\) must be linear in \(X\).” Zero covariance is an unconditional moment restriction; nonlinear conditional-mean differences can cancel when averaged over the distribution of \(X\).
• Correction: Conclude that the linear projection is the best predictor within its specified span and satisfies the corresponding normal equations. Conclude that it equals \(E[Y\mid X]\) only when that conditional expectation belongs to the span almost surely.