Omitted Curvature and Conditional-Mean Misspecification

Introduction

A straight-line regression can summarize how a response tends to change with a predictor, even when the true relationship bends. But a useful summary is not automatically a correct description at every predictor value. If the conditional mean is curved, a straight line generally leaves a pattern in its conditional errors: it tends to overpredict in some regions and underpredict in others.

The distinction matters when interpreting a regression slope. Under a genuinely linear conditional mean, the slope describes the same change in the conditional mean for every one-unit increase in the predictor. Under curvature, the best-fitting straight line still has a slope, but that slope summarizes the relationship over the predictor distribution; it need not equal the local change at any particular value.

Definition: Let $m(x)=E[Y\mid X=x]$ be the true conditional mean, and suppose $E[X^2]$ and $E[Y^2]$ are finite with $\operatorname{Var}(X)>0$. The best linear predictor is $\alpha+\beta X$, where $\alpha$ and $\beta$ minimize $E[(Y-a-bX)^2]$ over constants $a$ and $b$. Conditional-mean misspecification occurs when the true function $m(x)$ is not equal to $\alpha+\beta x$ throughout the predictor values under consideration.

What the Straight Line Leaves Behind

Write the response as $Y=m(X)+U$, where $U=Y-m(X)$. By the definition of conditional expectation, $E[U\mid X]=0$. This separates random variation around the true conditional mean from the systematic difference between that mean and a fitted straight line.

For a particular predictor value $x$, define the conditional discrepancy as $r(x)=m(x)-(\alpha+\beta x)$. The total regression error relative to the line is $Y-(\alpha+\beta X)=r(X)+U$, so its conditional expectation is $r(x)$. If curvature is omitted, this conditional expectation will generally not be zero at every $x$, even though the line is the best linear predictor overall.

E[Y-(\alpha+\beta X)\mid X=x] =m(x)-(\alpha+\beta x)=r(x)

The word “best” here means best according to average squared prediction error over the distribution of $X$. It does not mean that the line agrees with the conditional mean at every value of $X$. A best linear predictor balances discrepancies across the predictor distribution, and its residuals satisfy average orthogonality conditions. Those conditions can hold while $r(x)$ is nonzero at particular values.

Why the Projection Coefficients Have Their Usual Form

The best linear predictor is the population linear approximation. Its coefficients are determined by the joint distribution of $X$ and $Y$, not solely by the shape of $m(x)$. To see the standard formulas, center the predictor at its mean and write $Z=X-E[X]$. For a fixed slope $b$, the intercept minimizing $E[(Y-a-bX)^2]$ is $a=E[Y]-bE[X]$: the average prediction error must be zero. With that intercept, the mean squared error is $E[((Y-E[Y])-bZ)^2]$.

Expanding that square gives a quadratic in $b$. Completing the square shows it is minimized at the covariance-to-variance ratio. This argument requires finite second moments and a nonzero predictor variance; if $X$ is constant, a unique slope cannot be determined.

\begin{aligned} E[((Y-E[Y])-bZ)^2] &=\operatorname{Var}(Y)-2b\operatorname{Cov}(X,Y)+b^2\operatorname{Var}(X)\\ &=\operatorname{Var}(Y)-\frac{\operatorname{Cov}(X,Y)^2}{\operatorname{Var}(X)} +\operatorname{Var}(X)\left(b-\frac{\operatorname{Cov}(X,Y)}{\operatorname{Var}(X)}\right)^2. \end{aligned}

Since $\operatorname{Var}(X)>0$, the final squared term is minimized when $b=\operatorname{Cov}(X,Y)/\operatorname{Var}(X)$. The intercept follows from the zero-average-error condition. Also, because $E[U\mid X]=0$, the covariance of $X$ and $U$ is zero, so the slope can equivalently be calculated using the true conditional mean.

Result: When the required moments are finite and $\operatorname{Var}(X)>0$, the best linear predictor has $$ \beta=\frac{\operatorname{Cov}(X,Y)}{\operatorname{Var}(X)} =\frac{\operatorname{Cov}(X,m(X))}{\operatorname{Var}(X)}, \qquad \alpha=E[Y]-\beta E[X]. $$ Its conditional discrepancy $r(X)$ satisfies $E[r(X)]=0$ and $E[Xr(X)]=0$. These are average conditions; they do not require $r(x)=0$ for every $x$.

Worked Examples

Worked Example: Finding the Conditional Error Pattern

Hypothetical model. Let a workload indicator $X$ take the values $-1,0,1$, each with probability $1/3$. Suppose the conditional mean of a response, measured in thousands of dollars, is $m(x)=10+2x+3x^2$, and $Y=m(X)+U$, where $U$ is independent of $X$ and $U\sim N(0,1)$ in thousands of dollars. Find the best linear predictor and its conditional discrepancies.

Method. Calculate the means, variance, and covariance, then compare the line with the true conditional mean at each possible $X$. This avoids assuming that the best line equals the curved conditional mean.

The predictor moments are $E[X]=(-1+0+1)/3=0$ and $E[X^2]=(1+0+1)/3=2/3$, so $\operatorname{Var}(X)=2/3$. The three conditional means are $m(-1)=10-2+3=11$, $m(0)=10$, and $m(1)=10+2+3=15$. Thus $E[Y]=E[m(X)]=(11+10+15)/3=12$, since $E[U]=0$.

Using $E[XY]=E[Xm(X)]$, because $E[XU]=0$, gives $E[XY]=((-1)(11)+0(10)+1(15))/3=4/3$. Therefore $\operatorname{Cov}(X,Y)=4/3-(0)(12)=4/3$, and $\beta=(4/3)/(2/3)=2$. The intercept is $\alpha=12-2(0)=12$. The best linear predictor is $12+2X$.

At $X=-1,0,1$, respectively, the line predicts $10,12,14$. Subtracting each prediction from the corresponding conditional mean $11,10,15$ gives discrepancies $1,-2,1$ thousand dollars. Their average is $(1-2+1)/3=0$, as required. The line underpredicts at the two endpoints and overpredicts at the middle value; its average balance does not eliminate the conditional pattern.

Worked Example: A Slope Is Not a Constant Conditional-Mean Change

Hypothetical model. Let an exposure index $X$ take values $0,2,4$, each with probability $1/3$. Suppose $m(x)=302/3+2x+x^2$, and $Y=m(X)+U$, where $U$ is independent of $X$ and $U\sim N(0,1)$. Find the best linear predictor, then compare its slope with the changes in $m(x)$. Response values are in thousands of dollars.

Method. Use the covariance formula for the best linear predictor, then calculate the conditional means and discrepancies. This distinguishes the overall linear summary from the local changes in the true mean.

The predictor moments are $E[X]=(0+2+4)/3=2$ and $E[X^2]=(0+4+16)/3=20/3$. Hence $\operatorname{Var}(X)=20/3-2^2=8/3$. Also, $E[X^3]=(0+8+64)/3=24$, so $\operatorname{Cov}(X,X^2)=24-2(20/3)=32/3$.

Since the constant term in $m(X)$ has zero covariance with $X$, $\operatorname{Cov}(X,Y)=2\operatorname{Var}(X)+\operatorname{Cov}(X,X^2)=2(8/3)+32/3=16$. It follows that $\beta=16/(8/3)=6$. The conditional means are $302/3$, $326/3$, and $374/3$, so $E[Y]=(302/3+326/3+374/3)/3=334/3$. Therefore, $\alpha=334/3-6(2)=334/3-12=298/3$, and the predictor is $298/3+6X$.

Check the discrepancies. The line predicts $298/3$, $334/3$, and $370/3$ at $X=0,2,4$. The true conditional means minus these predictions are $4/3,-8/3,4/3$. Their average is zero. Their predictor-weighted average is $[0(4/3)+2(-8/3)+4(4/3)]/3=0$, also confirming the second orthogonality condition.

The increase in the true conditional mean from $x=0$ to $x=2$ is $326/3-302/3=8$, or 4 per unit of $X$. From $x=2$ to $x=4$, it is $374/3-326/3=16$, or 8 per unit. The fitted slope of 6 is neither of these two local rates: it summarizes the relationship over the three equally likely predictor values.

Worked Example: Checking a Linear Approximation Beyond Its Support

Hypothetical model. A service index $X$ takes values $1,2,3,4$, each with probability $1/4$. The true conditional mean of a response, in thousands of dollars, is $m(x)=20+3x+x^2$, and $Y=m(X)+U$, where $U$ is independent of $X$ and $U\sim N(0,1)$. Find the best linear predictor on these four values and compare its prediction at $x=5$ with the specified conditional mean there.

Method. First calculate the population projection using the stated distribution of $X$. Then evaluate both functions at 5. This avoids treating a line fitted over one range as an exact conditional-mean model outside that range.

The moments are $E[X]=(1+2+3+4)/4=5/2$, $E[X^2]=(1+4+9+16)/4=30/4=15/2$, and $\operatorname{Var}(X)=15/2-(5/2)^2=5/4$. Also $E[X^3]=(1+8+27+64)/4=25$, giving $\operatorname{Cov}(X,X^2)=25-(5/2)(15/2)=25-75/4=25/4$.

Thus $\operatorname{Cov}(X,Y)=3(5/4)+25/4=10$, and $\beta=10/(5/4)=8$. The conditional means at $1,2,3,4$ are $24,30,38,48$, so $E[Y]=(24+30+38+48)/4=35$. Therefore $\alpha=35-8(5/2)=15$, and the best linear predictor is $15+8X$.

At $x=5$, the line predicts $15+8(5)=55$, while the specified conditional mean is $m(5)=20+3(5)+5^2=60$. The linear prediction is 5 thousand dollars below that conditional mean. Since 5 is outside the values used to determine the projection, this calculation illustrates extrapolation risk; it does not show that the fitted line was incorrectly calculated for the stated predictor distribution.

Interpretation and Common Exam Traps

When the conditional mean is curved, the regression slope is a distribution-dependent summary. The formula $\operatorname{Cov}(X,m(X))/\operatorname{Var}(X)$ shows why: changing how probability is distributed over predictor values can change the slope, even if the function $m(x)$ itself stays fixed. A one-unit change in $X$ therefore does not generally correspond to a single, constant change in the conditional mean.

A frequent mistake is to read a nonzero conditional discrepancy as proof that the best linear predictor was not calculated correctly. For example, in the second worked example, the discrepancies are not zero at the possible predictor values, but their average and predictor-weighted average are both zero. Another mistake is to claim that the error relative to a line has conditional mean zero merely because the line is a best linear predictor. Under misspecification, that conditional mean is $r(x)$, which need not vanish.

Do not confuse average balance with pointwise correctness. Projection orthogonality concerns averages over the distribution of $X$, not every $x$.
Do not treat a projected slope as a local rate. With curvature, the conditional mean’s rate of change can differ across predictor values.
Do not infer reliable extrapolation from a good in-range summary. The line is chosen to summarize the stated predictor distribution; it need not track the true conditional mean outside that range.

The central question is not simply whether a straight line can be calculated. It is whether the interpretation being attached to that line is justified. A line may be useful as a compact average summary, while its systematic conditional errors reveal that it does not fully describe the response pattern.

Key takeaway: If the true conditional mean is curved, the best linear predictor can still be well defined, but its conditional error is generally systematic at particular predictor values. Interpret its slope as an average linear summary over the predictor distribution, not automatically as a constant conditional-mean effect.

Check Your Understanding

For each problem, identify the best linear predictor or the conditional discrepancy requested. Assume each stated response model has an independent $U\sim N(0,1)$ added to its conditional mean.

$X$ takes $0,1,2$ with equal probabilities, and $m(x)=5+2x+x^2$. Find the best linear predictor. (Answer: $\alpha=14/3,\ \beta=4$.)
$X$ takes $-2,0,2$ with equal probabilities, and $m(x)=30+4x+\tfrac12x^2$. Find the best linear predictor and the discrepancy $m(0)-(\alpha+\beta\cdot0)$. (Answer: $\alpha=94/3,\ \beta=4$; discrepancy at $x=0$ is $-4/3$.)
$X$ takes $1,2,3$ with equal probabilities, and $m(x)=8+3x+2x^2$. Find the best linear predictor. (Answer: $\alpha=4/3,\ \beta=11$.)
$X$ takes $0,1,2,3$ with equal probabilities, and $m(x)=50+2x+x^2$. Find the best linear predictor, then give its prediction at $x=4$ and the true conditional mean there. (Answer: $\alpha=49,\ \beta=5$; line prediction $69$, true conditional mean $74$.)

What You'll Learn