Two Different Questions About the Response
For a random response \(Y\) and random explanatory variable \(X\), two useful summaries answer different questions. The conditional mean asks for the average response among cases with a specified value of \(X\). The population best linear predictor asks which straight line has the smallest average squared prediction error across the population. The line is a useful summary even when the conditional mean is curved, but it need not give the conditional mean at any particular value of \(X\).
This distinction matters when describing what a regression line says. Its coefficients summarize a population-wide linear approximation; they do not, by themselves, establish that the mean response changes linearly at every value of the predictor. The calculations below use random \(X\), finite second moments for \(X\) and \(Y\), and \(\operatorname{Var}(X)>0\). No normality assumption is needed.
The conditional mean is not restricted to be linear in \(x\). The best linear predictor is restricted to have the form \(a+bX\). The word best here refers specifically to minimizing expected squared error among straight lines, not among all possible prediction functions.
How the Best Linear Predictor Is Determined
For any fixed slope \(b\), the intercept that minimizes the mean squared error centers the prediction error: it is \(a=E(Y)-bE(X)\). Substituting this intercept leaves a one-variable minimization problem in \(b\). The resulting formulas are the population counterparts of the familiar slope and intercept expressions.
These formulas require finite second moments and \(\operatorname{Var}(X)>0\). If \(X\) is constant with probability one, its variance is zero and no unique slope can be determined from this criterion: changing the slope can be offset by changing the intercept without changing fitted values on the population’s support.
The first equation gives \(a=E(Y)-bE(X)\). Use this expression in the second equation:
Because \(\operatorname{Var}(X)>0\), the unique solution is \(b=b^*=\operatorname{Cov}(X,Y)/\operatorname{Var}(X)\), followed by \(a=a^*=E(Y)-b^*E(X)\). This also explains why the slope is determined by covariance relative to the spread of \(X\): the numerator measures how \(X\) and \(Y\) vary together, while the denominator puts that association on the scale of a change in \(X\).
The normal equations say that the error \(Y-a^*-b^*X\) has mean zero and is uncorrelated with \(X\). They do not say that its conditional mean given \(X\) is zero. That stronger statement holds if the conditional mean itself is the fitted line, but not in general.
Worked Examples
Worked Example: A Quadratic Conditional Mean on a Uniform Range
In a hypothetical study, \(X\sim\operatorname{Uniform}(0,3)\) measures a device’s operating level, and its response is \(Y=1+2X^2+\varepsilon\). Assume \(E(\varepsilon\mid X)=0\) and finite second moments. Find the conditional mean and the population best linear predictor, then compare them at \(X=1\).
Method. First obtain the conditional mean using the zero conditional mean of \(\varepsilon\). Then calculate the means, variance, and covariance needed for the population line.
For a uniform variable on \((0,3)\), \(E(X^k)=3^k/(k+1)\). Thus
The conditional mean is \(m(x)=1+2x^2\), since \(E(\varepsilon\mid X)=0\). Taking expectations gives \(E(Y)=1+2E(X^2)=7\). Also, \(E(X\varepsilon)=E[XE(\varepsilon\mid X)]=0\), so \(E(XY)=E(X+2X^3)=1.5+13.5=15\). Therefore,
The slope is \(b^*=4.5/(3/4)=6\), and the intercept is \(a^*=7-6(1.5)=-2\). The population best linear predictor is therefore \(-2+6X\). At \(X=1\), it predicts \(-2+6=4\), while the conditional mean is \(1+2(1)^2=3\). The values differ because the conditional mean is quadratic, not because either calculation is inconsistent.
Check. The line’s average fitted value is \(-2+6E(X)=-2+9=7=E(Y)\). The common mistake this avoids is using an incorrect third moment: \(E(X^3)=6.75\), not \(6\), for this uniform distribution.
Worked Example: Zero Slope Despite a Changing Conditional Mean
For a hypothetical population, let \(X\sim\operatorname{Uniform}(-1,1)\) and \(Y=2+3X^2+\varepsilon\), where \(E(\varepsilon\mid X)=0\) and the second moments are finite. Find the best linear predictor and compare it with the conditional mean at \(X=1/2\).
Method. Use symmetry to calculate the needed moments, then apply the covariance-slope formula. For this uniform distribution, \(E(X)=0\), \(E(X^2)=1/3\), and \(E(X^3)=0\).
The conditional mean is \(m(x)=2+3x^2\). Consequently, \(E(Y)=2+3(1/3)=3\). Since \(E(X\varepsilon)=0\),
The variance of \(X\) is \(1/3\), so \(b^*=0/(1/3)=0\). The intercept is \(a^*=E(Y)-b^*E(X)=3\). Thus the best linear predictor is the constant line \(3\). At \(X=1/2\), the conditional mean is \(2+3(1/4)=2.75\), whereas the line predicts \(3\).
Check. The line’s average fitted value is \(3=E(Y)\), as required. Zero covariance means the best linear slope is zero; it does not mean that the conditional mean is constant. Here the conditional mean changes with \(x^2\), but its variation is symmetric about zero and does not produce a linear association with \(X\).
Worked Example: A Nonlinear Conditional Mean with a Discrete Predictor
In a hypothetical portfolio study, the predictor \(X\) takes values \(0\), \(1\), and \(3\), with probabilities \(1/2\), \(1/4\), and \(1/4\), respectively. Suppose \(Y=X^2+\varepsilon\), where \(E(\varepsilon\mid X)=0\) and the second moments are finite. Calculate the population line and compare it with the conditional mean at \(X=1\).
Method. Calculate expectations by summing over the three possible values of \(X\), then use the covariance-slope formula.
The conditional mean is \(m(x)=x^2\). The moments needed are
Because \(E(\varepsilon\mid X)=0\), \(E(Y)=E(X^2)=5/2\) and \(E(XY)=E(X^3)=7\). Hence
The slope is \(b^*=(9/2)/(3/2)=3\), and the intercept is \(a^*=5/2-3(1)=-1/2\). The best linear predictor is \(-1/2+3X\). At \(X=1\), it predicts \(-1/2+3=5/2\); the conditional mean is \(1^2=1\).
Check. The predictor’s mean is \(-1/2+3E(X)=5/2=E(Y)\). Its conditional-mean comparison at \(X=1\) is different, as expected for a quadratic conditional mean. The common mistake this avoids is treating a population linear summary as though it had to equal the conditional average at each predictor value.
When Do the Two Quantities Agree?
If the conditional mean is linear on the population’s support, so that \(E(Y\mid X)=\alpha+\beta X\) with probability one, then the population best linear predictor is that same line. To see why, take expectations to obtain \(E(Y)=\alpha+\beta E(X)\), and use the conditional expectation property to obtain \(\operatorname{Cov}(X,Y)=\beta\operatorname{Var}(X)\). Since \(\operatorname{Var}(X)>0\), the formulas give \(b^*=\beta\) and \(a^*=\alpha\).
The converse can be stated precisely: the conditional mean and the best linear predictor agree with probability one if and only if the conditional mean is affine in \(X\) with probability one. Values of \(x\) outside the population’s support do not affect this statement. When the conditional mean is nonlinear on the support, the line still minimizes average squared error among lines, but it need not match the conditional mean point by point.
More generally, for any conditional mean \(m(X)=E(Y\mid X)\), the iterated expectation property gives \(E(Y)=E[m(X)]\) and \(\operatorname{Cov}(X,Y)=\operatorname{Cov}(X,m(X))\). Thus the population line summarizes the linear component of the conditional mean as measured by its overall mean and covariance with \(X\). If one writes \(Y=m(X)+\varepsilon\) with \(E(\varepsilon\mid X)=0\), no independence assumption between \(\varepsilon\) and \(X\) is needed for these identities.
Common Mistake and Why the Distinction Matters
A frequent error is to write \(E(Y\mid X=x)=a^*+b^*x\) merely because \(a^*+b^*X\) is the population best linear predictor. That equality is justified only when the conditional mean is linear, up to values that occur with probability zero. In the first worked example, the line predicts \(4\) at \(X=1\), while the conditional mean is \(3\). These numbers answer different questions: the first comes from the best overall straight-line approximation, and the second is the conditional average at that predictor value.
Another trap is to interpret a zero linear slope as proof of no relationship of any kind. The second example has zero covariance and a constant best linear predictor, even though the conditional mean varies with \(X\). When a task asks for a population linear predictor, calculate its coefficients using the mean, variance, and covariance; when it asks for the conditional mean, use the specified conditional distribution or model. Keeping those targets separate is essential for interpreting regression quantities accurately.
Check Your Understanding
For each problem, find the requested population best linear predictor and use the conditional mean, where requested, as a separate quantity.
- Let \(X\sim\operatorname{Uniform}(-2,2)\) and \(Y=4+X^2+\varepsilon\), with \(E(\varepsilon\mid X)=0\). Find the population line and compare its prediction at \(X=1\) with the conditional mean. (Answer: \(E(Y\mid X=x)=4+x^2\); the line is \(16/3\); at \(X=1\), the line predicts \(16/3\) and the conditional mean is \(5\).)
- Let \(X\sim\operatorname{Uniform}(1,3)\) and \(Y=2+X^2+\varepsilon\), with \(E(\varepsilon\mid X)=0\). Find the population line and compare both predictions at \(X=2\). (Answer: \(E(Y\mid X=x)=2+x^2\); the line is \(-5/3+4X\); at \(X=2\), the line predicts \(19/3\) and the conditional mean is \(6\).)
- Let \(X=0,1,3\) with probabilities \(1/2,1/4,1/4\), respectively, and let \(Y=X^2+\varepsilon\), with \(E(\varepsilon\mid X)=0\). Find the population line and compare it with the conditional mean at \(X=1\). (Answer: \(E(Y\mid X=x)=x^2\); the line is \(-1/2+3X\); at \(X=1\), the line predicts \(5/2\) and the conditional mean is \(1\).)
- Suppose \(E(Y\mid X)=\alpha+\beta X\) and \(0<\operatorname{Var}(X)<\infty\). State the population best linear predictor and give a reason its coefficients have this form. (Answer: \(\alpha+\beta X\); its mean is \(E(Y)\), and its covariance with \(X\) is \(\beta\operatorname{Var}(X)\), giving intercept \(\alpha\) and slope \(\beta\).)