Why a Linear Conditional Mean Determines Covariance
A regression model describes how the conditional mean of a response changes with a predictor. Under the right conditions, that conditional-mean slope also determines the covariance between the two variables. This gives a useful connection between a model stated conditionally and a summary defined from their joint distribution.
The conditions matter. A linear conditional mean alone is not enough if the error term has a conditional mean different from zero. We will derive the identity, identify the moment assumptions that make the calculations valid, and work through examples that check the result directly.
The equivalence follows by taking the conditional expectation of the model given \(X\): the constants and \(X\) are known once we condition on \(X\), so \(E[Y\mid X]=\beta_0+\beta_1X+E[\varepsilon\mid X]\). Thus the zero conditional mean condition makes the stated line the conditional mean of \(Y\), rather than merely one component of the response.
Conditions and the Covariance-Slope Identity
The identity concerns population quantities, not estimates calculated from a sample. A sufficient set of moment conditions is \(E[X^2]<\infty\) and \(E[\varepsilon^2]<\infty\). These ensure the relevant means and covariances exist: in particular, the Cauchy–Schwarz inequality gives \(E[|X\varepsilon|]<\infty\). The model then also gives \(E[Y^2]<\infty\).
Here \(\operatorname{Cov}(X,Y)=E[(X-E[X])(Y-E[Y])]\) and \(\operatorname{Var}(X)=E[(X-E[X])^2]\). The key step is that zero conditional mean makes the error uncorrelated with \(X\). “Uncorrelated” is the conclusion needed here; the condition does not assert that the error and \(X\) are independent.
From \(E[\varepsilon\mid X]=0\), the model gives \(E[Y\mid X]=\beta_0+\beta_1X\). Taking expectations again gives \(E[\varepsilon]=E[E[\varepsilon\mid X]]=0\).
By the conditional-expectation rule and the finite-moment conditions, \(E[X\varepsilon]=E[X E[\varepsilon\mid X]]=0\). Therefore \(\operatorname{Cov}(X,\varepsilon)=E[X\varepsilon]-E[X]E[\varepsilon]=0\).
Using the model and covariance linearity, \(\operatorname{Cov}(X,Y)=\operatorname{Cov}(X,\beta_0+\beta_1X+\varepsilon)=\beta_1\operatorname{Var}(X)+\operatorname{Cov}(X,\varepsilon)=\beta_1\operatorname{Var}(X)\).
This argument uses no assumption of constant conditional variance and no assumption of normal errors. Those assumptions may be useful for other regression tasks, but they are not needed for this identity. The requirements here are a linear conditional mean, a zero conditional mean error, and finite second moments.
Worked Example: Computing Covariance From the Conditional Mean
A hypothetical insurer studies \(X\), the annual number of service calls for a portfolio segment, and \(Y\), its annual administrative cost in thousands of dollars. Suppose \(E[X]=10\), \(\operatorname{Var}(X)=4\), and \(E[Y\mid X]=3+2X\). Assume \(E[X^2]<\infty\), the response has a finite second moment, and define \(\varepsilon=Y-(3+2X)\). Find \(\operatorname{Cov}(X,Y)\), then check it using expectations.
Method. The given conditional mean implies \(E[\varepsilon\mid X]=0\). The finite-moment conditions allow the identity to be applied with \(\beta_1=2\). Thus
For a direct check, first compute \(E[X^2]\):
The mean response is
Next,
Therefore,
The direct calculation agrees with the identity. Common mistake avoided: using the intercept in the covariance calculation as if it contributed a term. A constant has covariance zero with \(X\), so only the slope term and the error covariance can contribute.
Checking the Identity With a Discrete Predictor
A discrete distribution makes each expectation visible. The error need not be constant across predictor values, and it need not be normally distributed. It must, however, have conditional mean zero and finite second moment for the result to apply.
Worked Example: A Discrete Portfolio Predictor
In a hypothetical study, \(X\) is the number of inspections assigned to a small facility, with probabilities \(P(X=0)=0.2\), \(P(X=2)=0.5\), and \(P(X=5)=0.3\). Let \(Y=4+1.5X+\varepsilon\), where \(\varepsilon\) is independent of \(X\) and takes values \(-2\) and \(2\) with equal probability. Find \(\operatorname{Cov}(X,Y)\) using the identity and verify it directly.
Method. The two possible error values have mean zero and finite second moment, and independence gives \(E[\varepsilon\mid X]=0\). First calculate the moments of \(X\):
so
The covariance-slope identity gives
For the direct check, \(E[Y]=4+1.5E[X]+E[\varepsilon]=4+1.5(2.5)+0=7.75\). Also \(E[X\varepsilon]=0\), so
It follows that
The two methods agree. Common mistake avoided: treating the variance of \(\varepsilon\) as an additional term in \(\operatorname{Cov}(X,Y)\). Error variance affects response variability, but the covariance contribution is \(\operatorname{Cov}(X,\varepsilon)\), which is zero under the stated condition.
Why Zero Conditional Mean Cannot Be Dropped
If \(E[\varepsilon\mid X]\ne 0\), the same covariance expansion still holds when the moments exist, but it includes an additional term:
There is then no general reason for the error covariance to vanish. A linear conditional mean is still possible, but its slope is the slope of the actual conditional mean, not necessarily the coefficient \(\beta_1\) of a model component whose error has a nonzero conditional mean.
Worked Example: A Nonzero Conditional Error Mean
A hypothetical maintenance study uses \(X\), the number of machines at a site, and \(Y\), annual maintenance cost in thousands of dollars. Let \(X\) be 0 or 4 with probability \(1/2\) each. Suppose \(Y=5+2X+\varepsilon\), where \(\varepsilon=X-2+\eta\), and \(\eta\) is independent of \(X\), taking values \(-1\) and \(1\) with equal probability. Compute the covariance and identify why applying the identity with slope 2 would fail.
Method. The predictor moments are \(E[X]=(0+4)/2=2\) and \(E[X^2]=(0+16)/2=8\), so
Because \(E[\eta\mid X]=0\), the conditional error mean is
which is not zero for both possible values of \(X\). The theorem therefore does not apply to the coefficient 2 in this representation. Instead, rewrite the response:
Since \(\eta\) is independent of \(X\) and has mean zero, direct covariance expansion gives
Equivalently, the extra error term is \(\operatorname{Cov}(X,\varepsilon)=\operatorname{Cov}(X,X-2+\eta)=4\). Thus the original expansion gives \(2(4)+4=12\), not \(2(4)=8\). Common mistake avoided: using the identity solely because a response has been written as a line plus a residual. The residual must also have zero conditional mean given \(X\).
Common Exam Traps and Interpretation
- Confusing unconditional and conditional means. The condition is \(E[\varepsilon\mid X]=0\), not merely \(E[\varepsilon]=0\). An error can average to zero overall while still varying systematically with \(X\).
- Adding an intercept contribution. Since \(\operatorname{Cov}(X,\beta_0)=0\), the intercept does not appear in the identity.
- Assuming constant error variance is necessary. It is not. The identity needs the zero conditional mean and finite moments, not a constant value of \(\operatorname{Var}(\varepsilon\mid X)\).
- Ignoring whether moments exist. Covariance calculations require integrability. The stated finite second moments are a convenient sufficient condition for the products and variances in the derivation to be well-defined.
- Claiming independence from zero conditional mean. Zero conditional mean ensures the required zero covariance, but does not by itself imply that \(\varepsilon\) and \(X\) are independent.
Check Your Understanding
For each problem, check the moment conditions and the conditional error mean before applying the covariance-slope identity.
- A hypothetical claims team records \(X\), the monthly number of files reviewed, and \(Y\), review expense in hundreds of dollars. Suppose \(\operatorname{Var}(X)=9\), \(E[Y\mid X]=6+1.2X\), and the relevant second moments are finite. Find \(\operatorname{Cov}(X,Y)\). (Answer: 10.8)
- A predictor \(X\) takes values 1, 3, and 6 with probabilities 0.2, 0.5, and 0.3. A response satisfies \(Y=8-0.4X+\varepsilon\), where \(E[\varepsilon\mid X]=0\) and the second moments are finite. Calculate \(\operatorname{Var}(X)\), then find \(\operatorname{Cov}(X,Y)\). (Answer: \(\operatorname{Var}(X)=3.25\); \(\operatorname{Cov}(X,Y)=-1.3\))
- In a hypothetical equipment study, \(X\) takes values 0, 2, and 4 with probabilities 0.25, 0.5, and 0.25. Let \(Y=7+3X+\varepsilon\), with \(E[\varepsilon\mid X]=0\). Calculate \(E[X]\), \(\operatorname{Var}(X)\), and \(\operatorname{Cov}(X,Y)\). (Answer: \(E[X]=2\); \(\operatorname{Var}(X)=2\); \(\operatorname{Cov}(X,Y)=6\))
- A model is \(Y=10+1.1X+\varepsilon\), with \(\operatorname{Var}(X)=16\), \(E[X]=6\), and \(E[\varepsilon\mid X]=0.5(X-6)\). Assuming finite second moments, calculate \(\operatorname{Cov}(X,Y)\), including the error contribution. (Answer: 25.6)
- A response is written \(Y=4-2X+\varepsilon\), where \(\operatorname{Var}(X)=2.5\), \(E[X^2]<\infty\), \(E[\varepsilon^2]<\infty\), and \(E[\varepsilon\mid X]=0\). Find the covariance and state whether normality or constant conditional error variance is needed. (Answer: \(\operatorname{Cov}(X,Y)=-5\); neither normality nor constant conditional error variance is required)