Why a Conditional Mean Implies a Correlation
A linear regression model describes how the mean of a response changes with a predictor. Under an additional condition on the errors, that mean relationship also determines the direction of the population association between the two variables. The resulting correlation depends not only on the slope, but also on how variable the predictor and response are.
This connection is useful for interpreting a regression model and checking whether proposed summary values are compatible with it. The result is a population relationship: it does not say that a sample correlation must equal a formula using a fitted sample slope and sample standard deviations exactly.
The assumptions concern the conditional mean and the existence of moments. They do not require normally distributed errors, a constant conditional error variance, or independent observations. In particular, the conditional variance of \(\varepsilon\) may change with \(X\), as long as the variance calculations below are finite.
Deriving the Correlation Formula
Write \(\mu_X=E[X]\) and \(\mu_Y=E[Y]\). Since \(E[\varepsilon\mid X]=0\), taking expectations in the model gives \(E[\varepsilon]=0\), and therefore \(\mu_Y=\beta_0+\beta_1\mu_X\). The zero conditional error mean also makes the error uncorrelated with \(X\). The steps below show how these facts determine the covariance and variance.
Center the model and take the expected product with \(X-\mu_X\). The error term contributes zero because its conditional mean given \(X\) is zero.
The linear part and the error have zero covariance, so their variances add. The error variance equals the expected conditional error variance.
Divide by the product of the standard deviations to obtain the correlation.
The fourth line uses conditional expectation: \(X-\mu_X\) is determined once \(X\) is known, so the conditional expectation of its product with \(\varepsilon\) is \((X-\mu_X)E[\varepsilon\mid X]\). The last term is zero by the model assumption. For the variance, use \(Y-\mu_Y=\beta_1(X-\mu_X)+\varepsilon\); the cross-product has expectation zero by the same argument.
For the second equality, the law of total variance gives \(\operatorname{Var}(\varepsilon)=E[\operatorname{Var}(\varepsilon\mid X)]+\operatorname{Var}(E[\varepsilon\mid X])\). The final term is zero because \(E[\varepsilon\mid X]=0\).
The formula gives the sign immediately: because both standard deviations are positive, the correlation has the same sign as \(\beta_1\). If the conditional mean rises as \(X\) increases, the correlation is positive; if it falls, the correlation is negative. If \(\beta_1=0\), the correlation is zero, provided \(Y\) still has positive variance.
Its magnitude is also constrained by the error variation. The response variance contains the variation \(\beta_1^2\operatorname{Var}(X)\) contributed by the linear conditional mean, plus the nonnegative expected conditional error variance. Consequently,
If \(E[\operatorname{Var}(\varepsilon\mid X)]>0\), the inequality is strict. If it is zero, then the error is zero with probability one under the zero conditional mean assumption; for a nonzero slope, \(Y\) lies exactly on the line and the correlation is \(1\) or \(-1\). If \(\operatorname{Var}(Y)=0\), correlation is undefined, so the formula for correlation cannot be used.
Worked Applications
Worked Example: Positive Conditional-Mean Slope
A hypothetical insurer models annual claim cost \(Y\), measured in thousands of dollars, using a risk score \(X\). The model is \(Y=40+3X+\varepsilon\). Assume \(E[\varepsilon\mid X]=0\), \(\operatorname{SD}(X)=2\), and \(\operatorname{Var}(\varepsilon\mid X)=16\) for every value of \(X\). No normal distribution is assumed. Find the population correlation.
Method. Use the conditional-mean correlation formula. The given conditional error variance is constant, so \(E[\operatorname{Var}(\varepsilon\mid X)]=16\).
First, \(\operatorname{Var}(X)=2^2=4\). The slope is \(\beta_1=3\), so the variance contribution from the linear part is \(3^2(4)=36\). Therefore, \(\operatorname{Var}(Y)=36+16=52\), and \(\operatorname{SD}(Y)=\sqrt{52}\).
The covariance is \(\operatorname{Cov}(X,Y)=3(4)=12\). Hence, \(\operatorname{Corr}(X,Y)=12/(2\sqrt{52})=6/\sqrt{52}\approx0.8321\), to four decimal places.
Check. The slope is positive, and the computed correlation is positive. Its magnitude is below one because the error variance is positive. A common mistake is to report the slope, \(3\), as the correlation; the calculation shows that the correlation is \(0.8321\), not \(3\).
Worked Example: Negative Slope With Changing Error Variance
In a hypothetical risk-monitoring study, \(X\) is a centered inspection score and \(Y\) is a loss-cost measure. Assume \(E[X]=20\), \(\operatorname{Var}(X)=25\), and \(Y=70-0.8X+\varepsilon\), with \(E[\varepsilon\mid X]=0\). The conditional error variance is \(\operatorname{Var}(\varepsilon\mid X)=4+0.04(X-20)^2\). No named distribution is specified. Find the correlation.
Method. First average the conditional error variance over \(X\); do not substitute a single value of \(X\) into that variance function.
Since \(E[(X-20)^2]=\operatorname{Var}(X)=25\), \(E[\operatorname{Var}(\varepsilon\mid X)]=4+0.04(25)=4+1=5\). The linear part contributes \((-0.8)^2(25)=0.64(25)=16\) to the response variance. Thus, \(\operatorname{Var}(Y)=16+5=21\), so \(\operatorname{SD}(Y)=\sqrt{21}\).
The predictor standard deviation is \(\sqrt{25}=5\). The correlation is \((-0.8)(5)/\sqrt{21}=-4/\sqrt{21}\approx-0.8729\), to four decimal places.
Check. The result is negative, as required by the negative slope, and its magnitude is less than one. The conditional error variance is not constant, but that does not invalidate the formula. The common mistake is to assume constant variance is required; here its expected value, \(5\), is what enters the response variance.
Worked Example: Checking Whether Summary Values Are Compatible
A proposed population model has a linear conditional mean with slope \(\beta_1=2\), \(\operatorname{SD}(X)=3\), and \(\operatorname{SD}(Y)=8\). Assume the zero conditional error mean and finite second moments. No distribution is specified. Find the implied correlation and the expected conditional error variance.
Method. Apply the correlation formula, then use the variance decomposition to recover the expected conditional error variance.
The variances are \(\operatorname{Var}(X)=3^2=9\) and \(\operatorname{Var}(Y)=8^2=64\). The covariance is \(\beta_1\operatorname{Var}(X)=2(9)=18\). Therefore, \(\operatorname{Corr}(X,Y)=18/(3\cdot8)=18/24=0.7500\).
The linear component contributes \(\beta_1^2\operatorname{Var}(X)=2^2(9)=36\) to \(\operatorname{Var}(Y)\). The remaining variance is \(64-36=28\), so \(E[\operatorname{Var}(\varepsilon\mid X)]=28\).
Check. The correlation is positive and below one, and the recovered error variance is nonnegative, so these summaries are compatible with the model. A negative value for the remaining variance would signal an inconsistency, not a negative error variance.
Common Mistakes and Scope
The relationship follows from a population conditional-mean model, not from a claim that a straight-line pattern describes every observation. The error may be substantial, and its variance may depend on \(X\). What matters for the covariance result is the zero conditional error mean. If that condition fails, the error can contribute to \(\operatorname{Cov}(X,Y)\), and the stated formula need not hold.
- Using the slope as the correlation. The slope has the units of \(Y\) per unit of \(X\); the correlation is obtained only after multiplying by \(\operatorname{SD}(X)/\operatorname{SD}(Y)\).
- Dropping a standard deviation. The formula is \(\beta_1\operatorname{SD}(X)/\operatorname{SD}(Y)\), not \(\beta_1\operatorname{SD}(Y)/\operatorname{SD}(X)\).
- Ignoring conditional error variation. To calculate \(\operatorname{SD}(Y)\), include \(E[\operatorname{Var}(\varepsilon\mid X)]\); the variance of the conditional mean alone is not generally the full response variance.
- Assuming the formula needs normality or constant variance. Neither is required for this population identity. Finite second moments and a zero conditional error mean are the key conditions.
- Applying it to an incompatible model. A correlation outside \([-1,1]\), or a negative implied expected error variance, indicates that the supplied summaries cannot all satisfy the stated assumptions.
Check Your Understanding
For each problem, use the stated population model and assumptions. Answers are given at the end of each question.
- A hypothetical model has slope \(\beta_1=1.2\), \(\operatorname{SD}(X)=2.5\), and \(\operatorname{SD}(Y)=5\), with zero conditional error mean. Find the correlation and the expected conditional error variance. (Answer: \(\operatorname{Corr}(X,Y)=0.6000\); \(E[\operatorname{Var}(\varepsilon\mid X)]=16\).)
- A model has slope \(\beta_1=-3\), \(\operatorname{Var}(X)=4\), and \(E[\operatorname{Var}(\varepsilon\mid X)]=20\). The conditional error mean is zero. Find \(\operatorname{Var}(Y)\) and the correlation. (Answer: \(\operatorname{Var}(Y)=56\); \(\operatorname{Corr}(X,Y)=-6/\sqrt{56}\approx-0.8018\).)
- A zero-conditional-mean model has slope \(0.5\), \(\operatorname{SD}(X)=4\), and \(\operatorname{SD}(Y)=3\). Find the correlation and expected conditional error variance. (Answer: \(\operatorname{Corr}(X,Y)=2/3\approx0.6667\); \(E[\operatorname{Var}(\varepsilon\mid X)]=5\).)
- A model has \(\beta_1=0\), \(E[\varepsilon\mid X]=0\), and \(0<\operatorname{Var}(Y)<\infty\). What is the correlation, and why is it defined? (Answer: \(0\); the covariance is zero and both variable variances are positive and finite.)
- A proposed model has \(\beta_1=2\), \(\operatorname{SD}(X)=3\), and \(\operatorname{SD}(Y)=5\), with zero conditional error mean. Check both the implied correlation and the implied expected conditional error variance. Are the summaries compatible? (Answer: implied correlation \(1.2000\) and implied expected conditional error variance \(-11\); no, the summaries are incompatible.)