The Regression Line in a Bivariate Normal Model

Introduction

In a bivariate normal model, the conditional mean of one variable given the other is a straight line. This makes it possible to predict the average value of \(X\) among outcomes associated with a specified value of \(Y\), while keeping clear that this is a conditional mean, not a guarantee about an individual outcome.

The direction matters. The regression line of \(X\) on \(Y\) predicts \(X\) from \(Y\); its slope uses the standard deviation of \(X\) in the numerator and the standard deviation of \(Y\) in the denominator. Reversing those variables generally gives a different line. This tutorial develops the line from the conditional normal formula, then applies it to hypothetical exam-style settings.

The Conditional Mean as a Regression Line

Definition: Suppose \(X\) and \(Y\) have a nondegenerate bivariate normal distribution, with means \(\mu_X\) and \(\mu_Y\), standard deviations \(\sigma_X>0\) and \(\sigma_Y>0\), and correlation \(\rho\), where \(-1<\rho<1\). The regression line of \(X\) on \(Y\) is the graph of the conditional mean \(E[X\mid Y=y]\) as a function of \(y\). Its centered form is \(E[X\mid Y=y]=\mu_X+\rho\frac{\sigma_X}{\sigma_Y}(y-\mu_Y)\).

The formula is a line because it consists of a constant, \(\mu_X\), plus a constant multiple of \(y-\mu_Y\). Define its slope as \(\beta_{X\mid Y}\). Expanding the centered form gives the usual intercept-slope form:

\beta_{X\mid Y}=\rho\frac{\sigma_X}{\sigma_Y} =\frac{\operatorname{Cov}(X,Y)}{\operatorname{Var}(Y)},\qquad \alpha_{X\mid Y}=\mu_X-\beta_{X\mid Y}\mu_Y, \qquad E[X\mid Y=y]=\alpha_{X\mid Y}+\beta_{X\mid Y}y.

Here \(\operatorname{Cov}(X,Y)=\rho\sigma_X\sigma_Y\) and \(\operatorname{Var}(Y)=\sigma_Y^2\), so their ratio is \(\rho\sigma_X/\sigma_Y\). The slope has units of \(X\) per unit of \(Y\). The intercept is the line's value at \(y=0\); depending on the setting, zero may or may not be a meaningful value of \(Y\). The centered form is often easier to interpret because it shows directly that the line passes through \((\mu_Y,\mu_X)\).

Why the Formula Holds

For a bivariate normal pair, the conditional distribution of \(X\) given \(Y=y\) is normal with mean \(\mu_X+\rho\frac{\sigma_X}{\sigma_Y}(y-\mu_Y)\). This is the conditional normal result. The regression-line formula follows by taking the mean of that conditional distribution: a normal distribution's mean is its location parameter. The remaining algebra shows why its slope can also be written using covariance and variance.

\begin{aligned} E[X\mid Y=y] &=\mu_X+\rho\frac{\sigma_X}{\sigma_Y}(y-\mu_Y)\\ &=\mu_X+\frac{\operatorname{Cov}(X,Y)}{\operatorname{Var}(Y)}(y-\mu_Y)\\ &=\left(\mu_X-\frac{\operatorname{Cov}(X,Y)}{\operatorname{Var}(Y)}\mu_Y\right) +\frac{\operatorname{Cov}(X,Y)}{\operatorname{Var}(Y)}y. \end{aligned}

Thus the slope is the change in conditional mean \(X\) per one-unit increase in \(Y\), and the intercept is whatever constant makes the line pass through the means. This justification depends on the bivariate normal model: for arbitrary distributions, the conditional mean need not be a straight line. The covariance-to-variance ratio can still define a linear least-squares predictor in broader settings, but that does not by itself make it the exact conditional mean.

Using the Regression Line

A reliable calculation keeps the conditioning direction visible at each step. Identify which variable is being predicted, find the slope in that direction, write the line, and then substitute the specified value. The slope's sign must agree with the direction of the association: positive correlation gives an increasing conditional-mean line, while negative correlation gives a decreasing one.

Identify the response and conditioning variable.
For the regression of \(X\) on \(Y\), predict \(X\) and use \(Y\) as the input.

Calculate the slope.
Use \(\beta_{X\mid Y}=\rho\sigma_X/\sigma_Y\), or \(\operatorname{Cov}(X,Y)/\operatorname{Var}(Y)\) when covariance and variance are supplied.

Write and apply the line.
Use \(E[X\mid Y=y]=\mu_X+\beta_{X\mid Y}(y-\mu_Y)\), or calculate the intercept \(\alpha_{X\mid Y}=\mu_X-\beta_{X\mid Y}\mu_Y\) first.

Check the result.
At \(y=\mu_Y\), the predicted conditional mean must be \(\mu_X\). A positive or negative slope must also match the sign of \(\rho\).

Worked Example: Finding and Interpreting a Positive-Slope Line

Method and model. A hypothetical bivariate normal model describes \(X\), a service-time measure in minutes, and \(Y\), a workload index in points. Suppose \(\mu_X=50\) minutes, \(\mu_Y=40\) points, \(\sigma_X=8\) minutes, \(\sigma_Y=5\) points, and \(\rho=0.60\). Find the regression line of \(X\) on \(Y\), then predict the conditional mean of service time when \(Y=45\).

Calculate the slope. Since \(X\) is predicted from \(Y\), the appropriate standard-deviation ratio is \(\sigma_X/\sigma_Y\):

\beta_{X\mid Y}=0.60\left(\frac{8}{5}\right)=0.60(1.6)=0.96 \quad\text{minutes per point}.

Calculate the intercept. Use the mean pair \((\mu_Y,\mu_X)=(40,50)\):

\alpha_{X\mid Y}=\mu_X-\beta_{X\mid Y}\mu_Y =50-(0.96)(40)=50-38.4=11.6.

Therefore the line is \(E[X\mid Y=y]=11.6+0.96y\). At \(y=45\),

E[X\mid Y=45]=11.6+(0.96)(45)=11.6+43.2=54.8\text{ minutes}.

Check and interpretation. At the mean workload, \(y=40\), the line gives \(11.6+(0.96)(40)=50\), which is \(\mu_X\). The positive slope agrees with the positive correlation. The prediction is a conditional mean of 54.8 minutes, not a claim that every observation with workload index 45 has that service time. The main mistake avoided here is using \(\sigma_Y/\sigma_X\), which would be the wrong standard-deviation ratio for predicting \(X\) from \(Y\).

Worked Example: Recovering a Negative Slope from Conditional Means

Method and model. In a hypothetical bivariate normal model, \(X\) is a performance measure and \(Y\) is a workload measure. The means are \(\mu_X=12\) units and \(\mu_Y=30\) points; the standard deviations are \(\sigma_X=4\) units and \(\sigma_Y=5\) points. The conditional mean of \(X\) at \(Y=24\) is 15.2 units. Find the line's slope, the correlation, and the conditional mean at \(Y=33\).

Because the line passes through \((\mu_Y,\mu_X)=(30,12)\), the two known points on the graph of conditional mean against \(Y\) are \((30,12)\) and \((24,15.2)\). Calculate the slope using change in the predicted \(X\) divided by change in \(Y\):

\beta_{X\mid Y}=\frac{15.2-12}{24-30} =\frac{3.2}{-6}=-0.533333\ldots

The negative sign is essential: the conditional mean is higher at \(Y=24\) than at \(Y=30\), so the line decreases as \(Y\) increases. From \(\beta_{X\mid Y}=\rho\sigma_X/\sigma_Y\), solve for the correlation:

\rho=\beta_{X\mid Y}\frac{\sigma_Y}{\sigma_X} =(-0.533333\ldots)\frac{5}{4} =-0.666666\ldots

At \(Y=33\), use the centered line to preserve the known point:

E[X\mid Y=33] =12+(-0.533333\ldots)(33-30) =12-1.6 =10.4\text{ units}.

Check and interpretation. Moving from \(Y=30\) to \(Y=33\) increases \(Y\) by 3, so the conditional mean changes by \(3(-0.533333\ldots)=-1.6\), from 12 to 10.4. The implied correlation is between \(-1\) and \(1\), as required for this nondegenerate model. A common mistake is to compute \(15.2-12\) correctly but divide by \(30-24\), reversing the order in only the denominator and producing a false positive slope. The numerator and denominator must use the same point order.

Worked Example: Using Covariance and Solving for an Input

Method and model. A hypothetical bivariate normal model relates \(X\), an operating cost in thousands of dollars, to \(Y\), a utilization index. Suppose \(\mu_X=72\), \(\mu_Y=18\), \(\sigma_X=10\), \(\sigma_Y=5\), and \(\operatorname{Cov}(X,Y)=-30\). Find the regression line of \(X\) on \(Y\), predict the conditional mean at \(Y=21\), and find the value of \(Y\) at which the conditional mean of \(X\) is 75.

First verify consistency of the supplied covariance with the standard deviations. The implied correlation is \(\rho=-30/[(10)(5)]=-0.60\), which is a valid correlation. The slope can be found directly from covariance and the variance of the conditioning variable:

\operatorname{Var}(Y)=5^2=25,\qquad \beta_{X\mid Y}=\frac{-30}{25}=-1.2.

Calculate the intercept and write the line:

\alpha_{X\mid Y}=72-(-1.2)(18)=72+21.6=93.6, \qquad E[X\mid Y=y]=93.6-1.2y.

At \(Y=21\), the conditional mean is

E[X\mid Y=21]=93.6-(1.2)(21)=93.6-25.2=68.4 \text{ thousand dollars}.

To find the utilization index associated with conditional mean 75, solve the line equation rather than substituting 75 for \(Y\):

75=93.6-1.2y,\qquad 1.2y=93.6-75=18.6,\qquad y=\frac{18.6}{1.2}=15.5.

Check and interpretation. At \(y=18\), the line gives \(93.6-21.6=72\), the specified mean of \(X\). The slope is negative, consistent with the negative covariance and correlation. The predicted conditional mean at 21 is 68.4 thousand dollars; the conditional mean reaches 75 thousand dollars at \(Y=15.5\). The mistake avoided is dividing covariance by \(\operatorname{Var}(X)\): the regression of \(X\) on \(Y\) uses the variance of the conditioning variable, \(Y\).

Interpretation and Common Exam Traps

The regression line describes how the conditional mean of \(X\) changes with \(Y\). It is not an equation that every pair of observations must satisfy. Even under the bivariate normal model, individual values of \(X\) at a fixed \(Y\) need not equal their conditional mean. A requested prediction from this line should therefore be reported as \(E[X\mid Y=y]\), or described in words as the model's predicted average \(X\) for that specified \(Y\).

The slope depends on both the correlation and the units of the variables. Its sign is the sign of \(\rho\), but its magnitude is not generally \(|\rho|\). For example, changing the units of \(X\) changes \(\sigma_X\) and hence the numerical slope, even though the strength and direction of association do not change. The centered formula is often safest for numerical work because it starts at the known mean pair and applies the change in \(Y\) directly.

Reversing the variables: The slope for \(X\) on \(Y\) is \(\rho\sigma_X/\sigma_Y\); for \(Y\) on \(X\), it is \(\rho\sigma_Y/\sigma_X\). These are generally different.
Dropping a negative sign: Compute change in predicted \(X\) divided by change in \(Y\), keeping the point order consistent in both differences.
Using the wrong variance: In the covariance form for the regression of \(X\) on \(Y\), divide by \(\operatorname{Var}(Y)\), not by \(\operatorname{Var}(X)\).
Confusing intercept and mean: The intercept is the predicted value at \(Y=0\); the line's predicted value at \(Y=\mu_Y\) is \(\mu_X\).
Calling the prediction an observed value: A point on the line is a conditional mean, not a deterministic value for an individual pair.

Key takeaway: For the regression line of \(X\) on \(Y\), use \(E[X\mid Y=y]=\mu_X+\rho\frac{\sigma_X}{\sigma_Y}(y-\mu_Y)\). Its slope is \(\operatorname{Cov}(X,Y)/\operatorname{Var}(Y)\), it passes through \((\mu_Y,\mu_X)\), and its direction must agree with the sign of the correlation.

Check Your Understanding

For each question, identify the line of \(X\) on \(Y\) before making any prediction. Each setting is hypothetical and assumes a nondegenerate bivariate normal model.

Suppose \(\mu_X=24\), \(\mu_Y=10\), \(\sigma_X=6\), \(\sigma_Y=3\), and \(\rho=0.50\). Find the intercept and slope, then find \(E[X\mid Y=12]\). (Answer: slope \(1\), intercept \(14\), and conditional mean \(26\).)
Suppose \(\mu_X=80\), \(\mu_Y=50\), \(\sigma_X=12\), \(\sigma_Y=8\), and \(\rho=-0.25\). Find the regression line and \(E[X\mid Y=42]\). (Answer: slope \(-0.375\), intercept \(98.75\), and conditional mean \(83\).)
Suppose \(\mu_X=9\), \(\mu_Y=4\), \(\operatorname{Var}(Y)=36\), and \(\operatorname{Cov}(X,Y)=18\). Find the line and \(E[X\mid Y=7]\). (Answer: slope \(0.5\), intercept \(7\), and conditional mean \(10.5\).)
The means are \(\mu_X=40\) and \(\mu_Y=16\). The conditional mean at \(Y=10\) is 43, and \(\sigma_X=9,\ \sigma_Y=6\). Find the slope, correlation, and conditional mean at \(Y=22\). (Answer: slope \(-0.5\), correlation \(-1/3\), and conditional mean \(37\).)
Suppose \(\mu_X=100\), \(\mu_Y=25\), \(\sigma_X=15\), \(\sigma_Y=5\), and \(\rho=0.40\). Find the regression line, the conditional mean at \(Y=30\), and the value of \(Y\) at which the conditional mean is 112. (Answer: slope \(1.2\), intercept \(70\), conditional mean \(106\) at \(Y=30\), and \(Y=35\) for conditional mean \(112\).)

What You'll Learn