Worked Bivariate Gaussian Conditioning Problems

From a Joint Gaussian Model to a Conditional Distribution

In a bivariate Gaussian model, conditioning on one variable changes the distribution of the other in a particularly structured way: the conditional mean is linear in the observed value, and the conditional variance does not depend on that value. The sign and size of the correlation determine how strongly the conditional mean moves. The calculations below make these statements concrete, while also distinguishing conditional association from independence.

Assume throughout that $(X,Y)$ is jointly normal, with means $\mu_X,\mu_Y$, positive standard deviations $\sigma_X,\sigma_Y$, and correlation $\rho$ satisfying $-1<\rho<1$. The strict inequality makes the covariance matrix positive definite, so the joint distribution has a density. Conditioning on an exact value such as $X=x$ is understood through the conditional density (a regular conditional distribution), not as an event with positive probability.

Definition: The covariance matrix and correlation are $$ \Sigma = \begin{pmatrix} \sigma_X^2 & \rho\sigma_X\sigma_Y\\ \rho\sigma_X\sigma_Y & \sigma_Y^2 \end{pmatrix}, \qquad \rho=\frac{\operatorname{Cov}(X,Y)}{\sigma_X\sigma_Y}. $$ The conditional regression slope of $Y$ on $X$ is the coefficient of $x$ in $E[Y\mid X=x]$. It is not, in general, the same as the slope of $X$ on $Y$.

The Conditional Distribution Formula

For a nondegenerate bivariate normal pair, the conditional law of $Y$ given $X=x$ is normal. Its mean is the marginal mean of $Y$, adjusted by the standardized distance of $x$ from $\mu_X$; its variance is the marginal variance of $Y$ multiplied by $1-\rho^2$.

Theorem: If $(X,Y)$ is jointly normal with $\sigma_X,\sigma_Y>0$ and $|\rho|<1$, then, for every real $x$, $$ Y\mid X=x \sim N\left( \mu_Y+\rho\frac{\sigma_Y}{\sigma_X}(x-\mu_X), \ \sigma_Y^2(1-\rho^2) \right). $$ Interchanging $X$ and $Y$ gives $$ X\mid Y=y \sim N\left( \mu_X+\rho\frac{\sigma_X}{\sigma_Y}(y-\mu_Y), \ \sigma_X^2(1-\rho^2) \right). $$

A derivation also clarifies why the formula holds at each conditioning value. Standardize the variables by setting $U=(X-\mu_X)/\sigma_X$ and $V=(Y-\mu_Y)/\sigma_Y$. Their joint density is $$ f_{U,V}(u,v)=\frac{1}{2\pi\sqrt{1-\rho^2}} \exp\left\{-\frac{u^2-2\rho uv+v^2}{2(1-\rho^2)}\right\}. $$ Define $W=V-\rho U$, so $v=w+\rho u$. The change of variables from $(u,w)$ to $(u,v)$ has Jacobian determinant 1. Substitution gives $$ u^2-2\rho u(w+\rho u)+(w+\rho u)^2 =(1-\rho^2)u^2+w^2. $$ Therefore the joint density of $(U,W)$ factors as $$ f_{U,W}(u,w)= \frac{1}{\sqrt{2\pi}}e^{-u^2/2} \frac{1}{\sqrt{2\pi(1-\rho^2)}}e^{-w^2/[2(1-\rho^2)]}. $$ The factors are the densities of independent $N(0,1)$ and $N(0,1-\rho^2)$ variables. Thus $W$ is independent of $U$, and $V=\rho U+W$. Given $U=u$, $V$ has mean $\rho u$ and variance $1-\rho^2$. Returning to the original scales, $u=(x-\mu_X)/\sigma_X$ and $Y=\mu_Y+\sigma_Y V$, which yields the stated conditional mean and variance. The same argument with $X$ and $Y$ exchanged proves the other formula. $\square$

Worked Example: Positive Correlation and Conditioning on X

Suppose $X$ and $Y$ are jointly normal with $\mu_X=10$, $\mu_Y=20$, $\sigma_X=2$, $\sigma_Y=5$, and $\rho=0.6$. Find the conditional distribution of $Y$ given $X=12$, and the regression line of $Y$ on $X$.

The conditional mean is $$ E[Y\mid X=12] =20+0.6\frac{5}{2}(12-10) =20+1.5(2)=23. $$ The conditional variance is $$ \operatorname{Var}(Y\mid X=12) =5^2(1-0.6^2) =25(0.64)=16. $$ Hence $Y\mid X=12\sim N(23,16)$, with conditional standard deviation $4$.

For a general value $x$, the slope is $\rho\sigma_Y/\sigma_X=0.6(5/2)=1.5$. The intercept is $20-1.5(10)=5$, so $E[Y\mid X=x]=5+1.5x$. At $x=12$, this line gives $5+18=23$, agreeing with the conditional mean calculation. Since the correlation is positive, conditioning on a value of $X$ above its mean raises the conditional mean of $Y$ above $\mu_Y$.

Slopes, Signs, and Conditional Variance

The conditional mean can be written directly using covariance: $$ E[Y\mid X=x]=\mu_Y+ \frac{\operatorname{Cov}(X,Y)}{\operatorname{Var}(X)}(x-\mu_X). $$ Thus the slope of $Y$ on $X$ is $\operatorname{Cov}(X,Y)/\operatorname{Var}(X)$. The slope of $X$ on $Y$, by contrast, is $\operatorname{Cov}(X,Y)/\operatorname{Var}(Y)$. They have the same sign, but usually different magnitudes. Reversing the conditioning direction also changes which marginal variance appears in the conditional variance.

For positive $\rho$, values of $X$ above its mean imply a conditional mean of $Y$ above its mean. For negative $\rho$, the conditional mean moves in the opposite direction. In either case, the conditional variance of $Y$ is $\sigma_Y^2(1-\rho^2)$, which is no larger than its marginal variance $\sigma_Y^2$. The amount of reduction depends on $\rho^2$, not on the sign.

Worked Example: Negative Correlation and Both Conditioning Directions

Let $\mu_X=4$, $\mu_Y=-1$, $\sigma_X=3$, $\sigma_Y=4$, and $\rho=-0.5$. Find the conditional law of $Y$ given $X=7$, then find the conditional mean and variance of $X$ given $Y=3$.

For $Y\mid X=7$, the slope of the conditional mean is $$ \rho\frac{\sigma_Y}{\sigma_X}=-0.5\frac{4}{3}=-\frac{2}{3}. $$ Since $7-\mu_X=3$, the conditional mean is $-1-(2/3)(3)=-3$. The conditional variance is $$ 16(1-(-0.5)^2)=16(0.75)=12. $$ Thus $Y\mid X=7\sim N(-3,12)$. The observed $X$ is above its mean, and the negative correlation shifts the conditional mean of $Y$ below its own mean.

For the reverse direction, $3-\mu_Y=4$, so $$ E[X\mid Y=3] =4+(-0.5)\frac{3}{4}(4) =4-1.5=2.5. $$ The conditional variance is $9(1-0.25)=6.75$. Therefore $X\mid Y=3\sim N(2.5,6.75)$. The two conditional variances differ because the two marginal variances differ.

When Does Conditioning Leave the Distribution Unchanged?

In a jointly Gaussian pair, zero correlation is exactly the condition for independence. This equivalence is special to the joint Gaussian assumption: for general distributions, zero covariance alone does not establish independence. Here the density makes both directions of the Gaussian result explicit.

Theorem: Under the nondegenerate bivariate normal assumptions above, $X$ and $Y$ are independent if and only if $\rho=0$. Equivalently, conditioning on $X=x$ leaves the distribution of $Y$ equal to its marginal distribution for every $x$ if and only if $\rho=0$.

Proof. If $\rho=0$, the joint density displayed above reduces, after standardization, to the product of the standard normal densities for $U$ and $V$. Thus $U$ and $V$ are independent, and so are $X$ and $Y$, because each is a nonzero affine transformation of the corresponding standardized variable. The conditional formula also gives $Y\mid X=x\sim N(\mu_Y,\sigma_Y^2)$, its marginal law.

Conversely, if $X$ and $Y$ are independent and have finite variances, then $\operatorname{Cov}(X,Y)=0$, so $\rho=0$. Alternatively, if the conditional distribution of $Y$ equals its marginal distribution for every $x$, their conditional means must agree: $\mu_Y+\rho(\sigma_Y/\sigma_X)(x-\mu_X)=\mu_Y$ for every real $x$. Since $\sigma_X,\sigma_Y>0$, this forces $\rho=0$. $\square$

Worked Example: Zero Correlation and No Conditional Update

Suppose $(X,Y)$ is jointly normal with means $\mu_X=2$ and $\mu_Y=-3$, standard deviations $\sigma_X=2$ and $\sigma_Y=3$, and covariance $0$. Determine the conditional distribution of $Y$ given $X=5$, and decide whether $X$ and $Y$ are independent.

The correlation is $0/(2\cdot3)=0$. The slope of the conditional mean is $\rho\sigma_Y/\sigma_X=0$, and the conditional variance is $9(1-0^2)=9$. Therefore $$ Y\mid X=5\sim N(-3,9). $$ This is exactly the marginal distribution $Y\sim N(-3,9)$, so the observed value $X=5$ does not update the distribution of $Y$. Because the pair is jointly Gaussian and has zero correlation, $X$ and $Y$ are independent.

Common Mistake: Reversing a Regression Slope

A frequent error is to use $\rho\sigma_X/\sigma_Y$ as the slope of $Y$ on $X$. That coefficient belongs to the reverse regression, $E[X\mid Y=y]$. For $Y$ on $X$, the coefficient is $\rho\sigma_Y/\sigma_X$, or equivalently $\operatorname{Cov}(X,Y)/\operatorname{Var}(X)$. Check the direction by asking which variable is being predicted and which variance is in the denominator.

Another error is to conclude that negative correlation increases conditional variance, or that positive correlation decreases it. The conditional variance uses $\rho^2$, so its change depends on the strength of association, not its direction. A final caution: the conditional variance is constant across conditioning values in this Gaussian model, but the conditional mean generally is not. A value far from $\mu_X$ changes the conditional mean according to the regression slope.

Key takeaway: For a nondegenerate bivariate normal pair, the conditional mean of $Y$ given $X=x$ has slope $\operatorname{Cov}(X,Y)/\operatorname{Var}(X)$, while the conditional variance is $\operatorname{Var}(Y)(1-\rho^2)$. Zero correlation is equivalent to independence for this joint Gaussian model.

Check Your Understanding

Use the stated assumptions in each question; unless specified otherwise, assume the variables are jointly normal and nondegenerate.

If $\rho<0$ and $x>\mu_X$, is $E[Y\mid X=x]$ above or below $\mu_Y$? Explain using the conditional mean formula.
For $\sigma_X=2$, $\sigma_Y=6$, and $\rho=0.5$, find the slopes of $Y$ on $X$ and $X$ on $Y$.
If $\rho=-0.8$, what fraction of the marginal variance of $Y$ remains in $\operatorname{Var}(Y\mid X=x)$? Does the sign of $\rho$ affect this fraction?
Why does zero covariance establish independence for a jointly Gaussian pair but not, in general, for an arbitrary pair of random variables?
Suppose $Y\mid X=x$ has a conditional mean that is constant in $x$. Under the nondegenerate bivariate Gaussian assumptions, what does this imply about $\rho$ and independence?