Tutorials › Exam P - Probability › Conditional Variance and Residual Independence

Home  /  Tutorials  /  Exam P / Bivariate Normal Basics

Exam P · Bivariate Normal Basics · Tutorial 860 of 915

Conditional Variance and Residual Independence

Learn to calculate conditional variance and verify, using the bivariate normal density, when the regression residual is independent of the observed variable.

Advanced 12 min read

What You'll Learn

  • Calculate the conditional variance of one bivariate normal variable given the other.
  • Express conditional variance using either standard deviations and correlation or covariance.
  • Define the regression residual from the conditional mean.
  • Derive residual independence by factoring the transformed bivariate normal density.
  • Apply the results to conditional probabilities and linear normal models.
  • Recognize why zero conditional mean alone does not prove independence.

Why Conditional Variance Matters

In a bivariate normal model, observing one variable changes the conditional mean of the other. It also changes our uncertainty about that variable. The conditional variance describes that remaining uncertainty after the observed value is known. A key feature of the model is that this variance does not depend on the particular value observed.

The same structure identifies a useful residual: subtract the conditional mean of \(X\) from \(X\). In the bivariate normal model, that residual is independent of \(Y\). This is stronger than saying that its conditional mean is zero or that it is uncorrelated with \(Y\); the density calculation below establishes independence directly.

Model and notation: Let \((X,Y)\) have a nondegenerate bivariate normal distribution, with means \(\mu_X,\mu_Y\), positive standard deviations \(\sigma_X,\sigma_Y\), and correlation \(\rho\), where \(-1<\rho<1\). Thus \(\operatorname{Var}(X)=\sigma_X^2\), \(\operatorname{Var}(Y)=\sigma_Y^2\), and \(\operatorname{Cov}(X,Y)=\rho\sigma_X\sigma_Y\).

The conditional distribution of \(X\) given \(Y=y\) is normal. Its conditional mean is the regression line \(E[X\mid Y=y]=\mu_X+\rho\frac{\sigma_X}{\sigma_Y}(y-\mu_Y)\). The conditional variance is \(\sigma_X^2(1-\rho^2)\), for every \(y\). These formulas apply to the nondegenerate bivariate normal model just stated; they are not general formulas for arbitrary pairs of random variables.

Theorem: For a nondegenerate bivariate normal pair, $$ X\mid Y=y\sim N\left(\mu_X+\rho\frac{\sigma_X}{\sigma_Y}(y-\mu_Y),\ \sigma_X^2(1-\rho^2)\right). $$ Equivalently, using covariance, $$ E[X\mid Y=y]=\mu_X+\frac{\operatorname{Cov}(X,Y)}{\operatorname{Var}(Y)}(y-\mu_Y), \qquad \operatorname{Var}(X\mid Y=y)=\operatorname{Var}(X)-\frac{\operatorname{Cov}(X,Y)^2}{\operatorname{Var}(Y)}. $$

Reading the Conditional Variance

The quantity \(\sigma_X^2(1-\rho^2)\) is the variance left in \(X\) after accounting for its linear relationship with \(Y\). If \(\rho=0\), it equals the original variance of \(X\). As \(|\rho|\) approaches 1, it approaches zero: knowing \(Y\) then leaves progressively less uncertainty about \(X\). In the nondegenerate model, \(-1<\rho<1\), so the conditional variance is positive.

The covariance version is often more convenient when a problem gives variances and covariance directly. It also provides a useful check: first calculate the regression slope as \(\operatorname{Cov}(X,Y)/\operatorname{Var}(Y)\), then calculate the conditional variance by subtracting \(\operatorname{Cov}(X,Y)^2/\operatorname{Var}(Y)\) from \(\operatorname{Var}(X)\).

Worked Example: Conditional Probability After an Observation

A hypothetical model for the annual costs \(X\) and \(Y\) of two related services is bivariate normal. Suppose \(E[X]=50\), \(\operatorname{Var}(X)=100\), \(E[Y]=20\), \(\operatorname{Var}(Y)=25\), and \(\operatorname{Corr}(X,Y)=0.6\). Find the conditional distribution of \(X\) given \(Y=25\), and calculate \(P(X>64\mid Y=25)\).

Method. Use the conditional normal formulas. The standard deviations are \(\sigma_X=\sqrt{100}=10\) and \(\sigma_Y=\sqrt{25}=5\).

The conditional mean is $$ 50+0.6\frac{10}{5}(25-20) =50+1.2(5) =56. $$ The conditional variance is $$ 100(1-0.6^2)=100(1-0.36)=64. $$ Thus \(X\mid Y=25\sim N(56,64)\), with conditional standard deviation \(\sqrt{64}=8\).

Standardizing the requested threshold gives $$ P(X>64\mid Y=25) =P\left(Z>\frac{64-56}{8}\right) =P(Z>1) \approx 0.1587, $$ where \(Z\sim N(0,1)\). The conditional mean and variance have been checked against the model: \(Y=25\) is one standard deviation above its mean, and the positive correlation makes the conditional mean of \(X\) exceed 50; the conditional variance is less than 100.

Common mistake avoided: The observation \(Y=25\) changes the conditional mean, but it does not change the conditional variance. Substituting \(25\) into the variance formula would be an error.

The Regression Residual

Define the conditional-mean function \(m_X(Y)=\mu_X+\rho\frac{\sigma_X}{\sigma_Y}(Y-\mu_Y)\). The regression residual is the difference between the realized value of \(X\) and this fitted conditional mean: \(R=X-m_X(Y)\). Since \(m_X(Y)=E[X\mid Y]\), the residual measures the part of \(X\) not accounted for by its conditional mean given \(Y\).

Definition: The bivariate normal regression residual of \(X\) on \(Y\) is $$ R=X-\left[\mu_X+\rho\frac{\sigma_X}{\sigma_Y}(Y-\mu_Y)\right]. $$ In the nondegenerate bivariate normal model, \(R\sim N(0,\sigma_X^2(1-\rho^2))\) and \(R\) is independent of \(Y\).

Why the Residual Is Independent of \(Y\)

A zero conditional mean for \(R\) follows from the conditional-mean formula, but that fact alone does not establish independence. To prove independence here, standardize both variables and factor their joint density after transforming the standardized \(X\) coordinate into a residual coordinate.

1
Standardize the pair.
Let \(U=(X-\mu_X)/\sigma_X\) and \(V=(Y-\mu_Y)/\sigma_Y\). Their joint bivariate normal density is proportional to \(\exp\left[-(u^2-2\rho uv+v^2)/(2(1-\rho^2))\right]\).
2
Complete the square.
The exponent's quadratic expression satisfies \(u^2-2\rho uv+v^2=(u-\rho v)^2+(1-\rho^2)v^2\).
3
Transform to the residual coordinate.
Set \(W=(U-\rho V)/\sqrt{1-\rho^2}\). Then \(U=\rho V+\sqrt{1-\rho^2}W\); the absolute Jacobian determinant for the transformation from \((w,v)\) to \((u,v)\) is \(\sqrt{1-\rho^2}\).
4
Factor the transformed density.
Substitution into the density, including that Jacobian, gives \(f_{W,V}(w,v)=\frac{1}{2\pi}\exp(-w^2/2)\exp(-v^2/2)=\phi(w)\phi(v)\). The joint density is the product of the marginal standard normal densities, so \(W\) and \(V\) are independent.

Returning to the original variables, $$ R=\sigma_X(U-\rho V)=\sigma_X\sqrt{1-\rho^2}\,W. $$ Because \(R\) is a function of \(W\), and \(W\) is independent of \(V\), \(R\) is independent of \(V\), and therefore of \(Y\). The same expression shows that \(R\) is normal with mean zero and variance \(\sigma_X^2(1-\rho^2)\). This proves both the residual variance and the independence claim for the stated bivariate normal model.

The density argument is essential to the claim as presented: a zero covariance or zero conditional mean on its own is not a general proof of independence. The result follows here from the particular form of the bivariate normal density.

Worked Example: Using Covariance to Find Conditional Spread

A hypothetical bivariate normal model describes the annual amounts \(X\) and \(Y\) recorded for two related portfolios. Let \(E[X]=200\), \(E[Y]=40\), \(\operatorname{Var}(X)=36\), \(\operatorname{Var}(Y)=64\), and \(\operatorname{Cov}(X,Y)=24\). Given \(Y=48\), find the conditional mean and variance of \(X\), then calculate the probability that \(X\) exceeds the conditional mean by one conditional standard deviation.

Method. Use the covariance forms of the conditional formulas. The regression slope is $$ \frac{\operatorname{Cov}(X,Y)}{\operatorname{Var}(Y)} =\frac{24}{64} =\frac{3}{8}. $$

The conditional mean at \(Y=48\) is $$ 200+\frac{3}{8}(48-40) =200+\frac{3}{8}(8) =203. $$ The conditional variance and standard deviation are $$ 36-\frac{24^2}{64} =36-\frac{576}{64} =36-9 =27, \qquad \sqrt{27}=3\sqrt{3}. $$

One conditional standard deviation above the conditional mean is \(203+3\sqrt{3}\). Since the conditional distribution is normal, $$ P(X>203+3\sqrt{3}\mid Y=48)=P(Z>1)\approx 0.1587. $$ As a check, the given covariance implies \(\rho=24/(\sqrt{36}\sqrt{64})=24/(6\cdot8)=0.5\). The alternative variance calculation is \(36(1-0.5^2)=36(0.75)=27\), matching the covariance calculation.

Common mistake avoided: The denominator in the regression slope is \(\operatorname{Var}(Y)=64\), not the standard deviation \(\sqrt{64}=8\). Using 8 would give the wrong conditional mean.

Worked Example: A Model Built from an Independent Residual

Suppose a hypothetical annual measurement is modeled as \(X=5+2Y+\varepsilon\), where \(Y\sim N(10,4)\), \(\varepsilon\sim N(0,9)\), and \(\varepsilon\) is independent of \(Y\). Find the marginal mean and variance of \(X\), the conditional distribution of \(X\) given \(Y=12\), and the probability \(P(X>32\mid Y=12)\).

Method. First calculate the moments from the model. Since \(E[\varepsilon]=0\), $$ E[X]=5+2E[Y]+E[\varepsilon]=5+2(10)+0=25. $$ Independence gives \(\operatorname{Cov}(Y,\varepsilon)=0\), so $$ \operatorname{Var}(X)=\operatorname{Var}(2Y+\varepsilon) =4\operatorname{Var}(Y)+\operatorname{Var}(\varepsilon) =4(4)+9=25. $$ Also, \(\operatorname{Cov}(X,Y)=2\operatorname{Var}(Y)+\operatorname{Cov}(\varepsilon,Y)=2(4)+0=8\).

Given \(Y=12\), the model itself gives $$ X=5+2(12)+\varepsilon=29+\varepsilon. $$ Therefore \(X\mid Y=12\sim N(29,9)\), with conditional mean 29 and variance 9. The requested probability is $$ P(X>32\mid Y=12) =P\left(\frac{X-29}{3}>\frac{32-29}{3}\,\middle|\,Y=12\right) =P(Z>1) \approx 0.1587. $$

The bivariate normal formulas provide an independent check. The correlation is \(8/(\sqrt{25}\sqrt{4})=8/(5\cdot2)=0.8\). The conditional mean at 12 is \(25+0.8(5/2)(12-10)=25+2(2)=29\), and the conditional variance is \(25(1-0.8^2)=25(0.36)=9\). The residual from the conditional mean is exactly \(\varepsilon\), which is independent of \(Y\) by the model assumption.

Common mistake avoided: The intercept 5 is constant, so it contributes nothing to the variance. The variance comes from \(2Y+\varepsilon\), with the covariance term equal to zero because independence was specified.

Common Errors and Scope

  • Using the wrong variance expression. The conditional variance is \(\sigma_X^2(1-\rho^2)\), not \(\sigma_X^2(1-\rho)\). For example, with \(\sigma_X^2=100\) and \(\rho=0.6\), the correct value is \(100(1-0.36)=64\); the incorrect expression gives \(100(1-0.6)=40\).
  • Letting the observed value change the conditional variance. In this model, \(y\) appears in the conditional mean, but not in the conditional variance.
  • Confusing variance with standard deviation. The conditional standard deviation is \(\sigma_X\sqrt{1-\rho^2}\), the square root of the conditional variance.
  • Claiming independence from a zero conditional mean alone. The residual's independence follows from the bivariate normal structure and the factorization of its transformed density, not merely from \(E[R\mid Y]=0\).
Key takeaway: For a nondegenerate bivariate normal pair, the conditional variance of \(X\) given \(Y=y\) is \(\sigma_X^2(1-\rho^2)\) for every \(y\). Subtracting the conditional mean from \(X\) produces a normal residual independent of \(Y\), as shown by the factorization of the transformed joint density.

Check Your Understanding

For each problem, use the bivariate normal conditional formulas or the stated independent-residual model. Answers are given at the end of each question.

  1. A bivariate normal pair has \(\operatorname{Var}(X)=16\) and correlation \(-0.5\). Find \(\operatorname{Var}(X\mid Y=y)\) for any \(y\), and give the conditional standard deviation. (Answer: conditional variance \(12\); conditional standard deviation \(2\sqrt{3}\).)
  2. A bivariate normal pair has \(\mu_X=80\), \(\mu_Y=12\), \(\sigma_X=9\), \(\sigma_Y=6\), and \(\rho=2/3\). Find the conditional mean and variance of \(X\) given \(Y=15\). (Answer: conditional mean \(83\); conditional variance \(45\).)
  3. A bivariate normal pair has \(\mu_X=120\), \(\mu_Y=30\), \(\operatorname{Var}(X)=100\), \(\operatorname{Var}(Y)=64\), and \(\operatorname{Cov}(X,Y)=48\). Given \(Y=38\), find the conditional mean and variance of \(X\). (Answer: conditional mean \(126\); conditional variance \(64\).)
  4. Let \(Y\sim N(5,9)\), and let \(\varepsilon\sim N(0,16)\) be independent of \(Y\). Define \(X=7+0.5Y+\varepsilon\). Find \(E[X]\), \(\operatorname{Var}(X)\), and the conditional variance of \(X\) given \(Y=y\). (Answer: \(E[X]=9.5\), \(\operatorname{Var}(X)=18.25\), and \(\operatorname{Var}(X\mid Y=y)=16\).)
  5. A bivariate normal pair has \(\mu_X=30\), \(\mu_Y=10\), \(\sigma_X=5\), \(\sigma_Y=4\), and \(\rho=-0.6\). Given \(Y=6\), find the conditional mean and variance of \(X\), and calculate \(P(X>37\mid Y=6)\). (Answer: conditional mean \(33\), conditional variance \(16\), and probability \(0.1587\).)

Previous

Next

Back to Tutorials
Clinical Biostats