The Law of Total Variance

Why Decompose Variance?

A variance can reflect more than one source of variation. Suppose a population contains distinct groups: observations may vary within each group, and the groups may also have different average values. A single overall variance combines both effects. Conditioning on a random variable $Y$ makes this distinction precise: $Y$ identifies the information or group being conditioned on, and the conditional distribution of $X$ describes the remaining variation in $X$ once $Y$ is known.

The law of total variance states that the overall variance is the sum of expected within-condition variance and between-condition variance. The first term averages the conditional variances; the second measures how much the conditional means vary across conditions. The identity applies to discrete and continuous conditioning variables alike.

Conditional Means and Variances

Let $X$ and $Y$ be real-valued random variables with $E[X^2]<\infty$. The conditional expectation $E[X\mid Y]$ is shorthand for $E[X\mid \sigma(Y)]$: it is a function of $Y$, defined up to almost-sure equality, whose expectation over any event determined by $Y$ agrees with that of $X$. Write $m(Y)=E[X\mid Y]$. The finite-second-moment assumption ensures that $m(Y)$ is square-integrable.

A conditional distribution gives a direct way to describe these quantities. For real-valued $X,Y$, a regular conditional distribution of $X$ given $Y$ exists. Write it as $P(X\in A\mid Y=y)$. For values of $y$ outside a set of probability zero, its conditional mean and variance are

m(y)=E[X\mid Y=y], \qquad v(y)=\operatorname{Var}(X\mid Y=y) =E\!\left[(X-m(y))^2\mid Y=y\right].

The resulting random quantities are $m(Y)=E[X\mid Y]$ and $v(Y)=\operatorname{Var}(X\mid Y)$. In a discrete model, conditioning on $Y=y$ means using the conditional probability mass function for each $y$ with $P(Y=y)>0$. In a continuous model, it means using a conditional density when one is available. Conditional distributions, and thus the functions $m$ and $v$, may be chosen differently on conditioning values that occur with probability zero; this does not affect expectations such as $E[v(Y)]$.

Definition: The law of total variance is the identity $$ \operatorname{Var}(X) =E\!\left[\operatorname{Var}(X\mid Y)\right] +\operatorname{Var}\!\left(E[X\mid Y]\right), $$ valid for real-valued $X,Y$ when $E[X^2]<\infty$. The first term is expected conditional variance; the second is the variance of the conditional mean.

Worked Examples

Worked Example: Two Discrete Groups

Let $Y$ identify one of two groups, with $P(Y=0)=0.7$ and $P(Y=1)=0.3$. Suppose $X\mid Y=0$ is Bernoulli with success probability $0.2$, while $X\mid Y=1$ is Bernoulli with success probability $0.8$. Find the total variance by decomposing it.

For a Bernoulli random variable with success probability $p$, the mean is $p$ and the variance is $p(1-p)$. Therefore,

\begin{aligned} E[X\mid Y=0]&=0.2, & \operatorname{Var}(X\mid Y=0)&=0.2(0.8)=0.16,\\ E[X\mid Y=1]&=0.8, & \operatorname{Var}(X\mid Y=1)&=0.8(0.2)=0.16. \end{aligned}

The expected conditional variance is $0.7(0.16)+0.3(0.16)=0.16$. The overall mean, which is also the mean of the conditional means, is $E[X]=0.7(0.2)+0.3(0.8)=0.38$. Thus the variance of the conditional mean is

\begin{aligned} \operatorname{Var}(E[X\mid Y]) &=0.7(0.2-0.38)^2+0.3(0.8-0.38)^2\\ &=0.7(0.0324)+0.3(0.1764)\\ &=0.02268+0.05292=0.0756. \end{aligned}

Adding the components gives $\operatorname{Var}(X)=0.16+0.0756=0.2356$. As a check, marginally $X$ is Bernoulli with success probability $0.38$, so its variance is $0.38(0.62)=0.2356$, as required.

Worked Example: A Normal Conditional Model

Let $Y\sim N(2,4)$, and suppose that conditionally on $Y=y$, $X$ has distribution $N(y,1)$. Find $\operatorname{Var}(X)$ using the decomposition.

The conditional mean and variance are $E[X\mid Y=y]=y$ and $\operatorname{Var}(X\mid Y=y)=1$. Hence $E[\operatorname{Var}(X\mid Y)]=1$. Also, $E[X\mid Y]=Y$, so $\operatorname{Var}(E[X\mid Y])=\operatorname{Var}(Y)=4$. The law gives

\operatorname{Var}(X)=1+4=5.

The finite-second-moment condition holds: $E[X^2]=E[\operatorname{Var}(X\mid Y)]+E[(E[X\mid Y])^2]=1+E[Y^2]=1+(4+2^2)=9$. In particular, $X$ has finite variance. Its mean is $E[X]=E[Y]=2$, and $E[X^2]-(E[X])^2=9-4=5$, an independent check.

Worked Example: A Continuous Uniform Conditional Model

Let $Y$ be uniform on $(0,1)$. Given $Y=y$, let $X$ be uniform on $(0,y)$. Find $\operatorname{Var}(X)$ by conditioning, and verify the result from the marginal moments.

The conditional density is $f_{X\mid Y}(x\mid y)=1/y$ for $0<x<y$. Its mean and variance are those of a uniform distribution on $(0,y)$:

E[X\mid Y=y]=\frac y2,\qquad \operatorname{Var}(X\mid Y=y)=\frac{y^2}{12}.

Since $Y$ is uniform on $(0,1)$, $E[Y^2]=1/3$ and $\operatorname{Var}(Y)=1/12$. Therefore,

E[\operatorname{Var}(X\mid Y)] =\frac{E[Y^2]}{12}=\frac{1}{36}, \qquad \operatorname{Var}(E[X\mid Y]) =\operatorname{Var}(Y/2)=\frac{1}{4}\cdot\frac{1}{12}=\frac{1}{48}.

The total variance is $1/36+1/48=4/144+3/144=7/144$. For a check, the conditional moments give $E[X]=E[Y/2]=1/4$ and $E[X^2]=E[\operatorname{Var}(X\mid Y)+(E[X\mid Y])^2]=E[Y^2/12+Y^2/4]=1/9$. Thus $\operatorname{Var}(X)=1/9-(1/4)^2=16/144-9/144=7/144$.

Deriving the Decomposition

The key idea is to subtract the overall mean in two stages: first subtract the conditional mean, and then subtract the overall mean from that conditional mean. The cross term vanishes because the conditional residual $X-E[X\mid Y]$ has conditional expectation zero.

Theorem: If $X$ is real-valued with $E[X^2]<\infty$, and $Y$ is real-valued, then $$ \operatorname{Var}(X) =E[\operatorname{Var}(X\mid Y)] +\operatorname{Var}(E[X\mid Y]). $$

Proof. Set $m=E[X\mid Y]$ and $\mu=E[X]$. The tower property gives $E[m]=\mu$. Also, conditional expectation is an $L^2$ contraction, so $E[m^2]\leq E[X^2]<\infty$. Consequently $X-m$ and $m-\mu$ are square-integrable, and their product is integrable. Since $m-\mu$ is a function of $Y$, the defining property of conditional expectation gives

E[(X-m)(m-\mu)] =E\!\left[(m-\mu)E[X-m\mid Y]\right] =0,

because $E[X-m\mid Y]=0$ almost surely. Now write $X-\mu=(X-m)+(m-\mu)$, square, and take expectations. The cross term is zero, so

\begin{aligned} \operatorname{Var}(X) &=E[(X-\mu)^2]\\ &=E[(X-m)^2]+E[(m-\mu)^2]\\ &=E\!\left[E[(X-m)^2\mid Y]\right]+\operatorname{Var}(m)\\ &=E[\operatorname{Var}(X\mid Y)]+\operatorname{Var}(E[X\mid Y]). \end{aligned}

The third line uses the tower property and $E[m]=\mu$; the last line uses the definition of conditional variance. This proves the identity. $\square$

Consequences and Interpretation

Both terms on the right-hand side are nonnegative. Therefore, each is at most the total variance:

E[\operatorname{Var}(X\mid Y)]\leq \operatorname{Var}(X), \qquad \operatorname{Var}(E[X\mid Y])\leq \operatorname{Var}(X).

These inequalities follow directly by subtracting either nonnegative term from the decomposition. The first component is zero exactly when $\operatorname{Var}(X\mid Y)=0$ almost surely, which means $X=E[X\mid Y]$ almost surely: knowing $Y$ determines $X$, apart from probability-zero exceptions. The second component is zero exactly when $E[X\mid Y]$ is constant almost surely; by the tower property, that constant must be $E[X]$. In that case the conditional means do not vary with $Y$, although $X$ may still vary within conditions.

The decomposition is useful whenever a model has natural groups, strata, latent states, or predictor values. It distinguishes variation remaining after conditioning from variation associated with differences in conditional means. For the two-group example, the first component measured variation in outcomes within each group, while the second measured the contribution of the groups having different success probabilities.

Common Pitfall

A frequent error is to replace the expected conditional variance by the variance of the conditional means, or to omit one of the terms. They describe different quantities: $\operatorname{Var}(X\mid Y)$ is itself a random quantity that can vary with $Y$, and $E[\operatorname{Var}(X\mid Y)]$ averages it. By contrast, $E[X\mid Y]$ is a random conditional mean, and $\operatorname{Var}(E[X\mid Y])$ measures its variation across values of $Y$.

Another tempting but incorrect shortcut is to claim that the cross term vanishes merely because $X-E[X\mid Y]$ has mean zero. An unconditional mean of zero alone would not establish that its product with $E[X\mid Y]-E[X]$ has mean zero. The needed fact is the stronger conditional identity $E[X-E[X\mid Y]\mid Y]=0$, which makes the cross term vanish after conditioning.

Check Your Understanding

Use the definitions and decomposition above to answer these questions.

In the law of total variance, which term describes within-condition variation, and which describes variation among conditional means?
Suppose $E[X\mid Y]$ is constant almost surely. What does the decomposition then say about $E[\operatorname{Var}(X\mid Y)]$?
For the two-group example, why is the between-group component $0.0756$, rather than $0.05292$?
In the uniform conditional example, calculate $E[X]$ and $E[X^2]$ from the conditional moments, then recover the variance.
Why is an unconditional mean-zero residual not, by itself, enough to justify dropping the cross term in the proof?

What You'll Learn