Proving the Law of Total Variance

Why Variance Splits Into Two Parts

A conditional distribution describes variation after some information is known. The unconditional distribution combines that remaining variation with changes in the conditional mean across possible information values. The law of total variance makes this separation exact: total variation is the average variation within the conditional distributions plus the variation among their means.

This identity is useful when a model is naturally organized into groups, when a conditional distribution is easier to analyze than a marginal one, or when we want to distinguish individual-level variation from differences between groups. Its proof rests on a precise fact: once we subtract the conditional mean, the residual has conditional mean zero and is uncorrelated with any square-integrable quantity determined by the conditioning information.

Definition and Assumptions

Let \(X\) be a random variable with \(E[X^2]<\infty\), and let \(\mathcal G\) be a sigma-field representing the information on which we condition. Write \(M=E[X\mid\mathcal G]\). Conditional expectation is characterized by being \(\mathcal G\)-measurable and satisfying \(E[M1_A]=E[X1_A]\) for every \(A\in\mathcal G\). In particular, \(E[M]=E[X]\). Conditional Jensen's inequality gives \(E[M^2]\leq E[X^2]\), so \(M\) is square-integrable too.

The definition centers \(X\) at its conditional mean before taking the conditional expectation of the squared deviation. Thus \(\operatorname{Var}(X\mid\mathcal G)\) can vary with the information observed; it is not generally a fixed number. Its expectation, \(E[\operatorname{Var}(X\mid\mathcal G)]\), is an ordinary nonnegative number.

Proof of the Decomposition

Set \(M=E[X\mid\mathcal G]\), and define the residual \(Z=X-M\). Then \(Z\) is square-integrable and \(E[Z\mid\mathcal G]=0\). We first justify why the residual contributes no cross-product with \(M-E[X]\). Let \(H=M-E[X]\), which is square-integrable and \(\mathcal G\)-measurable. For each positive integer \(k\), truncate it to \(H_k=\max(-k,\min(H,k))\). This is bounded and \(\mathcal G\)-measurable, so the defining property of conditional expectation gives \( E[ZH_k]=E[E[Z\mid\mathcal G]H_k]=0 \). Also \(H_k\to H\) in mean square: \(|H-H_k|^2\leq |H|^2\), and \(|H-H_k|^2\to0\) almost surely, so dominated convergence applies. By Cauchy–Schwarz, \( |E[Z(H-H_k)]|\leq (E[Z^2])^{1/2}(E[(H-H_k)^2])^{1/2}\to0 \). Consequently \(E[ZH]=0\).

Now \(X-E[X]=Z+H\). Squaring and taking expectations, with all terms integrable by square-integrability, yields

By the tower property applied to the nonnegative integrable variable \(Z^2\), \(E[Z^2]=E[E[Z^2\mid\mathcal G]]=E[\operatorname{Var}(X\mid\mathcal G)]\). Since \(M=E[X\mid\mathcal G]\), substituting gives the stated decomposition. Each term is finite because \(X\) and \(M\) are square-integrable. \(\square\)

Finite Groups: The Within- and Between-Group Form

If \(Y\) takes finitely many values, conditioning on \(Y\) gives the familiar weighted form. For each value \(y\) with \(P(Y=y)>0\), let \(m_y=E[X\mid Y=y]\) and \(v_y=\operatorname{Var}(X\mid Y=y)\). Then the theorem says

The first sum averages the conditional variances. The second is the variance, under the group probabilities, of the group-specific conditional means. Values of \(y\) with probability zero make no contribution and need not be assigned conditional moments.

A Continuous Conditional Model

The same theorem applies when the conditioning variable is continuous. Conditional moments are then functions of the observed conditioning value, and the expectation of a conditional variance averages those functions over the conditioning variable's distribution.

Interpretation and a Common Pitfall

The first contribution, \(E[\operatorname{Var}(X\mid\mathcal G)]\), is the average residual variation after conditioning. The second, \(\operatorname{Var}(E[X\mid\mathcal G])\), measures how much the conditional mean changes with the information. Since the second term is nonnegative, the expected conditional variance cannot exceed the unconditional variance. If the conditional mean is constant almost surely, the between-means term is zero; otherwise it records an additional source of variation.

A common mistake is to average conditional standard deviations and call the result the unconditional standard deviation. The theorem is an identity for variances, not standard deviations: it adds the expected conditional variance to the variance of the conditional mean. Even after finding the total variance, the standard deviation is obtained by taking the square root of the entire sum. For example, in the continuous model above, the terms are \(3/2\) and \(1/12\); the total standard deviation is \(\sqrt{19/12}\), not \(\sqrt{3/2}+\sqrt{1/12}\).

A reliable calculation therefore proceeds in three steps: find the conditional mean and variance, average the conditional variance and calculate the variance of the conditional mean, then add those two quantities. In discrete settings, the averages use group probabilities; in continuous settings, they use expectation over the conditioning variable.