The Tower Property for Nested Conditioning

Why Conditioning in Stages Works

Sometimes information arrives in layers. We may first learn a broad category and then a more specific outcome within that category. The tower property says that if we calculate an expectation using the more detailed information, then average that result using only the broad information, we recover the conditional expectation based on the broad information alone. The averaging must respect the probabilities of the more detailed outcomes.

We work on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), and assume the random variable \(X\) is integrable, meaning \(\mathbb{E}|X|<\infty\). Conditional expectations are defined up to equality almost surely. When conditioning on an event \(A\) in a finite or countable partition, the quotient \(\mathbb{E}[X\mathbf{1}_A]/\mathbb{P}(A)\) is used only when \(\mathbb{P}(A)>0\).

For a finite partition \(\mathcal{A}=\{A_1,\ldots,A_r\}\), the conditional expectation given that partition is constant on each cell with positive probability: \(\mathbb{E}[X\mid\mathcal{A}]=\mathbb{E}[X\mathbf{1}_{A_i}]/\mathbb{P}(A_i)\) on \(A_i\). Its value on a zero-probability cell can be chosen arbitrarily without changing the random variable almost surely.

Proof. Fix a cell \(A\in\mathcal{A}\) with \(\mathbb{P}(A)>0\). The cells \(B\in\mathcal{B}\) contained in \(A\) partition \(A\). On each such \(B\) with positive probability, \(\mathbb{E}[X\mid\mathcal{B}]\) has value \(\mathbb{E}[X\mathbf{1}_B]/\mathbb{P}(B)\). Therefore its average over \(A\) is \(\sum_{B\subseteq A,\ \mathbb{P}(B)>0}\mathbb{P}(B)\mathbb{E}[X\mathbf{1}_B]/(\mathbb{P}(B)\mathbb{P}(A))\). The terms with \(\mathbb{P}(B)=0\) contribute zero to \(\mathbb{E}[X\mathbf{1}_A]\), and additivity over the cells gives \(\sum_{B\subseteq A}\mathbb{E}[X\mathbf{1}_B]=\mathbb{E}[X\mathbf{1}_A]\). Thus the average is \(\mathbb{E}[X\mathbf{1}_A]/\mathbb{P}(A)\), exactly the value of \(\mathbb{E}[X\mid\mathcal{A}]\) on \(A\). This holds on every positive-probability cell \(A\); all other cells together have probability zero. Hence the two conditional expectations are equal almost surely. \(\square\)

A Finite-Partition Calculation

Nested Conditioning on Discrete Random Variables

For a discrete random variable \(V\), conditioning on \(V\) means conditioning on its value: \(\mathbb{E}[X\mid V]\) takes the value \(\mathbb{E}[X\mid V=v]\) when \(V=v\), for values \(v\) with \(\mathbb{P}(V=v)>0\). Suppose \(U=g(V)\), so \(U\) is determined by \(V\). The events \(\{V=v\}\) refine the events \(\{U=u\}\). Consequently, for every \(u\) with positive probability, the conditional mean given \(U=u\) is the probability-weighted average of the conditional means given the possible \(V\)-values that map to \(u\).

Proof. For each \(v\) with \(\mathbb{P}(V=v)>0\), the definition gives \(\mathbb{E}[X\mathbf{1}_{\{V=v\}}]=\mathbb{P}(V=v)\mathbb{E}[X\mid V=v]\). The event \(\{U=u\}\) is the disjoint union of the events \(\{V=v\}\) for which \(g(v)=u\). Summing the preceding identity over those \(v\) gives \(\mathbb{E}[X\mathbf{1}_{\{U=u\}}]=\sum_{v:g(v)=u}\mathbb{P}(V=v)\mathbb{E}[X\mid V=v]\). Divide by \(\mathbb{P}(U=u)>0\), and use \(\mathbb{P}(V=v\mid U=u)=\mathbb{P}(V=v)/\mathbb{P}(U=u)\) for \(g(v)=u\). This proves the formula. For countably many values, the sums are legitimate because \(\mathbb{E}|X|<\infty\), so the corresponding expectations of absolute values are summable. Values \(u\) with zero probability do not affect an almost-sure identity. \(\square\)

Nested Conditioning with Densities

A parallel formula applies when the variables being conditioned on have densities. It is important to state the density assumptions precisely. In the result below, \((X,Z,W)\) has a joint density with respect to three-dimensional Lebesgue measure and \(X\) is integrable. The resulting conditional expectations are specified for almost every conditioning value; where a denominator density is zero, the corresponding conditional density need not be defined.

Proof. The integrability assumption gives \(\int |x|f_{X,Z,W}(x,z,w)\,dx\,dz\,dw<\infty\). Thus the integrals defining conditional means and their averages exist for almost every relevant conditioning value, and Fubini's theorem permits integrating in stages. For almost every \(z\) with \(f_Z(z)>0\), the right-hand side equals \(\int\int x f_{X\mid Z,W}(x\mid z,w) f_{W\mid Z}(w\mid z)\,dx\,dw\). Substituting the density ratios gives \(\int\int x [f_{X,Z,W}(x,z,w)/f_{Z,W}(z,w)][f_{Z,W}(z,w)/f_Z(z)]\,dx\,dw =\int\int x f_{X,Z,W}(x,z,w)/f_Z(z)\,dx\,dw\), where terms with zero marginal density are immaterial to the integral. The last expression is the conditional mean obtained from the marginal density \(f_{X,Z}(x,z)=\int f_{X,Z,W}(x,z,w)\,dw\), namely \(\int x f_{X,Z}(x,z)/f_Z(z)\,dx=\mathbb{E}[X\mid Z=z]\). This proves the identity for almost every \(z\). \(\square\)

Weights, Assumptions, and a Common Pitfall

At each stage, the more detailed conditional means must be averaged using the probabilities of their detailed outcomes conditional on the coarser information. Equal weighting is correct only when those conditional probabilities are equal. In the discrete example, an unweighted average of the means for \(a\) and \(b\) would give \((1+3)/2=2\) for \(U=0\). This is wrong: given \(U=0\), the probabilities are \(\tfrac14\) and \(\tfrac34\), so the correct calculation is \(\tfrac14(1)+\tfrac34(3)=\tfrac52\).

The density formula also has a real hypothesis: a joint density for the variables displayed in the formula. For example, if one instead defines \(X=Z+W^2\) deterministically, then \((Z,W,X)\) is supported on the surface \(x=z+w^2\) and has no joint density with respect to three-dimensional Lebesgue measure. The tower identity still holds, but that particular joint-density formula cannot be applied as stated. Conditional distributions that allow point masses can handle such cases; the density theorem above is deliberately restricted to the setting where its density ratios are valid.

In applications, nested conditioning reduces a difficult expectation to simpler calculations at successive levels of information. It is useful in prediction, latent-variable models and sequential analysis: first compute a conditional mean under detailed information, then average that mean over the uncertainty that remains at the coarser level. The essential checks are that the conditioning is genuinely nested, that the relevant conditional probabilities or densities are defined almost everywhere, and that the weights used in the averaging step are correct.