A Conditional-Law Proof of the Tower Property

Why Conditional Laws Give the Tower Property

When information arrives in stages, a conditional mean based on the finer information can be averaged over the possibilities left open by the coarser information. The average recovers the conditional mean based only on the coarser information. This is the tower property, viewed through the conditional laws that make each averaging step explicit.

Let $Y$ represent the information retained at the outer stage and let $Z$ represent additional information. Knowing both $Y$ and $Z$ is at least as informative as knowing $Y$ alone. The relevant calculation first averages the law of $X$ given $(Y,Z)$ over the conditional law of $Z$ given $Y$, then takes the mean of the resulting law of $X$ given $Y$.

Theorem: Suppose $X$ is integrable, so $E[|X|]<\infty$, and $Y,Z$ are random variables. Then $$ E\!\left[E[X\mid Y,Z]\mid Y\right]=E[X\mid Y]\quad\text{almost surely}. $$ In conditional-law terms, the law of $X$ given $Y=y$ is obtained by averaging the law of $X$ given $Y=y,Z=z$ over the conditional law of $Z$ given $Y=y$, for almost every $y$. Consequently, their means obey the displayed identity.

Conditional distributions can be chosen as probability kernels for real-valued random variables. Values of a conditional distribution or conditional mean at exceptional conditioning values need not be unique. Accordingly, the theorem asserts equality almost surely, which is the appropriate meaning of equality between conditional expectations.

The Discrete Averaging Argument

First suppose $X,Y,Z$ are discrete. For a value $y$ with $P(Y=y)>0$, conditional total probability gives, for each possible $x$,

P(X=x\mid Y=y) =\sum_z P(X=x\mid Y=y,Z=z)\,P(Z=z\mid Y=y),

where terms with $P(Z=z\mid Y=y)=0$ can be omitted. To get the conditional mean, multiply by $x$ and sum over $x$. If $E[|X|\mid Y=y]<\infty$, the resulting sums are absolutely convergent: the sum of the absolute values of the terms before rearrangement is at most

\sum_z P(Z=z\mid Y=y)\sum_x |x|P(X=x\mid Y=y,Z=z) =E[|X|\mid Y=y]<\infty.

Thus the order of summation can be changed, yielding

\begin{aligned} E[X\mid Y=y] &=\sum_x x\sum_z P(X=x\mid Y=y,Z=z)P(Z=z\mid Y=y)\\ &=\sum_z P(Z=z\mid Y=y)\sum_x xP(X=x\mid Y=y,Z=z)\\ &=\sum_z E[X\mid Y=y,Z=z]P(Z=z\mid Y=y). \end{aligned}

The last expression is precisely the conditional mean of the finer conditional mean when $Y=y$ is known. If $Y$ can take countably many values, $E[|X|]<\infty$ ensures the required conditional integrability for every $y$ with positive probability.

Worked Example: Averaging a Discrete Conditional Law

Suppose $Y$ is either A or B, and $Z$ is either 0 or 1. Given $Y=\mathrm{A}$, let $P(Z=0\mid Y=\mathrm{A})=0.25$ and $P(Z=1\mid Y=\mathrm{A})=0.75$. Let $X$ be binary, with $P(X=1\mid \mathrm{A},0)=0.8$ and $P(X=1\mid \mathrm{A},1)=0.2$. Find the law and mean of $X$ given $Y=\mathrm{A}$.

Average the conditional probability of $X=1$ over the two values of $Z$:

P(X=1\mid Y=\mathrm{A}) =(0.8)(0.25)+(0.2)(0.75) =0.20+0.15=0.35.

Since $X$ is binary, $P(X=0\mid Y=\mathrm{A})=1-0.35=0.65$. Therefore $E[X\mid Y=\mathrm{A}]=0.35$. Averaging the finer conditional means gives the same result:

E[X\mid Y=\mathrm{A}] =(0.8)(0.25)+(0.2)(0.75)=0.35.

The weights are the probabilities of the $Z$ values given $Y=\mathrm{A}$, not their probabilities given $X$ or their unconditional probabilities.

The Density and Integral Argument

Now suppose $(X,Y,Z)$ has a joint density. Write $f_Y(y)$ for the marginal density of $Y$, and use the usual conditional densities wherever the relevant denominators are positive. For almost every such $y$, the conditional joint density of $(X,Z)$ given $Y=y$ factors as

f_{X,Z\mid Y}(x,z\mid y) =f_{X\mid Y,Z}(x\mid y,z)f_{Z\mid Y}(z\mid y).

Marginalizing over $z$ gives the conditional density of $X$ given $Y=y$:

f_{X\mid Y}(x\mid y) =\int f_{X\mid Y,Z}(x\mid y,z)f_{Z\mid Y}(z\mid y)\,dz.

This is the continuous conditional-law mixture formula. To obtain the conditional mean, integrate $x$ against this density. When $E[|X|]<\infty$, the absolute integral is finite for almost every $y$; Tonelli's theorem applied to the nonnegative integrand $|x|f_{X\mid Y,Z}f_{Z\mid Y}$ justifies the order of integration, and Fubini's theorem then permits the corresponding rearrangement for $x$:

\begin{aligned} E[X\mid Y=y] &=\int x f_{X\mid Y}(x\mid y)\,dx\\ &=\int\!\left\{\int x f_{X\mid Y,Z}(x\mid y,z)\,dx\right\} f_{Z\mid Y}(z\mid y)\,dz\\ &=\int E[X\mid Y=y,Z=z]f_{Z\mid Y}(z\mid y)\,dz. \end{aligned}

The inner integral is the mean under the conditional law of $X$ given $Y=y,Z=z$. The outer integral averages that mean using the conditional density of $Z$ given $Y=y$. This proves the tower identity in the density setting, with all equalities understood for almost every $y$.

Worked Example: Two Nested Normal Laws

Let $Z\mid Y=y\sim N(y/2,1)$, let $\varepsilon\sim N(0,1)$ be independent of $(Y,Z)$, and define $X=Y+2Z+\varepsilon$. Find $E[X\mid Y,Z]$ and $E[X\mid Y]$, and verify the tower calculation.

Given $Y=y,Z=z$, only $\varepsilon$ remains random and has mean zero. Thus $X\mid Y=y,Z=z\sim N(y+2z,1)$, so

E[X\mid Y=y,Z=z]=y+2z.

Average this conditional mean over $Z\mid Y=y$. Since $E[Z\mid Y=y]=y/2$,

E[E[X\mid Y,Z]\mid Y=y] =\int (y+2z)f_{Z\mid Y}(z\mid y)\,dz =y+2(y/2)=2y.

Directly, write $Z=y/2+(Z-y/2)$ conditional on $Y=y$. Then $X=2y+2(Z-y/2)+\varepsilon$; both remaining terms have conditional mean zero. Hence $E[X\mid Y=y]=2y$, as the mixture calculation requires. In fact, conditional independence and the normal distributions give $X\mid Y=y\sim N(2y,5)$: the residual terms have variances $4$ and $1$, respectively.

Mixed Conditional Laws and the General Form

The same averaging principle applies when one conditional law is discrete and another is continuous. A sum is used over the discrete possibilities, and an integral over the continuous ones. More generally, for real-valued $X,Y,Z$, let $K_{y,z}$ be a conditional law of $X$ given $Y=y,Z=z$, and let $\nu_y$ be a conditional law of $Z$ given $Y=y$. The mixture law

M_y(A)=\int K_{y,z}(A)\,\nu_y(dz)

is a conditional law of $X$ given $Y=y$, for almost every $y$. Here $A$ is any measurable set. To see why, conditionally on $Y=y$, the joint law of $(X,Z)$ is obtained by first using $\nu_y$ for $Z$, then using $K_{y,z}$ for $X$. Integrating this joint law over $z$ leaves the marginal conditional law of $X$. Integrating $x$ against that mixture gives the tower identity; for integrable $X$, the absolute-integrability argument above justifies the integration.

Worked Example: A Continuous Condition and a Discrete Branch

Let $Y\sim\operatorname{Uniform}(0,1)$, and, given $Y=y$, let $Z$ be Bernoulli with success probability $y$. Given $Y=y,Z=0$, suppose $X$ is exponential with mean $y+1$; given $Y=y,Z=1$, suppose $X$ is exponential with mean $3y+2$. Find the conditional mean given $Y=y$, evaluate it at $y=1/2$, and check the unconditional mean.

The exponential distribution's mean is the specified conditional mean, so the finer conditional expectations are $y+1$ and $3y+2$. Since $P(Z=1\mid Y=y)=y$, the law-based averaging step gives

\begin{aligned} E[X\mid Y=y] &=(1-y)(y+1)+y(3y+2)\\ &=y+1-y^2-y+3y^2+2y\\ &=2y^2+2y+1. \end{aligned}

At $y=1/2$, the two branch means are $3/2$ and $7/2$, with weights $1/2$ and $1/2$. Their average is $(3/2+7/2)/2=5/2$, also given by $2(1/2)^2+2(1/2)+1=5/2$. Finally, averaging over the uniform $Y$ gives

E[X]=\int_0^1(2y^2+2y+1)\,dy =\frac{2}{3}+1+1=\frac{8}{3}.

The first average uses the discrete conditional law of $Z$ given $Y=y$; the final integral uses the continuous law of $Y$. Both are necessary to average over all the conditioning information.

Common Pitfall: Using the Wrong Averaging Weights

A conditional mean given $(Y,Z)$ is not generally averaged equally over possible $Z$ values. Nor should it be averaged using the unconditional distribution of $Z$. The required weights are $P(Z=z\mid Y=y)$ in the discrete case, or the conditional law of $Z$ given $Y=y$ in general. In the first example, equal weights would give $(0.8+0.2)/2=0.5$, which is not the correct conditional mean $0.35$. The reason is that the two $Z$ values are not equally likely given $Y=\mathrm{A}$.

A second caution is that conditional laws specified by densities are only determined almost everywhere with respect to the conditioning variables' distributions. A formula at a particular $y$ outside the relevant support, or on an exceptional set of probability zero, cannot be treated as a uniquely defined conditional expectation. The tower property is an almost-sure identity, not a pointwise promise for every conceivable conditioning value.

Key takeaway: To prove the tower property by conditional laws, first mix the finer conditional distribution over the conditional law of the added information. Then integrate $X$ against that mixture. Absolute integrability justifies the averaging, and the resulting conditional mean equals the one based on the coarser information.

Check Your Understanding

Use the relevant conditional law at each averaging step.

For discrete $X,Y,Z$, which probabilities weight $E[X\mid Y=y,Z=z]$ when calculating $E[X\mid Y=y]$?
Why is absolute integrability relevant when changing the order of the sums or integrals in the proof?
If $Z\mid Y=y$ has mean $y$ and $E[X\mid Y=y,Z=z]=2y+z$, what is $E[E[X\mid Y,Z]\mid Y=y]$?
In a continuous density argument, which conditional density is integrated out to recover the law of $X$ given $Y=y$?
Why does the tower identity assert equality almost surely instead of equality at every possible value of $Y$?