Proving the Tower Property

Why a Second Conditioning Step Can Simplify

Suppose \(Y\) describes a broad group and \(Z\) gives additional detail within that group. Knowing both variables can make the conditional mean of a quantity \(X\) easier to calculate. But if the desired answer is conditional only on \(Y\), the extra detail must be averaged out using the distribution of \(Z\) given \(Y\). The result is the same conditional mean that would be obtained by conditioning directly on \(Y\).

This identity is called the tower property of conditional expectation. It is useful when the joint information \((Y,Z)\) is easier to analyze than \(Y\) alone, provided the final answer is required at the less detailed level of conditioning.

Theorem: If \(E[|X|]<\infty\), then \[ E\!\left[E[X\mid Y,Z]\mid Y\right]=E[X\mid Y] \] with probability 1. The inner conditional expectation uses both \(Y\) and \(Z\); the outer conditional expectation averages that result over the conditional distribution of \(Z\) given \(Y\).

The statement is an equality of random variables, meaning that both sides agree except possibly on events of probability zero. In a discrete model, the outer conditional expectation is a weighted sum. In a continuous model with conditional densities, it is an integral. In either case, the weights or density must describe \(Z\) conditional on the known value of \(Y\), not the marginal distribution of \(Z\).

Deriving the Property by Conditional Averaging

First consider discrete variables. For values \(y\) with \(P(Y=y)>0\), and values \(z\) for which the conditional probabilities are defined, write \(m(y,z)=E[X\mid Y=y,Z=z]\). The conditional mean of \(m(Y,Z)\) given \(Y=y\) is a sum over the possible values of \(Z\):

E[E[X\mid Y,Z]\mid Y=y] =\sum_z m(y,z)P(Z=z\mid Y=y).

To see why this is \(E[X\mid Y=y]\), expand each conditional mean as an average of \(X\) within the corresponding \((y,z)\) group. For each \(z\) with positive conditional probability,

m(y,z)=\sum_x xP(X=x\mid Y=y,Z=z).

Substituting this into the first sum and using the multiplication rule for conditional probabilities gives

\begin{aligned} \sum_z m(y,z)P(Z=z\mid Y=y) &=\sum_z\sum_x xP(X=x\mid Y=y,Z=z)P(Z=z\mid Y=y)\\ &=\sum_z\sum_x xP(X=x,Z=z\mid Y=y)\\ &=\sum_x xP(X=x\mid Y=y)\\ &=E[X\mid Y=y]. \end{aligned}

The third equality sums over all possible \(z\), so the joint conditional probabilities for \(X\) and \(Z\) reduce to the conditional probabilities for \(X\). This establishes the identity in the discrete case.

For continuous variables, suppose the relevant conditional densities exist and the integrals are finite. Let \(f_{Z\mid Y}(z\mid y)\) be the conditional density of \(Z\) given \(Y=y\). Then the outer conditional expectation is an integral with respect to that density. Expanding the inner conditional mean gives

\begin{aligned} E[E[X\mid Y,Z]\mid Y=y] &=\int E[X\mid Y=y,Z=z]f_{Z\mid Y}(z\mid y)\,dz\\ &=\int\!\int x f_{X\mid Y,Z}(x\mid y,z)\,dx\, f_{Z\mid Y}(z\mid y)\,dz\\ &=\int\!\int x f_{X,Z\mid Y}(x,z\mid y)\,dx\,dz\\ &=\int x f_{X\mid Y}(x\mid y)\,dx\\ &=E[X\mid Y=y]. \end{aligned}

The second-to-third line uses the conditional density identity \(f_{X,Z\mid Y}(x,z\mid y)=f_{X\mid Y,Z}(x\mid y,z)f_{Z\mid Y}(z\mid y)\). Integrating out \(z\) leaves the conditional density of \(X\) given \(Y=y\). Thus, the two conditioning steps simply average over the intermediate variable using the correct conditional distribution.

Worked Examples

Worked Example: A Finite Model with Unequal Conditional Weights

Let \(Y\) take values 0 and 1, with \(P(Y=0)=0.4\) and \(P(Y=1)=0.6\). Given \(Y=0\), let \(P(Z=a\mid Y=0)=0.25\); given \(Y=1\), let \(P(Z=a\mid Y=1)=0.50\). In each case, \(Z\) has only the values \(a\) and \(b\). Suppose \(X\) is determined by \(Y,Z\), with values \(X=2\) for \((0,a)\), \(X=6\) for \((0,b)\), \(X=3\) for \((1,a)\), and \(X=9\) for \((1,b)\). Find \(E[X\mid Y=0]\) and verify it using the tower property.

Method. Since \(X\) is determined by \(Y,Z\), the inner conditional mean \(E[X\mid Y,Z]\) is the specified value of \(X\) in each \((Y,Z)\) case. Then average those values using \(P(Z\mid Y=0)\).

When \(Y=0\), the conditional probability of \(Z=b\) is \(1-0.25=0.75\). Therefore,

E[E[X\mid Y,Z]\mid Y=0] =(2)(0.25)+(6)(0.75)=0.50+4.50=5.

Directly, conditional on \(Y=0\), \(X=2\) with probability 0.25 and \(X=6\) with probability 0.75. Hence

E[X\mid Y=0]=(2)(0.25)+(6)(0.75)=5.

The two calculations agree. As a check on the conditional weighting, the marginal probability of \(Z=a\) is \((0.4)(0.25)+(0.6)(0.50)=0.40\), which is not \(P(Z=a\mid Y=0)=0.25\). Using the marginal weights would give \((2)(0.40)+(6)(0.60)=4.4\), not 5. The mistake is averaging over all cases of \(Z\), rather than over \(Z\) within the group \(Y=0\).

Worked Example: Integrating Over a Continuous Conditional Distribution

Let \(Y\) be uniform on \((0,1)\). Given \(Y=y\), let \(Z\) be uniform on \((0,y)\). Let \(\varepsilon\) be independent of \((Y,Z)\), with \(\varepsilon\sim N(0,1)\), and define \(X=Y+2Z+\varepsilon\). Find \(E[X\mid Y,Z]\), then use the tower property to find \(E[X\mid Y=y]\).

Inner conditional mean. Once \(Y=y\) and \(Z=z\) are known, the terms \(y+2z\) are fixed. The independent error has mean zero, so

E[X\mid Y=y,Z=z]=y+2z+E[\varepsilon]=y+2z.

Average over \(Z\) given \(Y=y\). For \(0<z<y\), the conditional density is \(f_{Z\mid Y}(z\mid y)=1/y\). Thus,

\begin{aligned} E[E[X\mid Y,Z]\mid Y=y] &=\int_0^y (y+2z)\frac{1}{y}\,dz\\ &=\frac{1}{y}\left(\int_0^y y\,dz+\int_0^y 2z\,dz\right)\\ &=\frac{1}{y}\left(y^2+y^2\right)=2y. \end{aligned}

Direct check. Given \(Y=y\), the conditional mean of \(Z\) is \[ E[Z\mid Y=y]=\int_0^y z\frac{1}{y}\,dz=\frac{y}{2}. \] Because \(E[\varepsilon\mid Y=y]=0\), direct averaging in \(X=y+2Z+\varepsilon\) gives \[ E[X\mid Y=y]=y+2\left(\frac{y}{2}\right)+0=2y. \] This matches the nested calculation for every \(y\) in the support of \(Y\). The conditional density integrates to \(\int_0^y (1/y)\,dz=1\), as required.

Worked Example: Continuous Grouping with a Discrete Intermediate Variable

Let \(Y\) be uniform on \((0,2)\). Given \(Y=y\), let \(Z\) be Bernoulli with success probability \(y/2\), so \(P(Z=1\mid Y=y)=y/2\) and \(P(Z=0\mid Y=y)=1-y/2\). Let \(\varepsilon\) be independent of \((Y,Z)\), with mean zero, and set \(X=5+2Y+3Z+\varepsilon\). Find \(E[X\mid Y=y]\) by conditioning first on both \(Y\) and \(Z\), then verify it directly.

Inner conditional mean. Given \(Y=y,Z=z\), the only random component in \(X\) is \(\varepsilon\), whose conditional mean is zero. Therefore,

E[X\mid Y=y,Z=z]=5+2y+3z.

Outer conditional mean. Average over the two possible values of \(Z\), using their probabilities given \(Y=y\):

\begin{aligned} E[E[X\mid Y,Z]\mid Y=y] &=(5+2y+3)(y/2)+(5+2y)(1-y/2)\\ &=(8+2y)(y/2)+(5+2y)(1-y/2)\\ &=4y+y^2+5+2y-\frac{5y}{2}-y^2\\ &=5+\frac{7y}{2}. \end{aligned}

Direct check. Since \(E[Z\mid Y=y]=y/2\), taking the conditional mean of the defining equation for \(X\) gives \[ E[X\mid Y=y]=5+2y+3\left(\frac{y}{2}\right)+0 =5+\frac{7y}{2}. \] The conditional success probability lies between 0 and 1 for \(0<y<2\), and the two nested calculations agree throughout that interval.

Conditions, Interpretation, and Common Traps

The integrability condition \(E[|X|]<\infty\) ensures the conditional means in the theorem are well-defined and finite in the usual setting. The conditioning information must be nested: the inner mean uses both \(Y\) and \(Z\), while the outer mean retains \(Y\) and averages out \(Z\). The statement does not say that conditioning on \(Y,Z\) has the same result as conditioning on \(Y\); the equality appears only after the inner conditional mean is itself averaged conditional on \(Y\).

Wrong weights: When averaging over \(Z\) after fixing \(Y=y\), use \(P(Z=z\mid Y=y)\) or \(f_{Z\mid Y}(z\mid y)\). Marginal probabilities or densities generally give a different answer.
Dropping the outer conditioning: \(E[X\mid Y]\) is generally a function of \(Y\), not a single constant. The tower property does not remove \(Y\).
Skipping an intermediate variable: The inner mean \(E[X\mid Y,Z]\) can depend on both \(Y\) and \(Z\). Average its full value over the conditional distribution of \(Z\) given \(Y\).
Ignoring the model's support: Conditional probabilities and densities must be valid for the values considered. In a discrete model, conditioning on an event of probability zero does not define an ordinary conditional mean by a ratio of probabilities.

The practical advantage is computational: a complicated conditional mean can sometimes be found by first fixing more information, calculating a simpler mean, and then averaging over the information that is no longer wanted. The key is to preserve the conditioning structure at every step.

Key takeaway: Under integrability, conditioning \(E[X\mid Y,Z]\) again on \(Y\) averages out \(Z\) according to the conditional distribution of \(Z\) given \(Y\), yielding \(E[X\mid Y]\).

Check Your Understanding

For each problem, first identify the conditional distribution used in the outer averaging step.

\(Y\) takes values 0 and 1. Given \(Y=0\), \(P(Z=1\mid Y=0)=0.2\); given \(Y=1\), \(P(Z=1\mid Y=1)=0.8\). Suppose \(E[X\mid Y=y,Z=z]=2+y+5z\). Find \(E[X\mid Y=0]\) and \(E[X\mid Y=1]\). (Answer: 3 and 7, respectively.)
Given \(Y=y\), \(Z\) is uniform on \((0,y)\), where \(0<y<3\). If \(X=4+Y-Z+\varepsilon\), with \(\varepsilon\) independent of \((Y,Z)\) and mean zero, find \(E[X\mid Y=y]\), then evaluate it at \(y=2\). (Answer: \(4+y/2\); at \(y=2\), 5.)
Let \(Y\) be uniform on \((0,1)\), with \(P(Z=1\mid Y=y)=y^2\). Let \(X=1+2Y+4Z+\varepsilon\), where \(\varepsilon\) is independent and has mean zero. Find \(E[X\mid Y=y]\), then evaluate it at \(y=0.5\). (Answer: \(1+2y+4y^2\); at \(y=0.5\), 3.)
\(Y\) takes values \(A,B\), with \(P(Z=1\mid Y=A)=0.2\) and \(P(Z=1\mid Y=B)=0.75\). Suppose \(E[X\mid A,0]=1\), \(E[X\mid A,1]=11\), \(E[X\mid B,0]=3\), and \(E[X\mid B,1]=7\). Find \(E[X\mid Y=A]\) and \(E[X\mid Y=B]\). (Answer: 3 and 6, respectively.)

What You'll Learn