Iterated Conditioning and the Tower Property

Why Average in Stages?

A conditional mean can summarize a fine-grained description of a population, while a coarser description may be easier to report or use. For instance, a study might first distinguish several clinical states and then combine those states into broader risk groups. The tower property says that if the broad groups are formed by combining the fine groups, averaging the fine conditional means within each broad group gives the conditional mean for that broad group. Averaging once more across broad groups recovers the unconditional mean.

The weights matter at each stage. A fine group contributes to its parent group according to its probability within that parent; a parent group contributes to the overall mean according to its probability in the whole population. These are not generally equal weights.

Nested Finite Partitions

A finite partition of the sample space is a finite collection of pairwise disjoint events whose union is the whole sample space. Its members are called atoms. A partition $\mathcal P$ is finer than a partition $\mathcal Q$ if every atom of $\mathcal P$ is contained in an atom of $\mathcal Q$. Equivalently, each coarse atom in $\mathcal Q$ is a union of fine atoms in $\mathcal P$.

Definition: Let $\mathcal P=\{A_1,\ldots,A_m\}$ be a finite partition, and let $X$ be integrable, meaning $E[|X|]<\infty$. The conditional expectation of $X$ given $\mathcal P$, written $E[X\mid\mathcal P]$, is constant on each positive-probability atom $A_i$, with value $$ E[X\mid A_i]=\frac{E[X\mathbf 1_{A_i}]}{P(A_i)}. $$ Its value on a zero-probability atom can be chosen arbitrarily, since that choice does not affect any expectation or probability statement.

Here $\mathbf 1_{A_i}$ is the indicator of event $A_i$. This definition is the finite-partition form of conditional expectation. A conditioning variable $Z$ with finitely many possible values gives the partition into events $\{Z=z\}$; thus $E[X\mid Z]$ is constant on each such event and takes the corresponding conditional-mean value.

Suppose a finite-valued variable $Y$ is a function of another finite-valued variable $Z$, say $Y=h(Z)$. Observing $Z$ identifies a particular fine group; observing only $Y$ identifies a coarse group that combines all $Z$-values mapped to it. This is nested conditioning: $Z$ gives at least as detailed a partition as $Y$.

The Tower Property

Theorem: Let $X$ be integrable, and let $\mathcal P$ and $\mathcal Q$ be finite partitions such that $\mathcal P$ is finer than $\mathcal Q$. Then, up to choices on probability-zero events, $$ E\!\left[E[X\mid\mathcal P]\mid\mathcal Q\right]=E[X\mid\mathcal Q]. $$ In particular, taking $\mathcal Q$ to be the one-atom partition $\{\Omega\}$ gives $$ E\!\left[E[X\mid\mathcal P]\right]=E[X]. $$

The expression on the left first replaces $X$ by its mean within each fine atom. It then averages those fine-atom means within each coarse atom. The theorem states that this two-step calculation has the same value on each coarse atom as calculating the mean of $X$ there directly.

To prove the result, fix a positive-probability coarse atom $B\in\mathcal Q$. Because $\mathcal P$ is finer, $B$ is a union of fine atoms $A_1,\ldots,A_k$. By the definition of conditional expectation on a finite partition,

\begin{aligned} E\!\left[E[X\mid\mathcal P]\mathbf 1_B\right] &=\sum_{j=1}^k E\!\left[E[X\mid\mathcal P]\mathbf 1_{A_j}\right]\\ &=\sum_{j=1}^k E[X\mathbf 1_{A_j}]\\ &=E[X\mathbf 1_B]. \end{aligned}

The second equality holds because on each positive-probability $A_j$, $E[X\mid\mathcal P]$ is the constant $E[X\mathbf 1_{A_j}]/P(A_j)$; multiplying that constant by $P(A_j)$ gives $E[X\mathbf 1_{A_j}]$. Zero-probability fine atoms contribute zero. Dividing both sides by $P(B)>0$ shows that the mean of $E[X\mid\mathcal P]$ on $B$ is $E[X\mathbf 1_B]/P(B)$, exactly the value of $E[X\mid\mathcal Q]$ on $B$. This holds on every positive-probability coarse atom, proving the first identity. For the one-atom partition, the conditional mean on $\Omega$ is $E[X]$, proving the second identity. $\square$

Calculating by Stages

When the partitions are represented by finite-valued variables $Z$ and $Y=h(Z)$, the tower property reads

E[E[X\mid Z]\mid Y]=E[X\mid Y], \qquad E[E[X\mid Z]]=E[X].

For a particular value $y$ with $P(Y=y)>0$, the conditional mean can be calculated by weighting the fine conditional means over all $z$-values for which $h(z)=y$:

E[X\mid Y=y] = \sum_{z:\,h(z)=y} E[X\mid Z=z]P(Z=z\mid Y=y).

The weights are conditional probabilities within the selected coarse group. For the unconditional mean, the corresponding weights are the probabilities of the fine groups in the whole population:

E[X]=\sum_z E[X\mid Z=z]P(Z=z).

Worked Example: From Fine Clinical States to an Overall Mean

A clinic classifies patients first by site and then by severity. Site North has probability $0.6$, and Site South has probability $0.4$. Within North, $70\%$ of patients are mild and $30\%$ are severe. Their mean costs are $20$ and $80$ cost units, respectively. Within South, $25\%$ are mild and $75\%$ are severe, with mean costs $30$ and $100$. Find the mean cost at each site and the overall mean cost.

Let $X$ be cost, $S$ be site, and $Z$ be the joint site-and-severity category. The fine categories are nested within the two site categories. Within North, average the severity-specific means using the severity probabilities conditional on North:

E[X\mid S=\text{North}] =0.70(20)+0.30(80) =14+24 =38.

Within South, use its own severity proportions:

E[X\mid S=\text{South}] =0.25(30)+0.75(100) =7.5+75 =82.5.

Now average the site means using the site probabilities:

$$ E[X] =0.60(38)+0.40(82.5) =22.8+33 =55.8. $$

As a check, the four fine categories have unconditional probabilities $0.6(0.7)=0.42$, $0.6(0.3)=0.18$, $0.4(0.25)=0.10$, and $0.4(0.75)=0.30$. Their direct weighted average is

$$ 0.42(20)+0.18(80)+0.10(30)+0.30(100) =8.4+14.4+3+30 =55.8. $$

Both routes agree. The staged route is especially useful when the subgroup summaries are available but the full fine-category table is not.

Worked Example: A Coarse Conditional Mean from Fine Categories

A finite-valued variable $Z$ takes values $a,b,c$ with probabilities $0.2,0.3,0.5$. The conditional means of an integrable outcome $X$ are $E[X\mid Z=a]=10$, $E[X\mid Z=b]=40$, and $E[X\mid Z=c]=70$. Define $Y=0$ for $Z=a$ or $Z=b$, and $Y=1$ for $Z=c$. Find $E[X\mid Y=0]$, $E[X\mid Y=1]$, and $E[X]$.

The coarse event $Y=0$ combines $Z=a$ and $Z=b$, and has probability $0.2+0.3=0.5$. The conditional weights within that event are

P(Z=a\mid Y=0)=\frac{0.2}{0.5}=0.4, \qquad P(Z=b\mid Y=0)=\frac{0.3}{0.5}=0.6.

Therefore,

E[X\mid Y=0]=0.4(10)+0.6(40)=4+24=28.

Since $Y=1$ means $Z=c$, $E[X\mid Y=1]=70$. The unconditional mean can now be found from the coarse partition:

E[X]=P(Y=0)E[X\mid Y=0]+P(Y=1)E[X\mid Y=1] =0.5(28)+0.5(70)=14+35=49.

Direct calculation on the fine partition confirms the result: $0.2(10)+0.3(40)+0.5(70)=2+12+35=49$.

Worked Example: Three Levels of Conditioning

A product is produced on one of two lines. Line A produces $75\%$ of units; Line B produces $25\%$. On Line A, $20\%$ of units require rework, with mean rework time 12 minutes, while the other units have mean rework time 2 minutes. On Line B, $60\%$ require rework, with mean time 15 minutes, and the other units have mean time 3 minutes. Find the overall mean rework time by conditioning first on line, then on rework status.

Let $X$ be rework time, $L$ be production line, and $R$ be rework status. The joint categories $(L,R)$ form the finest partition; the line categories form a coarser partition. First average within each line:

E[X\mid L=A]=0.20(12)+0.80(2)=2.4+1.6=4,

and

E[X\mid L=B]=0.60(15)+0.40(3)=9+1.2=10.2.

Then average the line-specific means over production:

E[X]=0.75(4)+0.25(10.2)=3+2.55=5.55\text{ minutes}.

The fine-category check uses unconditional probabilities: $0.75(0.20)=0.15$ for Line A with rework, $0.75(0.80)=0.60$ for Line A without rework, $0.25(0.60)=0.15$ for Line B with rework, and $0.25(0.40)=0.10$ for Line B without rework. Thus $0.15(12)+0.60(2)+0.15(15)+0.10(3)=1.8+1.2+2.25+0.3=5.55$.

Common Pitfall: Using the Wrong Weights

A frequent error is to average subgroup conditional means equally, even when the subgroup probabilities differ. In the clinic example, taking the simple average of the two site means would give $(38+82.5)/2=60.25$, not $55.8$. That calculation would be correct only if the two sites had equal probability. The correct overall mean uses $0.6$ and $0.4$, the probabilities of the sites.

There is a second weighting distinction when finding a coarse conditional mean. To calculate $E[X\mid Y=0]$ in the three-category example, the weights were $P(Z=z\mid Y=0)$, not the unconditional $P(Z=z)$. Conditional probabilities ensure that the weights for the selected coarse group sum to one.

Key takeaway: For nested conditioning, average fine conditional means within each coarse group using conditional probabilities inside that group. Then, to obtain an unconditional mean, average the coarse conditional means using the coarse-group probabilities.

Check Your Understanding

Work through these questions using the nesting structure and weights carefully.

A partition $\mathcal P$ is finer than $\mathcal Q$. What does this say about how the atoms of $\mathcal P$ and $\mathcal Q$ are related?
An outcome has conditional means 5 and 17 in two fine groups. Within a particular coarse group, their probabilities are $0.25$ and $0.75$. Find the conditional mean for that coarse group.
If $Y=h(Z)$ for finite-valued variables, explain why conditioning on $Z$ gives a partition at least as fine as conditioning on $Y$.
Why can the equal-weight average of two conditional means fail to equal the unconditional mean?
For a positive-probability coarse event $B$ that is a union of fine atoms, identify the quantity that must be divided by $P(B)$ to obtain $E[X\mid B]$.