Tutorials › Probability Theory (Graduate) › The Tower Property for Conditional Laws

Home  /  Tutorials  /  Conditional Laws and Disintegration

Conditional Laws and Disintegration · Tutorial 263 of 946

The Tower Property for Conditional Laws

See why conditioning on a coarser sigma-algebra averages the finer conditional law against the conditional distribution of the finer state.

Advanced 11 min read

What You'll Learn

  • Define composition of probability kernels for nested conditioning.
  • State the assumptions under which the composed kernel is a conditional law.
  • Prove the conditional-law tower identity using the defining integral property.
  • Distinguish integration over the intermediate state from evaluating at its observed value.
  • Work through finite, uniform-mixture, and Gaussian examples.
  • Understand how versions and null sets affect the identity.

Why Conditional Laws Need a Tower Property

A conditional law given detailed information can be turned into a conditional law given coarser information, but the detailed state must be averaged out. The relevant average is taken with respect to the conditional distribution of that state given the coarser information.

This is the kernel form of the tower property. It is more informative than applying the tower property separately to conditional expectations: it expresses an entire conditional probability distribution as a composition of probability kernels.

Setup and Definition

Let \(X\), \(V\), and \(U\) be random elements taking values in standard Borel spaces \(E\), \(T\), and \(S\), respectively. Assume that \(\sigma(U)\subseteq\sigma(V)\), so the information carried by \(U\) is contained in the information carried by \(V\). Suppose \(K\) is a regular conditional law of \(X\) given \(V\), and \(Q\) is a regular conditional law of \(V\) given \(U\). Thus \(K\) is a probability kernel from \(T\) to \(E\), and \(Q\) is a probability kernel from \(S\) to \(T\).

Definition: The composition \(QK\) is the probability kernel from \(S\) to \(E\) defined, for \(u\in S\) and Borel \(A\subseteq E\), by
$$ (QK)(u,A)=\int_T K(v,A)\,Q(u,dv). $$
The variable \(v\) is the integration variable: the law \(Q(u,\cdot)\) averages the conditional law \(K(v,\cdot)\) over possible intermediate states.

For each \(A\), the function \(v\mapsto K(v,A)\) is measurable and bounded, so its integral against \(Q(u,\cdot)\) is defined. Kernel integration gives measurability in \(u\). Also, \((QK)(u,E)=1\), since \(K(v,E)=1\) and \(Q(u,\cdot)\) is a probability measure. If \(A_1,A_2,\ldots\) are pairwise disjoint Borel sets, countable additivity of \(K(v,\cdot)\) and monotone convergence give

$$ (QK)\left(u,\bigcup_{n=1}^{\infty}A_n\right) =\sum_{n=1}^{\infty}(QK)(u,A_n). $$

Thus the composition is itself a probability kernel.

The Tower Property for Conditional Laws

Theorem: Under the setup above, \(QK(U,\cdot)\) is a regular conditional law of \(X\) given \(U\). In particular, for every Borel set \(A\subseteq E\),
$$ \mathbb{P}(X\in A\mid \sigma(U)) =\int_T K(v,A)\,Q(U,dv) \quad\text{almost surely}. $$
Proof. Fix Borel sets \(A\subseteq E\) and \(B\subseteq S\). Since \(Q\) is a conditional law of \(V\) given \(U\), integration against \(Q(U,\cdot)\) gives conditional expectation given \(\sigma(U)\) for bounded measurable functions of \(V\). Apply this to \(v\mapsto K(v,A)\):
$$ \begin{aligned} \mathbb{E}\!\left[ \mathbf{1}_{\{U\in B\}}\int_T K(v,A)\,Q(U,dv) \right] &=\mathbb{E}\!\left[ \mathbf{1}_{\{U\in B\}}K(V,A) \right]. \end{aligned} $$
Because \(\sigma(U)\subseteq\sigma(V)\), the event \(\{U\in B\}\) belongs to \(\sigma(V)\). Since \(K(V,A)\) is a version of \(\mathbb{P}(X\in A\mid\sigma(V))\), the defining property of conditional expectation yields
$$ \begin{aligned} \mathbb{E}\!\left[ \mathbf{1}_{\{U\in B\}}K(V,A) \right] &=\mathbb{E}\!\left[ \mathbf{1}_{\{U\in B\}}\mathbf{1}_{\{X\in A\}} \right]. \end{aligned} $$
The composed kernel is measurable in \(U\), and the displayed equality holds for every Borel \(B\subseteq S\). These are precisely the defining integral identities for \(QK(U,A)\) to be a version of \(\mathbb{P}(X\in A\mid\sigma(U))\). This holds for each Borel \(A\), proving the claimed conditional-law identity. \(\square\)

The key point is that the intermediate state is integrated out. The formula does not evaluate \(K\) at the realized \(V\) and then leave that value unchanged. Such an expression would generally depend on the finer information \(\sigma(V)\), rather than only on \(\sigma(U)\).

Worked Example: A Finite Intermediate State

Worked Example: A Finite Intermediate State

Let \(U\in\{0,1\}\) have probabilities \(\mathbb{P}(U=0)=\mathbb{P}(U=1)=1/2\). Given \(U\), let \(V\in\{a,b\}\) have conditional probabilities

$$ \begin{aligned} &Q(0,\{a\})=0.6,\qquad Q(0,\{b\})=0.4,\\ &Q(1,\{a\})=0.2,\qquad Q(1,\{b\})=0.8. \end{aligned} $$

Given \(V\), let \(X\in\{0,1\}\) satisfy \(K(a,\{1\})=0.1\) and \(K(b,\{1\})=0.7\). The complementary probabilities are \(K(a,\{0\})=0.9\) and \(K(b,\{0\})=0.3\), so each \(K(v,\cdot)\) is a probability distribution.

For \(U=0\), composition gives

$$ (QK)(0,\{1\}) =K(a,\{1\})Q(0,\{a\})+K(b,\{1\})Q(0,\{b\}) =(0.1)(0.6)+(0.7)(0.4)=0.34. $$

For \(U=1\), the corresponding probability is

$$ (QK)(1,\{1\}) =(0.1)(0.2)+(0.7)(0.8)=0.02+0.56=0.58. $$

Thus the conditional laws of \(X\) given \(U=0\) and \(U=1\) assign probabilities \(0.34\) and \(0.58\), respectively, to \(\{1\}\). The formula averages over both possible intermediate states, weighted by their probabilities given \(U\).

Worked Example: A Uniform Mixture of Bernoulli Laws

Worked Example: A Uniform Mixture of Bernoulli Laws

Let \(U\) be constant, let \(V\) be uniform on \([0,1]\), and, conditional on \(V=v\), let \(X\in\{0,1\}\) have a Bernoulli distribution with success probability \(v\). Here \(Q(U,\cdot)\) is the uniform distribution on \([0,1]\), and \(K(v,\{1\})=v\).

The composition gives the unconditional law of \(X\):

$$ (QK)(U,\{1\}) =\int_0^1 v\,dv =\left[\frac{v^2}{2}\right]_{0}^{1} =\frac12. $$

For the other value,

$$ (QK)(U,\{0\}) =\int_0^1 (1-v)\,dv =1-\frac12 =\frac12. $$

The resulting mixture is therefore Bernoulli with success probability \(1/2\). It is not obtained by choosing a particular value of \(V\); it is obtained by averaging the conditional Bernoulli laws over the distribution of \(V\).

Worked Example: A Gaussian Two-Stage Model

Worked Example: A Gaussian Two-Stage Model

Let \(Z_1,Z_2\) be independent standard normal random variables, independent of a real-valued random variable \(U\). Set \(V=U+Z_1\) and \(X=V+2Z_2\). Given \(U=u\), the intermediate variable \(V\) has law \(N(u,1)\). Given \(V=v\), the variable \(X\) has law \(N(v,4)\).

The composition density at \(x\), conditional on \(U=u\), is the integral of the \(N(v,4)\) density against the \(N(u,1)\) law of \(v\):

$$ \int_{\mathbb{R}} \frac{1}{\sqrt{2\pi}}e^{-(v-u)^2/2} \frac{1}{\sqrt{8\pi}}e^{-(x-v)^2/8}\,dv. $$

Complete the square using the identity

$$ \frac{(v-u)^2}{2}+\frac{(x-v)^2}{8} = \frac{5}{8}\left(v-\frac{4u+x}{5}\right)^2 +\frac{(x-u)^2}{10}. $$

The integral therefore equals

$$ \begin{aligned} &\frac{1}{4\pi}e^{-(x-u)^2/10} \int_{\mathbb{R}} e^{-\frac58(v-(4u+x)/5)^2}\,dv\\ &\qquad= \frac{1}{4\pi}e^{-(x-u)^2/10}\sqrt{\frac{8\pi}{5}} = \frac{1}{\sqrt{10\pi}}e^{-(x-u)^2/10}. \end{aligned} $$

This is the density of \(N(u,5)\), since a normal distribution with variance \(5\) has density \((10\pi)^{-1/2}\exp(-(x-u)^2/10)\). The composition thus gives the conditional law of \(X\) given \(U=u\).

Conditional Laws Along Nested Sigma-Algebras

The random-element formulation describes a common way to express nested conditioning. If \(\mathcal{G}\subseteq\mathcal{H}\), and the information in \(\mathcal{H}\) is represented by a standard Borel-valued random element \(V\), while the information in \(\mathcal{G}\) is represented by \(U\), then the theorem says to average the conditional law given \(\mathcal{H}\) against the conditional law of \(V\) given \(U\).

At the level of events, the conclusion is

$$ \mathbb{P}(X\in A\mid\mathcal{G}) = \int_T K(v,A)\,Q(U,dv) \quad\text{almost surely}. $$

This is consistent with the conditional-expectation tower property: for each fixed \(A\), the indicator \(\mathbf{1}_{\{X\in A\}}\) is first conditioned on the finer information and then on the coarser information. The kernel identity retains the whole conditional distribution, rather than only one conditional probability or one expectation.

Versions and a Common Pitfall

Conditional laws are determined only up to null sets. The identity is therefore an almost-sure statement, not a pointwise identity that must hold for every possible value of \(u\). Changes to \(K\) on a \(V\)-null set do not affect the composed law almost surely: if \(N\subseteq T\) has \(\mathbb{P}(V\in N)=0\), then

$$ \mathbb{E}[Q(U,N)]=\mathbb{P}(V\in N)=0, $$

and \(Q(U,N)=0\) almost surely. Standard Borel spaces also provide the regular conditional laws and measurable structure used in this kernel formulation.

A frequent error is to put the observed random element \(V\) inside the integrand instead of the integration variable \(v\). Because \(Q(U,\cdot)\) has total mass one, that incorrect expression reduces to \(K(V,A)\). It is generally a function of the finer information and need not equal the conditional probability given \(U\). In the finite example, on \(\{U=0,V=a\}\), it is \(0.1\), whereas the correct coarser conditional probability is \(0.34\). The distinction between \(K(V,A)\) and \(\int K(v,A)Q(U,dv)\) is essential.

Why the Kernel Tower Property Matters

The result is useful whenever a model has layers of information: a latent state is partly observed, a process is observed at progressively finer levels, or a conditional distribution is specified at an intermediate stage. It permits a conditional law to be moved to a coarser level without discarding its distributional meaning.

The operation is composition of kernels, not substitution of a random state into a conditional law. Keeping that distinction explicit makes the conditioning level clear, ensures the result is measurable with respect to the coarser information, and provides the correct formula for conditional distributions.

Check Your Understanding

Answer these questions by referring to the definitions and calculations above.

  1. In the composition \((QK)(u,A)\), what does the integration variable \(v\) represent?
  2. Which assumption ensures that \(\mathbf{1}_{\{U\in B\}}\) can be taken outside conditional expectation given \(\sigma(V)\)?
  3. In the finite example, calculate \((QK)(0,\{0\})\) and verify that it complements \((QK)(0,\{1\})\).
  4. Why is \(K(V,A)\) generally not the conditional law of \(X\) given \(U\)?
  5. In the Gaussian example, what are the mean and variance of the composed conditional distribution given \(U=u\)?

Previous

Next

Back to Tutorials
Clinical Biostats