Tutorials › Probability Theory (Graduate) › Associativity of Iterated Disintegration

Home  /  Tutorials  /  Conditional Laws and Disintegration

Conditional Laws and Disintegration · Tutorial 264 of 946

Associativity of Iterated Disintegration

Learn how to compose successive disintegrations, prove the resulting kernel is a conditional law, and understand why the equality is only guaranteed almost surely.

Advanced 10 min read

What You'll Learn

  • Define the conditional kernels used in a two-stage disintegration
  • Construct the composed kernel on a product space and verify its measurability
  • Prove that the composition is a version of the direct conditional law
  • Interpret equality of conditional kernels as equality of measures outside one null set
  • Work through finite, continuous, and null-set examples

Why Associativity Matters

When a joint law has several coordinates, it is often useful to condition in stages. One may first describe \(X\) given \((Y,Z)\), then describe \(Y\) given \(Z\). The resulting two-stage procedure should recover the conditional law of the pair \((X,Y)\) given \(Z\).

The important point is that the two stages must be composed by integrating over the intermediate coordinate \(Y\). They do not generally reduce to a pointwise identity between conditional distributions. The precise statement is an almost-sure equality between probability kernels.

Setup and Notation

Let \(S,T,U\) be standard Borel spaces, and let \((X,Y,Z)\) be a random element of \(S\times T\times U\). Write \(\mu_{YZ}\) for the law of \((Y,Z)\) and \(\mu_Z\) for the law of \(Z\). Suppose \(K\) is a regular conditional law of \(X\) given \((Y,Z)\), and \(L\) is a regular conditional law of \(Y\) given \(Z\). Thus \(K((y,z),dx)\) is a probability measure on \(S\), and \(L(z,dy)\) is a probability measure on \(T\).

For example, the defining disintegration identity for \(K\) is that for all Borel sets \(B\subseteq S\) and \(D\subseteq T\times U\),

$$ \mathbb{P}(X\in B,\ (Y,Z)\in D) = \int_D K((y,z),B)\,\mu_{YZ}(d(y,z)). $$

The corresponding identity for \(L\) disintegrates \(\mu_{YZ}\) over \(z\): for Borel \(D\subseteq T\) and \(C\subseteq U\),

$$ \mathbb{P}(Y\in D,\ Z\in C) = \int_C L(z,D)\,\mu_Z(dz). $$

These identities express the meaning of the two conditional laws. They also make clear that their values on exceptional conditioning points are not determined by the joint law.

Definition: The iterated kernel \(H\) on \(S\times T\), conditional on \(z\in U\), is defined by $$ H(z,A)=\int_T K((y,z),A_y)\,L(z,dy), \qquad A_y=\{x\in S:(x,y)\in A\}, $$ for every Borel set \(A\subseteq S\times T\). It first samples \(y\) according to \(L(z,\cdot)\), then uses \(K((y,z),\cdot)\) for \(x\).

The Composition Is a Probability Kernel

The formula for \(H\) must define both a probability measure in \(A\) for each fixed \(z\) and a measurable function of \(z\) for each fixed \(A\). For a rectangle \(A=B\times D\), it reads

$$ H(z,B\times D) = \int_D K((y,z),B)\,L(z,dy). $$

For fixed \(z\), this is the usual composition of probability kernels and defines a probability measure on rectangles. More generally, the kernel integration theorem gives a measure on the product space: equivalently, one integrates the probability measure \(K((y,z),dx)\otimes\delta_y(dy)\) against \(L(z,dy)\). Here \(\delta_y\) denotes unit mass at \(y\). For fixed Borel \(A\), the map \((y,z)\mapsto K((y,z),A_y)\) is measurable, and integration against \(L(z,dy)\) makes \(z\mapsto H(z,A)\) measurable. These facts follow first for rectangles and then for all Borel \(A\) by the monotone-class theorem. Thus \(H\) is a probability kernel from \(U\) to \(S\times T\).

Associativity Theorem

Theorem: Let \(J(z,\cdot)\) be any regular conditional law of \((X,Y)\) given \(Z=z\). Under the setup above, the iterated kernel \(H\) is also a regular conditional law of \((X,Y)\) given \(Z\). Consequently, there is a \(\mu_Z\)-null Borel set \(N\subseteq U\) such that $$ H(z,\cdot)=J(z,\cdot) \qquad\text{for every }z\in U\setminus N. $$

Proof. First take a rectangle \(A=B\times D\), where \(B\subseteq S\) and \(D\subseteq T\) are Borel, and take a Borel set \(C\subseteq U\). By the conditional-law identity for \(K\), followed by the disintegration identity for \(L\),

$$ \begin{aligned} \mathbb{P}(X\in B,\ Y\in D,\ Z\in C) &=\int_{D\times C}K((y,z),B)\,\mu_{YZ}(d(y,z))\\ &=\int_C\int_D K((y,z),B)\,L(z,dy)\,\mu_Z(dz)\\ &=\int_C H(z,B\times D)\,\mu_Z(dz). \end{aligned} $$

This is the defining conditional-law identity for \(H\) on rectangles. To extend it to every Borel \(A\subseteq S\times T\), consider, for fixed \(C\), the two finite measures on \(S\times T\) given by \(\mathbb{P}((X,Y)\in\cdot,\ Z\in C)\) and \(\int_C H(z,\cdot)\,\mu_Z(dz)\). They agree on rectangles. Rectangles form a generating \(\pi\)-system, so the uniqueness theorem for finite measures implies that they agree on every Borel \(A\). Therefore \(H\) is a regular conditional law of \((X,Y)\) given \(Z\).

It remains to express uniqueness as equality of measures off one common null set. Since \(S\times T\) is standard Borel, its Borel sets have a countable generating \(\pi\)-system. Almost-sure uniqueness of conditional laws gives \(H(z,A)=J(z,A)\) for almost every \(z\) for each set \(A\) in that system. Take the union of the resulting countably many null sets. Outside this union the two probability measures agree on the generating \(\pi\)-system, and hence agree on all Borel sets. This proves the asserted measure-valued equality. \(\square\)

What the Identity Says

For a bounded Borel function \(f:S\times T\to\mathbb{R}\), the same result can be written as an iterated integral. The conditional expectation of \(f(X,Y)\) given \(Z=z\) has a version given by

$$ \int_{S\times T} f(x,y)\,H(z,d(x,y)) = \int_T\left(\int_S f(x,y)\,K((y,z),dx)\right)L(z,dy), \quad \mu_Z\text{-almost surely}. $$

This form makes the order of operations explicit: evaluate the inner conditional average over \(x\), then average that result over \(y\) given \(z\). The theorem does not claim that a conditional law given \((Y,Z)\) can simply be replaced by one given \(Z\); the averaging over \(Y\) is essential.

Worked Example: A Finite Two-Stage Disintegration

Worked Example: A Finite Two-Stage Disintegration

Let \(Z\) take values \(0\) and \(1\), and suppose that, conditional on \(Z=0\), \(Y\) has probabilities \(2/3\) at \(0\) and \(1/3\) at \(1\). Given \((Y,Z)=(0,0)\), let \(X\) have probabilities \(3/4\) at \(0\) and \(1/4\) at \(1\); given \((Y,Z)=(1,0)\), let \(X\) have probabilities \(1/4\) at \(0\) and \(3/4\) at \(1\).

For \(z=0\), composition gives the four conditional probabilities of \((X,Y)\):

$$ \begin{aligned} H(0,\{(0,0)\})&=\frac23\cdot\frac34=\frac12, & H(0,\{(1,0)\})&=\frac23\cdot\frac14=\frac16,\\ H(0,\{(0,1)\})&=\frac13\cdot\frac14=\frac1{12}, & H(0,\{(1,1)\})&=\frac13\cdot\frac34=\frac14. \end{aligned} $$

They are nonnegative and sum to \(1/2+1/6+1/12+1/4=1\), as required. The marginal conditional probabilities for \(Y=0\) and \(Y=1\) are \(1/2+1/6=2/3\) and \(1/12+1/4=1/3\), recovering the specified conditional law of \(Y\) given \(Z=0\). This is precisely the joint conditional law obtained by first choosing \(Y\) and then choosing \(X\) given \((Y,Z)\).

Worked Example: A Continuous Composition

Worked Example: A Continuous Composition

Let \(Z\) be uniform on \([0,1]\), and let \(U,V\) be independent uniform random variables on \([0,1]\), independent of \(Z\). Set \(Y=Z+V\) and \(X=Y+U\). Conditional on \(Z=z\), \(Y\) is uniform on \([z,z+1]\). Conditional on \((Y,Z)=(y,z)\), \(X\) is uniform on \([y,y+1]\); the value of \(z\) adds no further information once \(y\) is given.

Consequently, the iterated conditional law of \((X,Y)\) given \(Z=z\) has density \(1\) on the region \(z\le y\le z+1\), \(y\le x\le y+1\), and density \(0\) elsewhere. Integrating over \(y\), the conditional density of \(X\) is the length of the intersection \([z,z+1]\cap[x-1,x]\). With \(t=x-z\), this length is

$$ f_{X\mid Z=z}(x) = \begin{cases} t, & 0\le t\le 1,\\ 2-t, & 1<t\le 2,\\ 0, & \text{otherwise}. \end{cases} $$

The two pieces integrate to \(\int_0^1 t\,dt+\int_1^2(2-t)\,dt=1/2+1/2=1\). Thus integrating out the intermediate coordinate produces a probability density, in agreement with the direct conditional law: \(X=z+V+U\), where the sum of two independent uniform variables has this triangular density.

Worked Example: Why Equality Is Only Almost Sure

Worked Example: Why Equality Is Only Almost Sure

Let \(X=Y=0\) almost surely and let \(Z\) be uniform on \([0,1]\). Choose \(K((y,z),\cdot)=\delta_0\) and \(L(z,\cdot)=\delta_0\) for every \(y,z\). Then the iterated kernel satisfies \(H(z,\cdot)=\delta_{(0,0)}\) for every \(z\).

Define another candidate direct conditional law \(J\) by setting \(J(z,\cdot)=\delta_{(0,0)}\) for \(z\ne 0\), but \(J(0,\cdot)=\delta_{(1,1)}\). Since \(\mathbb{P}(Z=0)=0\), changing the kernel at \(z=0\) does not change any disintegration identity: \(J\) is still a version of the conditional law of \((X,Y)\) given \(Z\). Yet \(J(0,\cdot)\ne H(0,\cdot)\). The theorem correctly guarantees equality only outside a \(\mu_Z\)-null set, not at every conditioning value.

Why the Result Is Useful

Associativity lets a complicated joint law be organized into a sequence of simpler conditional laws. This is useful whenever coordinates are revealed in stages: a conditional law for one coordinate given the others can be combined with the conditional law of those intermediate coordinates given a smaller collection. The theorem ensures that the resulting kernel represents the same joint conditional law as a direct disintegration.

A common pitfall is to treat conditional distributions as uniquely specified at every conditioning point. They are generally unique only almost surely under the conditioning variable's law. In particular, formulas for conditional laws at points of probability zero may be chosen differently by different versions. The invariant statement is equality almost surely as probability measures, not pointwise equality for all \(z\).

Check Your Understanding

Use the definitions and theorem to answer the following questions.

  1. In the definition of \(H\), which conditional law is integrated first, and which one supplies the distribution of the intermediate coordinate?
  2. Why is it enough to verify the disintegration identity first on rectangles \(B\times D\)?
  3. What role does the standard Borel assumption play in obtaining equality of the conditional kernels outside one common null set?
  4. In the continuous example, why does the conditional density of \(X\) given \(Z=z\) equal the length of an intersection of intervals?
  5. Why can two valid versions of the direct conditional law disagree at a particular value of \(z\) without contradicting the theorem?

Previous

Next

Back to Tutorials
Clinical Biostats