Disintegration of a Joint Probability

From a Joint Law to Conditional Laws

A probability measure on a product space describes two random elements jointly. Disintegration expresses that same measure in two stages: first choose the value of the first coordinate according to its marginal law, then use a probability measure on the second coordinate that depends measurably on the first.

This representation is useful because it turns joint probabilities into iterated integrals. Its central assertion is not merely a formula for rectangles: it gives a valid integration formula for every measurable subset of the product space.

Definition: A measurable space is standard Borel if it is measurably isomorphic to a Borel subset of a Polish space. A probability kernel $K$ from a measurable space $S$ to a measurable space $T$ assigns a probability measure $K(x,\cdot)$ on $T$ to each $x\in S$, with the property that $x\mapsto K(x,B)$ is measurable for every measurable $B\subseteq T$.

For a measurable set $C\subseteq S\times T$, write $C_x=\{y\in T:(x,y)\in C\}$ for its section at $x$. Sections of product-measurable sets are measurable. The integral in the disintegration formula will involve $K(x,C_x)$, so it is important that this quantity is also measurable as a function of $x$.

The Disintegration Theorem

Let $S$ and $T$ be standard Borel spaces, let $\mu$ be a probability measure on $S\times T$, and let $\mu_S$ be its first marginal, defined by $\mu_S(A)=\mu(A\times T)$. A disintegration of $\mu$ over $\mu_S$ is a probability kernel $K$ from $S$ to $T$ satisfying the identities below.

Theorem (Standard Borel disintegration theorem): For standard Borel spaces $S,T$ and every probability measure $\mu$ on $S\times T$, there exists a probability kernel $K$ from $S$ to $T$ such that, for every measurable $A\subseteq S$ and $B\subseteq T$, $$ \mu(A\times B)=\int_A K(x,B)\,\mu_S(dx). $$ Moreover, for every measurable $C\subseteq S\times T$, $$ \mu(C)=\int_S K(x,C_x)\,\mu_S(dx). $$

The rectangle identity says that the joint law of the two coordinates can be recovered by integrating the conditional law of the second coordinate over the marginal law of the first. The second identity extends the same representation from rectangles to arbitrary measurable sets.

Why the Formula Is Measurable

We first verify the measurability needed for the second identity. Fix a probability kernel $K$ from $S$ to $T$. Consider the collection of product-measurable sets $C$ for which $x\mapsto K(x,C_x)$ is measurable. This collection contains every measurable rectangle $A\times B$, because $$ K(x,(A\times B)_x)=\mathbf{1}_A(x)K(x,B). $$ It is closed under complements: the section of $C^c$ is $T\setminus C_x$, so $K(x,(C^c)_x)=1-K(x,C_x)$. It is also closed under countable disjoint unions, since the sections are disjoint unions and countable additivity gives $$ K\left(x,\left(\bigcup_{n=1}^{\infty}C_n\right)_x\right) =\sum_{n=1}^{\infty}K(x,(C_n)_x). $$ The sum on the right is a limit of measurable partial sums and is therefore measurable.

Measurable rectangles form a pi-system: they contain $S\times T$ and are closed under finite intersections. The pi-lambda theorem says that a lambda-system containing a pi-system contains the sigma-algebra generated by that pi-system. Since the product sigma-algebra is generated by measurable rectangles, $x\mapsto K(x,C_x)$ is measurable for every product-measurable $C$.

Proof of the Theorem

Proof. Regard $S\times T$, equipped with $\mu$, as a probability space, and let $X$ and $Y$ be its coordinate maps. Thus $X(x,y)=x$ and $Y(x,y)=y$. The regular conditional probability existence theorem for standard Borel state spaces gives a probability kernel $K$ from $S$ to $T$ that is a conditional law of $Y$ given $X$. In particular, for every measurable $A\subseteq S$ and $B\subseteq T$, $$ \mu(X\in A,\ Y\in B)=\int_{\{X\in A\}}K(X,B)\,d\mu =\int_A K(x,B)\,\mu_S(dx). $$ The left side is $\mu(A\times B)$, which proves the rectangle identity.

To extend it, define, for every product-measurable $C\subseteq S\times T$, $$ \nu(C)=\int_S K(x,C_x)\,\mu_S(dx). $$ The measurability argument above shows that this integral is defined. Also, $\nu(S\times T)=\int_S K(x,T)\,\mu_S(dx)=1$. If $C_1,C_2,\ldots$ are pairwise disjoint measurable sets, their sections are pairwise disjoint for every $x$. Countable additivity of $K(x,\cdot)$, followed by the monotone convergence theorem, gives $$ \nu\left(\bigcup_{n=1}^{\infty}C_n\right) =\int_S\sum_{n=1}^{\infty}K(x,(C_n)_x)\,\mu_S(dx) =\sum_{n=1}^{\infty}\nu(C_n). $$ Thus $\nu$ is a probability measure.

For a measurable rectangle $A\times B$, its section is $B$ when $x\in A$ and is empty otherwise. Consequently, $$ \nu(A\times B)=\int_A K(x,B)\,\mu_S(dx)=\mu(A\times B). $$ The probability measures $\nu$ and $\mu$ therefore agree on the pi-system of measurable rectangles, which contains $S\times T$. Uniqueness of finite measures on a generating pi-system implies that they agree on the product sigma-algebra. Hence $\nu(C)=\mu(C)$ for every measurable $C$, as required.

$\square$

Worked Examples

Worked Example: A Product Joint Law

Let $\alpha$ and $\beta$ be probability measures on standard Borel spaces $S$ and $T$, and let $\mu=\alpha\otimes\beta$. Define $K(x,B)=\beta(B)$ for every $x\in S$ and measurable $B\subseteq T$. This is a probability kernel because it is a probability measure in $B$, and its value is constant, hence measurable, in $x$.

The first marginal is $\mu_S=\alpha$. For measurable $A\subseteq S$ and $B\subseteq T$, $$ \int_A K(x,B)\,\alpha(dx) =\int_A \beta(B)\,\alpha(dx) =\alpha(A)\beta(B) =(\alpha\otimes\beta)(A\times B). $$ Thus the constant kernel gives the disintegration. For any measurable $C\subseteq S\times T$, the full-set formula becomes the product-measure integration formula $$ (\alpha\otimes\beta)(C)=\int_S\beta(C_x)\,\alpha(dx). $$ Here the conditional law of the second coordinate does not depend on the first.

Worked Example: A Joint Law on a Measurable Graph

Let $\alpha$ be a probability measure on a standard Borel space $S$, let $f:S\to T$ be measurable, and define $\mu$ to be the law of $(Z,f(Z))$ when $Z$ has law $\alpha$. Then $\mu_S=\alpha$. Set $K(x,\cdot)=\delta_{f(x)}$, the point mass at $f(x)$. For any measurable $B\subseteq T$, $$ K(x,B)=\mathbf{1}_B(f(x)), $$ which is measurable in $x$; hence $K$ is a probability kernel.

For any measurable $C\subseteq S\times T$, the section formula for this kernel gives $$ K(x,C_x)=\delta_{f(x)}(C_x)=\mathbf{1}_C(x,f(x)). $$ Therefore $$ \int_S K(x,C_x)\,\alpha(dx) =\int_S\mathbf{1}_C(x,f(x))\,\alpha(dx) =\mu(C). $$ This disintegration concentrates, for each $x$, all conditional mass at the single point $f(x)$.

Worked Example: A Density Supported Below the Diagonal

On $[0,1]^2$, define a probability measure $\mu$ by the density $$ p(x,y)=2\mathbf{1}_{\{0<y<x<1\}} $$ with respect to two-dimensional Lebesgue measure. It is a probability density because $$ \int_0^1\int_0^1 p(x,y)\,dy\,dx =\int_0^1 2x\,dx =1. $$ Integrating out $y$, the first marginal has density $q(x)=2x$ for $0<x<1$, and zero elsewhere.

Define $K(0,\cdot)=\delta_0$. For $0<x\leq 1$, define $$ K(x,B)=\frac{1}{x}\int_0^x\mathbf{1}_B(y)\,dy. $$ For each such $x$, this is a probability measure: it is normalized because $x^{-1}\int_0^x1\,dy=1$. It is measurable in $x$ for each Borel set $B$, since the integral of the measurable function $(x,y)\mapsto\mathbf{1}_{\{0<y<x\}}\mathbf{1}_B(y)$ over $y$ is measurable in $x$, and division by $x$ is measurable on $(0,1]$. The separately defined value at $x=0$ preserves measurability.

For Borel sets $A,B\subseteq[0,1]$, the rectangle formula follows by direct calculation: $$ \int_A K(x,B)\,\mu_S(dx) =\int_{A\cap(0,1)}\frac{1}{x}\left(\int_0^x\mathbf{1}_B(y)\,dy\right)2x\,dx =\int_A\int_0^x2\mathbf{1}_B(y)\,dy\,dx =\mu(A\times B). $$ The endpoints do not affect these integrals because the first marginal has no mass there. Thus the conditional measure at $x>0$ is uniform on $[0,x]$, and integrating these measures against the marginal density $2x$ recovers the original joint density.

Versions, Scope, and a Common Pitfall

A disintegration is generally determined only up to a $\mu_S$-null set of values of $x$. Changing $K(x,\cdot)$ on such a set does not change any of the integrals in the theorem. The almost-sure uniqueness result for conditional kernels explains this limitation; the theorem does not assert that the conditional measure at every individual $x$ is uniquely determined by $\mu$.

A common pitfall is to establish the rectangle formula and then use it for an arbitrary measurable set without justification. The extension is valid because the kernel produces a probability measure on the product sigma-algebra, and that measure agrees with $\mu$ on a generating pi-system. The theorem also depends on the standard Borel hypotheses: they ensure that the regular conditional probability needed to construct the kernel exists in this generality.

Disintegration is a general way to separate a joint law into a marginal and a family of conditional laws. The product example has a constant family, the graph example has point masses, and the density example has a continuously varying family of measures. In each case, the same theorem guarantees that integrating the family against the marginal recovers the entire joint probability.

Check Your Understanding

Use the definitions and identities in this tutorial to answer the following questions.

State the rectangle identity in the standard Borel disintegration theorem and identify the marginal measure used in its integral.
Why is $x\mapsto K(x,C_x)$ measurable for a product-measurable set $C$? Which closure argument is central?
In the graph example, what is $K(x,\cdot)$, and why does it recover the joint law?
For the density supported below the diagonal, verify that the marginal density integrates to one.
Why does agreement on rectangles imply agreement on every measurable subset of $S\times T$?