Almost-Sure Uniqueness of Conditional Kernels

Why Uniqueness Requires Care

A regular conditional distribution assigns a probability measure to each value of the conditioning variable. Its defining integral identities determine those measures only up to sets of conditioning values having probability zero. Consequently, two valid versions need not agree at every conditioning value.

There is a subtle point in turning this observation into a uniqueness statement. For each fixed measurable target set, the two conditional probabilities agree almost surely. But the exceptional null set might depend on that target set. To conclude that the entire probability measures agree simultaneously, we need a way to test them using only countably many sets.

Setup and Definition

Let $X$ and $Y$ be random elements taking values in measurable spaces $E$ and $F$, respectively. Write $\mu_X$ for the distribution of $X$. A probability kernel $K$ from $E$ to $F$ assigns a probability measure $K(x,\cdot)$ on $F$ to each $x\in E$, and $x\mapsto K(x,B)$ is measurable for each measurable $B\subseteq F$.

Definition: A regular conditional distribution of $Y$ given $X$ is a probability kernel $K$ such that, for every measurable $A\subseteq E$ and $B\subseteq F$, $$ \mathbb{P}(X\in A,\ Y\in B)=\int_A K(x,B)\,\mu_X(dx). $$

We compare kernels satisfying this identity for the same pair $(X,Y)$. In particular, they have the same conditioning law $\mu_X$. A class $\mathcal{C}$ of measurable subsets of $F$ is determining for probability measures if any two probability measures on $F$ that agree on every set in $\mathcal{C}$ are equal on all measurable subsets of $F$.

Definition: Two kernels $K$ and $L$ are equal $\mu_X$-almost surely if there is a measurable set $N\subseteq E$ with $\mu_X(N)=0$ such that $K(x,\cdot)=L(x,\cdot)$ for every $x\notin N$.

This is stronger than saying that for each target set $B$, the numbers $K(x,B)$ and $L(x,B)$ agree outside some null set that may depend on $B$.

The Almost-Sure Uniqueness Theorem

Theorem: Suppose $K$ and $L$ are regular conditional distributions of $Y$ given $X$. If there is a countable determining class $\mathcal{C}$ on $F$, then there is a measurable set $N\subseteq E$, with $\mu_X(N)=0$, such that $$ K(x,\cdot)=L(x,\cdot)\qquad\text{for every }x\in E\setminus N. $$

Proof. Fix $B\in\mathcal{C}$. The defining identity for $K$ and $L$ gives, for every measurable $A\subseteq E$, $$ \int_A K(x,B)\,\mu_X(dx) = \mathbb{P}(X\in A,\ Y\in B) = \int_A L(x,B)\,\mu_X(dx). $$ Both integrands are measurable and bounded. Therefore $K(x,B)=L(x,B)$ for $\mu_X$-almost every $x$. To see the uniqueness step directly, if the set where $K(x,B)>L(x,B)$ had positive measure, then for some $n\in\mathbb{N}$ the set where $K(x,B)\ge L(x,B)+1/n$ would have positive measure. Integrating over that set would contradict the displayed equality. Interchanging $K$ and $L$ rules out the reverse strict inequality. For each $B\in\mathcal{C}$, let $N_B=\{x:K(x,B)\ne L(x,B)\}$. This is measurable and $\mu_X(N_B)=0$. Since $\mathcal{C}$ is countable, the union $$ N=\bigcup_{B\in\mathcal{C}}N_B $$ is measurable and has $\mu_X(N)=0$. For $x\notin N$, the probability measures $K(x,\cdot)$ and $L(x,\cdot)$ agree on every $B\in\mathcal{C}$. The determining property then gives $K(x,\cdot)=L(x,\cdot)$. $\square$

The countability assumption is what permits one common exceptional set: a countable union of null sets remains null. If one instead took a union over an uncountable family of target sets, that union need not be null.

Why Countable Generating Classes Determine Measures

A useful way to obtain a determining class is to use a countable collection that generates the target sigma-algebra. In particular, suppose $\mathcal{C}$ is a countable pi-system containing $F$ and generating the measurable sets of $F$. A pi-system is a collection closed under finite intersections.

Proposition: Such a class $\mathcal{C}$ is determining for probability measures on $F$.

Proof. Let $\nu_1$ and $\nu_2$ be probability measures agreeing on $\mathcal{C}$. Define $$ \mathcal{D}=\{B:\ B\text{ is measurable and }\nu_1(B)=\nu_2(B)\}. $$ Because both measures have total mass one, $F\in\mathcal{D}$. If $B\in\mathcal{D}$, then $$ \nu_1(F\setminus B)=1-\nu_1(B)=1-\nu_2(B)=\nu_2(F\setminus B), $$ so $F\setminus B\in\mathcal{D}$. If $B_1,B_2,\ldots$ are pairwise disjoint members of $\mathcal{D}$, countable additivity gives $$ \nu_1\left(\bigcup_{j=1}^{\infty}B_j\right) =\sum_{j=1}^{\infty}\nu_1(B_j) =\sum_{j=1}^{\infty}\nu_2(B_j) =\nu_2\left(\bigcup_{j=1}^{\infty}B_j\right). $$ Thus $\mathcal{D}$ is a lambda-system. It contains the pi-system $\mathcal{C}$, so the pi-lambda theorem implies that it contains the sigma-algebra generated by $\mathcal{C}$. This is the full measurable sigma-algebra on $F$; hence $\nu_1=\nu_2$. $\square$

For a standard Borel target space, one can use a countable generating algebra, and thus obtain a countable determining class. This is why the theorem applies naturally to regular conditional distributions with standard Borel state spaces.

Worked Example: Testing Measures on the Real Line

On $F=\mathbb{R}$ with its Borel sets, take $$ \mathcal{C}=\{\mathbb{R}\}\cup\{(-\infty,q]:q\in\mathbb{Q}\}. $$ This is countable and closed under finite intersections: intersecting two rational rays gives the ray with the smaller endpoint, and intersecting with $\mathbb{R}$ changes nothing. The rays generate the Borel sigma-algebra.

To check the determining property explicitly, suppose probability measures $\nu_1,\nu_2$ agree on $\mathcal{C}$. For any $t\in\mathbb{R}$, choose rational numbers $q_n$ decreasing to $t$. Then the sets $(-\infty,q_n]$ decrease to $(-\infty,t]$. Continuity of finite measures from above gives $$ \nu_1((-\infty,t])=\lim_{n\to\infty}\nu_1((-\infty,q_n]) =\lim_{n\to\infty}\nu_2((-\infty,q_n]) =\nu_2((-\infty,t]). $$ These rays determine probability measures on the Borel sets of $\mathbb{R}$. Thus the countable class suffices, and two conditional distributions on $\mathbb{R}$ that agree almost surely on each rational ray agree as probability measures outside one common null set.

Examples of the Common Exceptional Set

Worked Example: A Countable Target Space

Let $F=\mathbb{N}_0$, with all subsets measurable, and take $\mathcal{C}=\{\{j\}:j\in\mathbb{N}_0\}$. This class is countable and determines probability measures: for any $D\subseteq\mathbb{N}_0$, countable additivity gives $$ \nu(D)=\sum_{j\in D}\nu(\{j\}). $$ Suppose $K$ and $L$ are conditional distributions on this target. For each $j$, the defining identities imply $K(x,\{j\})=L(x,\{j\})$ outside a null set $N_j$. The union $N=\bigcup_{j\in\mathbb{N}_0}N_j$ is null. For $x\notin N$, the two kernels assign the same mass to every singleton, so the displayed sum shows they assign the same mass to every subset. This produces equality of the whole probability measures on one common full-measure set.

Worked Example: Changing a Version at a Null Conditioning Value

Let $\Omega=[0,1]$ with Lebesgue probability, and set $X(\omega)=\omega$ and $Y(\omega)=0$, both viewed as real-valued random variables. Define $$ K(x,\cdot)=\delta_0,\qquad L(x,\cdot)= \begin{cases} \delta_1,&x=0,\\ \delta_0,&x\ne 0. \end{cases} $$ Here $\delta_a$ is the probability measure assigning mass one to $a$. Both assignments are probability kernels: for each Borel set $B$, $K(x,B)$ is constant in $x$, while $L(x,B)$ differs from that constant only on the measurable singleton $\{0\}$.

For any Borel sets $A,B\subseteq\mathbb{R}$, the joint probability is $$ \mathbb{P}(X\in A,\ Y\in B)=\lambda(A)\mathbf{1}_{\{0\in B\}}, $$ where $\lambda$ is Lebesgue measure on $[0,1]$. Integrating $K$ gives this value. Integrating $L$ can change the integrand only at $x=0$, and $\lambda(\{0\})=0$; hence it gives the same value. Both kernels are regular conditional distributions, yet $K(0,\cdot)=\delta_0\ne\delta_1=L(0,\cdot)$. They nevertheless agree for $\mu_X$-almost every $x$.

What Almost-Sure Uniqueness Does and Does Not Say

The theorem identifies a probability-one set of conditioning values on which the two kernels agree as measures. It does not select a unique value on the exceptional null set. As the example shows, a version can be modified there while preserving both measurability and the conditional integral identities.

Nor does the theorem say that an arbitrary pointwise edit produces a valid kernel. The modified assignment must still be measurable in the conditioning value and must still give a probability measure at every value. Replacing the kernel on a measurable null set by one fixed probability measure is a valid modification: the new assignments remain measurable, and integrals over the null set do not change.

The countability argument also explains a common pitfall. For each fixed $B$, conditional-expectation uniqueness gives a null set outside which $K(x,B)=L(x,B)$. Without a countable determining class, this statement alone does not justify choosing one null set that works for all measurable $B$. The theorem obtains simultaneous equality by first restricting to countably many tests, then using their determining property to recover the whole measure.

Key takeaway: Regular conditional distributions are not generally pointwise unique. When the target admits a countable determining class, they are equal as probability measures outside one common null set of conditioning values.

Check Your Understanding

Use the definitions and the countability argument to answer these questions.

Why does agreement almost surely for each fixed target set not by itself give one common exceptional null set?
In the uniqueness proof, where exactly is countability of the determining class used?
Why does a countable pi-system generating the target sigma-algebra determine probability measures?
In the real-line example, why do rational-endpoint rays suffice to determine the measure of every ray $(-\infty,t]$?
Why are the two kernels in the null-modification example both valid even though their values differ at $x=0$?