From Conditional Distributions to Random Measures
A conditional distribution is often introduced by asking for the conditional probability of an event, such as the probability that \(Y\) lies in a set \(A\) after observing \(X\). But a conditional law gives more than one probability at a time: for each observed value \(x\), it specifies an entire probability measure on the state space of \(Y\).
This viewpoint is useful when the object of interest is the conditional distribution itself. We can ask whether it varies measurably with the observation, evaluate it at a random observation, or integrate functions against it. The answers require a measurable structure on the space of probability measures.
The Space of Probability Measures
Let \(S\) be a Polish space, meaning a topological space whose topology is given by a complete separable metric. Write \(\mathcal{B}(S)\) for its Borel sets and \(\mathcal{P}(S)\) for the set of probability measures on \(\mathcal{B}(S)\). We equip \(\mathcal{P}(S)\) with the weak topology: the coarsest topology for which every map \(\mu\mapsto\int f\,d\mu\), with \(f:S\to\mathbb{R}\) bounded and continuous, is continuous. The Borel sets of this topology form \(\mathcal{B}(\mathcal{P}(S))\).
For each Borel set \(A\subseteq S\), define the evaluation map \(e_A:\mathcal{P}(S)\to[0,1]\) by \(e_A(\mu)=\mu(A)\). These maps connect the measurable structure of \(\mathcal{P}(S)\) to ordinary probabilities.
Proof. First consider an open set \(U\subseteq S\) with \(U\ne S\), and choose a compatible metric \(d\). The functions \(\phi_n(s)=\min\{1,n\,d(s,U^c)\}\) are bounded and continuous, and increase pointwise to \(\mathbf{1}_U\). Thus, by monotone convergence, \(\mu(U)=\lim_{n\to\infty}\int\phi_n\,d\mu\), so \(e_U\) is Borel measurable for the weak topology. For \(U=S\), \(e_U\) is the constant function \(1\).
The collection of Borel sets \(A\subseteq S\) for which \(e_A\) is measurable is a sigma-field: it is closed under complements because \(e_{A^c}=1-e_A\), and under countable disjoint unions because \(e_{\cup_n A_n}=\sum_n e_{A_n}\). Disjointifying a general countable union gives closure under all countable unions. This collection contains every open set, hence every Borel set. Therefore \(\sigma(e_A:A\in\mathcal{B}(S))\subseteq\mathcal{B}(\mathcal{P}(S))\).
For the reverse inclusion, if \(f:S\to\mathbb{R}\) is bounded and Borel measurable, then \(\mu\mapsto\int f\,d\mu\) is measurable with respect to \(\sigma(e_A)\). This follows first for indicators from the definition of \(e_A\), then for simple functions by finite linear combinations, and then for bounded Borel functions by pointwise simple-function approximation and limits. In particular, this holds for every bounded continuous \(f\). The weak topology is generated by these integral maps; since \(\mathcal{P}(S)\) with the weak topology is second countable, its Borel sigma-field is generated by those maps. Hence \(\mathcal{B}(\mathcal{P}(S))\subseteq\sigma(e_A:A\in\mathcal{B}(S))\). The two inclusions prove the claim.
\(\square\)
Probability Kernels as Measurable Maps
Let \(E\) be a measurable space. A probability kernel \(K\) from \(E\) to \(S\) assigns a probability measure \(K(x,\cdot)\) on \(S\) to each \(x\in E\), so that \(x\mapsto K(x,A)\) is measurable for every \(A\in\mathcal{B}(S)\). The assignment \(x\mapsto K(x,\cdot)\) therefore has values in \(\mathcal{P}(S)\). The theorem above shows that this assignment is measurable for the Borel structure of the weak topology.
Proof. For every Borel set \(A\subseteq S\), the composition \(e_A(K(x,\cdot))=K(x,A)\) is measurable by the definition of a probability kernel. The Borel sigma-field of \(\mathcal{P}(S)\) is generated by these evaluation maps. A map into \(\mathcal{P}(S)\) is consequently measurable if all these compositions are measurable, which proves the proposition.
\(\square\)
There is also an integral version of this fact. For any bounded Borel function \(f:S\to\mathbb{R}\), write \(Kf(x)=\int f(y)\,K(x,dy)\). The function \(x\mapsto Kf(x)\) is measurable: the assertion holds for indicators by the kernel property, extends to simple functions by linearity, and then to bounded Borel functions by approximation. This includes bounded continuous functions, whose integrals are the coordinates used to describe the weak topology.
Conditional Laws as Random Probability Measures
Let \((\Omega,\mathcal{F},\mathbb{P})\) be a probability space, let \(X:\Omega\to E\) and \(Y:\Omega\to S\) be measurable random elements, and suppose \(K\) is a conditional law of \(Y\) given \(X\). In kernel notation, this means that for every \(A\in\mathcal{B}(S)\), \(K(X,A)\) is a version of \(\mathbb{P}(Y\in A\mid\sigma(X))\). When \(E\) and \(S\) are standard Borel spaces, regular conditional laws of this kind exist.
Because \(x\mapsto K(x,\cdot)\) is measurable, its composition with \(X\) is a measurable map from \(\Omega\) into \(\mathcal{P}(S)\). We can therefore regard \(K(X,\cdot)\) as a random probability measure. Its evaluation at \(A\) is the random variable \(K(X,A)\); its integral of \(f\) is the random variable \(\int f\,dK(X,\cdot)\).
Proof. For \(f=\mathbf{1}_A\), the assertion is precisely the defining conditional-law property. By linearity it holds for simple functions. If \(f\) is nonnegative and bounded, choose nonnegative simple functions increasing pointwise to \(f\). Monotone convergence gives convergence of their integrals against \(K(X,\cdot)\), and the conditional monotone convergence theorem gives convergence of the corresponding conditional expectations. Thus the identity holds for nonnegative bounded Borel \(f\). For a bounded real-valued \(f\), write \(f=f^+-f^-\); both parts are nonnegative and bounded, so subtracting their identities proves the result.
\(\square\)
The two sides of this identity are versions of the same conditional expectation, so their almost-sure equality is asserted separately for each chosen \(f\). In particular, taking \(f=\mathbf{1}_A\) recovers conditional probabilities. Taking other bounded functions gives conditional expectations of transformations of \(Y\). For nonnegative unbounded \(f\), the same identity holds with extended nonnegative values by monotone convergence; integrable signed functions can be handled by their positive and negative parts.
Worked Example: A Conditional Law on a Finite Space
Worked Example: A Conditional Law on a Finite Space
Let \(X\) take values \(0\) and \(1\), with \(\mathbb{P}(X=0)=0.6\) and \(\mathbb{P}(X=1)=0.4\). Suppose \(Y\) takes values in \(\{0,1\}\), with conditional laws
For \(A=\{1\}\), its evaluation is \(K(X,A)=0.8\) when \(X=0\) and \(0.3\) when \(X=1\). It is the conditional probability \(\mathbb{P}(Y=1\mid\sigma(X))\). Averaging gives \(\mathbb{P}(Y=1)=0.6(0.8)+0.4(0.3)=0.6\), as required by the law of total probability. For \(f(y)=y\), the integral \(\int f\,dK(X,\cdot)\) has the same values, \(0.8\) and \(0.3\), and is \(\mathbb{E}[Y\mid\sigma(X)]\).
Worked Example: A Gaussian Conditional Law
Worked Example: A Gaussian Conditional Law
Let \(X\) and \(Z\) be independent standard normal random variables and set \(Y=X+Z\). Define
The map \(x\mapsto K(x,\cdot)\) is in fact continuous for the weak topology. If \(f\) is bounded and continuous, then \(Kf(x)=\int f(x+z)\varphi(z)\,dz\). Whenever \(x_n\to x\), continuity of \(f\) gives \(f(x_n+z)\to f(x+z)\) for every \(z\), and boundedness gives an integrable dominating function, namely \(\|f\|_\infty\varphi(z)\). Dominated convergence yields \(Kf(x_n)\to Kf(x)\).
For any \(a\in\mathbb{R}\), evaluating the random measure on \((-\infty,a]\) gives
Worked Example: A Symmetric Two-Point Conditional Law
Worked Example: A Symmetric Two-Point Conditional Law
Let \(U\) be uniform on \([0,1]\), and let \(R\) be independent of \(U\), taking each of \(-1\) and \(1\) with probability \(1/2\). Set \(X=U\) and \(Y=RU\). For \(x\in[0,1]\), define
For bounded continuous \(f\), the integral is \(Kf(x)=\tfrac12f(x)+\tfrac12f(-x)\), a continuous function of \(x\). Hence \(x\mapsto K(x,\cdot)\) is weakly continuous. Taking \(f(y)=y^2\) gives \(Kf(x)=x^2\), so \(\mathbb{E}[Y^2\mid\sigma(X)]=X^2\). Taking \(f=\mathbf{1}_{(0,\infty)}\) gives \(Kf(x)=1/2\) for \(x>0\) and \(Kf(0)=0\). The value at zero is consistent with the two atoms merging there; since \(\mathbb{P}(X=0)=0\), it does not affect any almost-sure conditional-expectation identity.
What the Random-Measure Viewpoint Clarifies
Thinking of a conditional law as a random measure separates three related objects. First is the measurable map \(x\mapsto K(x,\cdot)\). Second is its random evaluation \(K(X,A)\), a conditional probability for a chosen event. Third is its integral \(\int f\,dK(X,\cdot)\), a conditional expectation for a chosen function. The measurable-map result ensures that these operations are legitimate random variables rather than merely pointwise formulas.
A common pitfall is to assume that measurability of one evaluation automatically describes the whole random measure without specifying a measurable structure on \(\mathcal{P}(S)\). The weak topology supplies a natural structure when \(S\) is Polish, and the evaluation theorem identifies its Borel sets. Another subtlety is that a conditional law is determined only up to changes on a set of conditioning values having probability zero. Consequently, pointwise statements about \(K(x,\cdot)\) at every \(x\) may depend on the chosen version, while \(K(X,\cdot)\) and its evaluations are determined only up to almost-sure equality.
Check Your Understanding
Use the definitions and results above to answer these questions.
- What is the evaluation map associated with a Borel set \(A\), and what does its value mean for a probability measure?
- Why does measurability of \(x\mapsto K(x,A)\) for every Borel set \(A\) imply measurability of \(x\mapsto K(x,\cdot)\) into \(\mathcal{P}(S)\)?
- For a bounded Borel function \(f\), what conditional expectation is represented by \(\int f\,dK(X,\cdot)\)?
- In the Gaussian example, why is \(x\mapsto K(x,\cdot)\) continuous in the weak topology?
- Why can changing a conditional kernel on a null set of conditioning values leave the resulting random measure unchanged almost surely?