Compactness of Probability-Kernel Decision Rules

Why Compactness of Decision Rules Is a Measure-Theoretic Question

In a randomized decision rule, the action need not be a single function of the observation. Instead, each observation determines a probability distribution over possible actions. To obtain compactness, we therefore need a topology that allows distributions over actions to converge. The convenient representation is the joint distribution of the observation and the randomized action.

This representation also makes the role of the action space precise. If the action space is compact, probability cannot escape indefinitely toward actions “at infinity.” If it is not compact, compactness of a family of rules requires a uniform tightness condition. The results below concern the rules themselves; compactness alone does not establish that risks converge under weak limits.

Definition: Let \(\mathcal X\) be a standard Borel observation space and let \(\mathcal A\) be a Polish action space. A probability kernel \(K(da\mid x)\) assigns a probability measure on \(\mathcal A\) to each \(x\in\mathcal X\), such that \(x\mapsto K(B\mid x)\) is measurable for every Borel set \(B\subseteq\mathcal A\). Fix a probability measure \(Q\) on \(\mathcal X\). The kernel induces the joint probability measure \(\mu_K(dx,da)=Q(dx)K(da\mid x)\) on \(\mathcal X\times\mathcal A\).

The first marginal of every \(\mu_K\) is \(Q\). Moreover, two kernels that agree \(Q\)-almost everywhere induce the same joint measure. Conversely, the standard disintegration theorem says that every probability measure on \(\mathcal X\times\mathcal A\) with first marginal \(Q\) has a representation \(Q(dx)K(da\mid x)\), unique up to \(Q\)-null sets. Thus joint measures with fixed first marginal represent kernels exactly up to changes that cannot be seen under \(Q\).

We use the weak topology on kernels relative to \(Q\): \(K_n\) converges to \(K\) if \(\mu_{K_n}\) converges weakly to \(\mu_K\). Weak convergence of probability measures means convergence of integrals against every bounded continuous function on the product space. This topology records convergence of the joint laws, not necessarily convergence of \(K_n(\cdot\mid x)\) for every individual observation.

The Compactness Theorem

Theorem: Suppose \(\mathcal X\) is standard Borel, \(Q\) is a probability measure on \(\mathcal X\), and \(\mathcal A\) is compact metrizable. The collection of probability kernels from \(\mathcal X\) to \(\mathcal A\), identified when they agree \(Q\)-almost everywhere and equipped with the weak topology relative to \(Q\), is compact.

Here is the measure-theoretic argument. Equip \(\mathcal X\) with a Polish topology that generates its standard Borel sets. Let \(\mathcal M_Q\) be the set of probability measures on \(\mathcal X\times\mathcal A\) whose first marginal is \(Q\). By disintegration, this set is precisely the joint-measure representation of the kernels under consideration.

First, \(\mathcal M_Q\) is tight. Since \(Q\) is a probability measure on a Polish space, it is tight: for each \(\varepsilon>0\), there is a compact \(C\subseteq\mathcal X\) with \(Q(C)>1-\varepsilon\). Since \(\mathcal A\) is compact, \(C\times\mathcal A\) is compact. For any \(\mu\in\mathcal M_Q\),

\mu\big((C\times\mathcal A)^c\big) =\mu(C^c\times\mathcal A) =Q(C^c) <\varepsilon.

The bound is uniform over \(\mu\in\mathcal M_Q\), so Prokhorov’s theorem gives relative compactness.

Second, \(\mathcal M_Q\) is weakly closed. If \(\mu_n\in\mathcal M_Q\) and \(\mu_n\) converges weakly to \(\mu\), then for every bounded continuous \(g:\mathcal X\to\mathbb R\),

\int_{\mathcal X\times\mathcal A}g(x)\,\mu(dx,da) =\lim_{n\to\infty}\int_{\mathcal X\times\mathcal A}g(x)\,\mu_n(dx,da) =\int_{\mathcal X}g(x)\,Q(dx).

These equalities identify the first marginal of \(\mu\) as \(Q\), so \(\mu\in\mathcal M_Q\). A relatively compact, closed subset of the space of probability measures is compact. Disintegration then transfers this compactness to the kernels, with the stated identification modulo \(Q\)-null sets. \(\square\)

Worked Example: A Convergent Sequence With Two Observations

Let \(\mathcal X=\{0,1\}\), let \(\mathcal A=[0,1]\), and take \(Q(0)=Q(1)=1/2\). Define kernels \(K_n\) by \(K_n(\cdot\mid 0)=\delta_{1/n}\) and \(K_n(\cdot\mid 1)=p_n\delta_0+(1-p_n)\delta_1\), where \(p_n=1/2+(-1)^n/(4n)\). We identify the weak limit explicitly.

Let \(\varphi\) be any bounded continuous function on \(\{0,1\}\times[0,1]\). The integral under the joint law is

\int \varphi\,d\mu_{K_n} =\frac12\varphi(0,1/n) +\frac12\left[p_n\varphi(1,0)+(1-p_n)\varphi(1,1)\right].

As \(n\to\infty\), continuity gives \(\varphi(0,1/n)\to\varphi(0,0)\), and \(p_n\to1/2\). Therefore

\int \varphi\,d\mu_{K_n} \longrightarrow \frac12\varphi(0,0) +\frac14\varphi(1,0) +\frac14\varphi(1,1).

This is the joint law induced by the kernel \(K\) with \(K(\cdot\mid 0)=\delta_0\) and \(K(\cdot\mid 1)=\tfrac12\delta_0+\tfrac12\delta_1\). Thus \(K_n\) converges weakly to \(K\). The calculation shows why it is the distribution over actions, rather than a single selected action, that must be allowed to converge.

Tightness When the Action Space Is Not Compact

Compactness of \(\mathcal A\) makes tightness automatic for the full collection of kernels. When \(\mathcal A\) is not compact, a family of rules may place more and more probability on actions outside every compact set. The relevant condition is uniform tightness of the action marginals obtained by averaging over \(Q\).

Criterion: Let \(\mathcal X\) and \(\mathcal A\) be Polish, let \(Q\) be a probability measure on \(\mathcal X\), and let \(\mathcal K\) be a family of probability kernels. The associated joint measures \(\{\mu_K:K\in\mathcal K\}\) are relatively compact in the weak topology if and only if, for every \(\varepsilon>0\), there is a compact \(D\subseteq\mathcal A\) such that \(\sup_{K\in\mathcal K}\int K(D^c\mid x)\,Q(dx)<\varepsilon\). The family is compact if, in addition, its associated joint measures form a weakly closed set.

To verify the criterion, first note that \(Q\) is tight. For any \(\varepsilon>0\), choose compact \(C\subseteq\mathcal X\) with \(Q(C^c)<\varepsilon/2\), and use the criterion to choose compact \(D\subseteq\mathcal A\) with \(\sup_K\mu_K(\mathcal X\times D^c)<\varepsilon/2\). Then

\sup_{K\in\mathcal K}\mu_K\big((C\times D)^c\big) \le Q(C^c)+\sup_{K\in\mathcal K}\mu_K(\mathcal X\times D^c) <\varepsilon.

Thus the joint measures are uniformly tight, and Prokhorov’s theorem gives relative compactness. In the other direction, relative compactness implies uniform tightness of the joint measures. Projection of a compact subset of \(\mathcal X\times\mathcal A\) onto \(\mathcal A\) is compact, so uniform tightness of the joint measures implies the displayed action-tail condition. Finally, a relatively compact set is compact precisely when it is closed.

Worked Example: A Family That Is Not Tight on an Unbounded Action Space

Let the observation space consist of a single point and the action space be \(\mathbb R\). Consider deterministic rules \(K_n=\delta_n\). Their joint measures are simply \(\delta_n\) on \(\mathbb R\). If \(D\subseteq\mathbb R\) is compact, it is bounded, so there is an integer \(N\) such that \(n\notin D\) for all \(n\ge N\). Consequently,

\sup_n K_n(D^c)=1.

No compact \(D\) can make this supremum smaller than, for example, \(1/2\). The family fails the uniform tightness criterion and is not relatively compact. The failure is escape of action probability to infinity, not any irregularity in the observation model.

Finite Observation Models

Suppose the observation space is finite, \(\mathcal X=\{x_1,\ldots,x_m\}\), and the reference measure satisfies \(Q(x_i)>0\) for each \(i\). A kernel is then specified by the \(m\)-tuple of probability measures \((K(\cdot\mid x_1),\ldots,K(\cdot\mid x_m))\). Weak convergence of the joint measures is equivalent to weak convergence of each coordinate measure. Indeed, for a bounded continuous \(\varphi\) on \(\mathcal A\) and each \(i\), the function on the product that equals \(\varphi(a)\) when \(x=x_i\) and zero otherwise is bounded and continuous. Its integral is \(Q(x_i)\int\varphi\,dK(\cdot\mid x_i)\). Since \(Q(x_i)>0\), convergence of joint integrals implies convergence of the coordinate integrals. The reverse follows by summing the finitely many coordinate integrals.

Thus, with compact \(\mathcal A\), the kernel space has the familiar finite-product form \(\mathcal P(\mathcal A)^m\), where each factor is compact in its weak topology. This gives a direct route to subsequences in finite observation problems.

Worked Example: Extracting a Subsequence in a Finite Model

Let \(\mathcal X=\{x_1,x_2,x_3\}\), let every observation have positive reference probability, and let \(\mathcal A=[0,1]\). Consider any sequence of kernels \(K_n\). For \(x_1\), compactness of \(\mathcal P([0,1])\) gives a subsequence along which \(K_n(\cdot\mid x_1)\) converges weakly. From that subsequence, extract one along which the coordinate at \(x_2\) converges. Extract once more for \(x_3\). There are only three coordinates, so the final subsequence converges weakly at all of them.

For each \(i\), let the coordinate limit be \(\nu_i\), a probability measure on \([0,1]\). Define \(K(\cdot\mid x_i)=\nu_i\). For every bounded continuous \(\varphi\) on \(\{x_1,x_2,x_3\}\times[0,1]\), the joint integral is the finite sum

\int\varphi\,d\mu_{K_n} =\sum_{i=1}^3Q(x_i)\int_{[0,1]}\varphi(x_i,a)\,K_n(da\mid x_i).

Each term converges by coordinate-wise weak convergence, so the sum converges to \(\int\varphi\,d\mu_K\). This explicitly recovers a weakly convergent subsequence of rules.

Worked Example: Why the Reference Measure Defines the Relevant Equality

Let \(\mathcal X=[0,1]\), take \(Q\) to be the uniform distribution, and let \(\mathcal A=\{0,1\}\). Define two kernels \(K\) and \(K'\) to agree at every \(x>0\), but set \(K(\{1\}\mid0)=0\) and \(K'(\{1\}\mid0)=1\). The kernels differ at \(x=0\), but \(Q(\{0\})=0\). For any bounded measurable \(h\) on \(\mathcal X\times\mathcal A\),

\int h(x,a)\,Q(dx)K(da\mid x) =\int h(x,a)\,Q(dx)K'(da\mid x),

because the integrands can differ only on a set whose observation marginal has probability zero. The two rules therefore have the same joint measure and are the same point in the weak kernel space relative to \(Q\). If every observation distribution of interest is absolutely continuous with respect to \(Q\), this null-set change is also invisible under those observation distributions. Without such a common domination condition, equality relative to \(Q\) alone need not identify rules for other observation laws.

Common Pitfall and Practical Use

A common mistake is to say that a sequence of kernels has a convergent subsequence merely because every \(K_n(\cdot\mid x)\) is a probability measure. Compactness must be justified in a topology and with conditions that prevent probability from escaping. For compact actions, the joint-measure argument supplies those conditions. For noncompact actions, one must check uniform tightness; pointwise properties of individual kernels do not suffice.

A second mistake is to interpret weak convergence of joint measures as pointwise convergence of the conditional distributions at every observation. The topology only controls the joint law under \(Q\), and kernels are identified up to \(Q\)-null sets. In a finite model with all \(Q(x_i)>0\), this reduces to coordinate-wise weak convergence. In a general observation space, it does not.

These compactness results provide an existence tool for arguments involving sequences of randomized rules: with compact actions, a sequence always has a weakly convergent subsequence; with noncompact actions, a uniform action-tail bound is the key substitute. Any conclusion about risks along that subsequence requires additional analysis of the loss and its regularity.

Check Your Understanding

Use the joint-measure representation and the tightness criterion to answer the following.

What is the first marginal of \(\mu_K\), and why does it remain fixed as \(K\) varies?
Why does compactness of the action space make the full collection of kernels relatively compact?
For a noncompact action space, state the uniform tightness condition on a family of kernels relative to \(Q\).
In a finite observation model with positive \(Q(x_i)\), why is weak convergence of joint measures equivalent to weak convergence at each observation?
Why can two kernels that differ on a \(Q\)-null set represent the same point in the weak kernel space?