Why Risk Needs a One-Sided Limit Result
In a compactness argument, a sequence of randomized decision rules may have a weakly convergent subsequence. To use its limit, one needs to know what happens to risk. Weak convergence controls expectations of bounded continuous functions, but a loss need not be continuous or bounded above. The useful conclusion under weaker assumptions is one-sided: the risk of the limit is no greater than the lower limit of the risks along the sequence.
This is a statement about each fixed parameter value. It does not assert that risks converge, and it does not by itself establish any comparison between two decision rules. The conditions on the loss and on the probability measures are what make the one-sided inequality valid.
Setup and Definitions
Let \(\mathcal X\) be a Polish observation space and let \(\mathcal A\) be a compact metric action space. For a fixed parameter \(\theta\), write \(P_\theta\) for the distribution of the observation. A randomized decision rule \(\delta\) is a probability kernel from \(\mathcal X\) to \(\mathcal A\): for each observation \(x\), \(\delta(\,\cdot\mid x)\) is a probability distribution over actions, and for each measurable \(B\subseteq\mathcal A\), \(\delta(B\mid x)\) is measurable in \(x\).
The induced joint probability measure on \(\mathcal X\times\mathcal A\) is \[ Q_{\theta,\delta}(dx,da)=P_\theta(dx)\delta(da\mid x). \] For a measurable loss \(\ell_\theta(x,a)\), the risk is the extended expectation \[ R(\theta,\delta)=\int_{\mathcal X\times\mathcal A}\ell_\theta(x,a)\,Q_{\theta,\delta}(dx,da). \] We focus on losses bounded below by a finite constant, so this integral is well-defined as an extended real number and cannot involve the indeterminate form \(+\infty-\infty\).
For probability measures \(Q_n\) and \(Q\) on a metric space, \(Q_n\) converges weakly to \(Q\) if \(\int h\,dQ_n\to\int h\,dQ\) for every bounded continuous real-valued function \(h\). In the result below, it is the induced joint laws that must converge weakly. This specifies exactly which convergence is needed to control the risk integral.
The Lower-Semicontinuity Theorem
The proof uses the open-set consequence of weak convergence. We give the needed argument before applying it to the loss, so that the role of lower semicontinuity is explicit.
Proof. If \(G=S\), the assertion is equality. Otherwise define \(h_k(z)=\min\{1,k\,d(z,G^c)\}\), where \(d(z,G^c)\) is the distance to the closed complement. Each \(h_k\) is bounded and continuous, \(0\leq h_k\leq \mathbf 1_G\), and \(h_k(z)\) increases to \(\mathbf 1_G(z)\). For each fixed \(k\), weak convergence gives \[ \int h_k\,dQ=\lim_{n\to\infty}\int h_k\,dQ_n\leq\liminf_{n\to\infty}Q_n(G). \] By monotone convergence, \(\int h_k\,dQ\) increases to \(Q(G)\). Taking the limit in \(k\) proves the inequality. \(\square\)
Proof of the theorem. Set \(u=g-c\), so \(u\geq0\) and \(u\) is lower semicontinuous. For every \(t\geq0\), the set \(\{z:u(z)>t\}\) is open. The lemma therefore gives \[ Q\{u>t\}\leq\liminf_{n\to\infty}Q_n\{u>t\}. \] For a nonnegative measurable function, the layer-cake identity, obtained by integrating \(\mathbf 1_{\{u(z)>t\}}\) over \(t\) and applying Tonelli's theorem, is \[ \int u\,dQ=\int_0^\infty Q\{u>t\}\,dt. \] Apply Fatou's lemma to the nonnegative functions \(t\mapsto Q_n\{u>t\}\). It follows that \[ \int u\,dQ \leq\int_0^\infty\liminf_{n\to\infty}Q_n\{u>t\}\,dt \leq\liminf_{n\to\infty}\int_0^\infty Q_n\{u>t\}\,dt =\liminf_{n\to\infty}\int u\,dQ_n. \] Adding \(c\) to each expectation gives the stated inequality for \(g\). This remains valid if an integral is \(+\infty\), since \(u\) is nonnegative. \(\square\)
For decision rules, fix \(\theta\). Suppose the induced joint laws \(Q_{\theta,\delta_n}\) converge weakly to \(Q_{\theta,\delta}\), and suppose \(\ell_\theta\) is lower semicontinuous and bounded below on \(\mathcal X\times\mathcal A\). Applying the theorem directly yields
The statement applies separately at each fixed \(\theta\); it does not claim uniformity over the parameter space. Compactness of the action space is not needed for the integral inequality itself. It is useful in compactness arguments that produce convergent subsequences of decision-rule laws.
Worked Examples
Worked Example: A Strict Risk Inequality on a Compact Action Space
Let the observation space contain a single point, let \(\mathcal A=[0,1]\), and define \(\ell(a)=\mathbf 1_{\{a>0\}}\). This loss is bounded below and lower semicontinuous: at \(a=0\) its value is \(0\), no greater than the lower limit of nearby values, and at every \(a>0\) it is locally constant.
For each \(n\), take the randomized rule with action distribution \(Q_n=\delta_{1/n}\), and let \(Q=\delta_0\). For any bounded continuous \(\varphi\), \(\int\varphi\,dQ_n=\varphi(1/n)\to\varphi(0)=\int\varphi\,dQ\), so \(Q_n\) converges weakly to \(Q\). The risks are \[ \int\ell\,dQ_n=\ell(1/n)=1,\qquad \int\ell\,dQ=\ell(0)=0. \] Thus \(0\leq\liminf_n 1=1\), and the inequality is strict. Lower semicontinuity ensures the required direction, not equality or continuity of risk.
Worked Example: A Statistical Model With Continuous Loss
Let \(X\sim\operatorname{Bernoulli}(1/2)\), let \(\mathcal A=[0,1]\), and use loss \(\ell(x,a)=(a-x)^2\). For \(n\geq4\), define a deterministic rule by \(a_n(0)=1/4+1/n\) and \(a_n(1)=3/4-1/n\). Its limit rule uses \(a(0)=1/4\) and \(a(1)=3/4\). The loss is continuous, hence lower semicontinuous, and is bounded below by zero.
The joint law for rule \(n\) puts probability \(1/2\) on each of \((0,1/4+1/n)\) and \((1,3/4-1/n)\). These two support points converge to the corresponding points for the limit rule. Consequently, for every bounded continuous \(h\), the two terms in its joint expectation converge to the two terms under the limit law; this verifies weak convergence directly.
At \(X=0\), the squared loss is \((1/4+1/n)^2\). At \(X=1\), it is \((1-(3/4-1/n))^2=(1/4+1/n)^2\). Therefore \[ R(\delta_n)=\tfrac12(1/4+1/n)^2+\tfrac12(1/4+1/n)^2=(1/4+1/n)^2. \] For the limit rule, both possible observations give squared loss \(1/16\), so \(R(\delta)=1/16\). Since \((1/4+1/n)^2\to1/16\), the inequality holds with equality in the limit.
Worked Example: Lower-Bounded Does Not Mean Bounded Above
Again take \(\mathcal A=[0,1]\), and define the extended-valued loss by \(\ell(0)=+\infty\) and \(\ell(a)=1/a\) for \(a>0\). This loss is lower semicontinuous: it is continuous away from zero, and as \(a\) approaches zero through positive values, \(1/a\) tends to \(+\infty=\ell(0)\). It is bounded below by \(1\).
With \(Q_n=\delta_{1/n}\) and \(Q=\delta_0\), the measures converge weakly as in the first example. Their expected losses are \(\int\ell\,dQ_n=n\) and \(\int\ell\,dQ=+\infty\). Thus the theorem gives \(+\infty\leq\liminf_n n=+\infty\), valid as an extended-real inequality. This example shows why an upper bound on the loss is not part of the theorem; it also shows that the conclusion can be uninformative when the limiting risk is infinite.
When the Hypothesis Fails
Worked Example: A Bounded-Below Loss That Is Not Lower Semicontinuous
On \(\mathcal A=[0,1]\), let \(\ell(a)=\mathbf 1_{\{a=0\}}\). It is bounded below by zero, but it is not lower semicontinuous at zero: \(\ell(0)=1\), whereas \(\ell(1/n)=0\) for every \(n\). Let \(Q_n=\delta_{1/n}\) and \(Q=\delta_0\). As before, \(Q_n\) converges weakly to \(Q\), because \(\varphi(1/n)\to\varphi(0)\) for every bounded continuous \(\varphi\). Yet \[ \int\ell\,dQ_n=0\quad\text{for every }n,\qquad \int\ell\,dQ=1. \] The proposed inequality would say \(1\leq0\), which is false. The failure occurs precisely at a downward discontinuity of the loss along the approximating actions.
A common mistake is to infer convergence of risk merely from weak convergence of the decision-rule laws. Weak convergence guarantees convergence of integrals for bounded continuous functions, not for every measurable loss. Lower semicontinuity and a finite lower bound give a specific substitute: the limit risk is at most the lower limit of the sequence of risks. Replacing that inequality with equality, or omitting lower semicontinuity, can reverse the conclusion, as the examples show.
Check Your Understanding
Use the definitions and theorem to answer the following.
- Which probability measures must converge weakly for the risk inequality to apply to randomized decision rules?
- Why is the strict superlevel set \(\{z:g(z)>t\}\) useful in proving the theorem?
- In the first worked example, why does the risk of the limit rule differ from the risks along the sequence without contradicting the theorem?
- Does the theorem require the loss to be bounded above? Explain using the third worked example.
- For the loss \(\mathbf 1_{\{a=0\}}\), identify the failed hypothesis and the exact inequality that fails.