Passing Risk Domination Through Limits

Why Domination Must Be Checked at the Limit

In complete-class arguments, one often constructs a sequence of rules that is no worse than a fixed benchmark at every parameter value. Compactness may then provide a convergent subsequence and a limiting rule. The remaining question is essential: does the limit retain every one of those risk inequalities? Convergence of rules alone does not answer this. A suitable lower-semicontinuity condition on risk supplies the needed direction of inequality.

The transfer is coordinatewise: each parameter value has its own risk comparison. It does not require risk convergence, uniform convergence over the parameter space, or a countable parameter space. The key is that the risk of the limit cannot exceed the lower limit of the sequence of risks.

Setup and the Transfer Theorem

Let $(\mathcal X,\mathcal A)$ be the sample space, let $\{P_\theta:\theta\in\Theta\}$ be a statistical model, and let $(\mathcal D,\mathcal B)$ be the action space. A randomized decision rule $\delta$ is a Markov kernel from $\mathcal X$ to $\mathcal D$: for each observation $x$, $\delta(\,\cdot\mid x)$ is a probability measure on actions, and for each measurable action set $B$, $\delta(B\mid x)$ is measurable in $x$. Let $L(\theta,x,a)$ be a nonnegative measurable loss. The risk is

R(\theta,\delta) = \int_{\mathcal X}\int_{\mathcal D} L(\theta,x,a)\,\delta(da\mid x)\,P_\theta(dx).

Write $Q_{\theta,\delta}$ for the joint probability distribution of $(X,A)$ when $X\sim P_\theta$ and, conditionally on $X=x$, the action $A$ is drawn from $\delta(\,\cdot\mid x)$. Thus $R(\theta,\delta)=\int L(\theta,x,a)\,Q_{\theta,\delta}(dx,da)$, with the integral allowed to be infinite.

Definition: A sequence of rules $\delta_n$ has a risk-dominating limit relative to a benchmark rule $\delta_*$ if $R(\theta,\delta)\leq R(\theta,\delta_*)$ for every $\theta\in\Theta$, where $\delta$ is the limiting rule. A function $g$ is lower semicontinuous if $g(z)\leq\liminf_n g(z_n)$ whenever $z_n\to z$.

Here is a useful sufficient condition for lower semicontinuity of risk. Suppose $\mathcal X\times\mathcal D$ is a metric space, and for each fixed $\theta$, $L(\theta,\cdot,\cdot)$ is nonnegative and lower semicontinuous. If $Q_{\theta,\delta_n}$ converges weakly to $Q_{\theta,\delta}$ for each $\theta$, Portmanteau gives $R(\theta,\delta)\leq\liminf_n R(\theta,\delta_n)$. More generally, the transfer theorem below applies whenever this risk inequality is known, regardless of how it was obtained.

Theorem: Let $\delta_n$ be a sequence of rules and $\delta$ a rule such that, for every $\theta\in\Theta$, $$ R(\theta,\delta)\leq\liminf_{n\to\infty}R(\theta,\delta_n). $$ If $B(\theta)\in[0,\infty]$ is a fixed benchmark and $R(\theta,\delta_n)\leq B(\theta)$ for every $n$ and every $\theta$, then $R(\theta,\delta)\leq B(\theta)$ for every $\theta$. In particular, the conclusion holds with $B(\theta)=R(\theta,\delta_*)$.

Proof. Fix an arbitrary $\theta$. The assumed bound on every term of the sequence implies $\liminf_n R(\theta,\delta_n)\leq B(\theta)$. Combining this with the assumed lower-semicontinuity inequality gives

R(\theta,\delta) \leq \liminf_{n\to\infty}R(\theta,\delta_n) \leq B(\theta).

Since $\theta$ was arbitrary, the inequality holds throughout $\Theta$. $\square$

Worked Examples

Worked Example: A Finite Decision Problem

There is no informative observation, the parameter space is $\Theta=\{0,1\}$, and the available actions are $u,v,c$. The loss table is

Action	$\theta=0$	$\theta=1$
$u$	0	2
$v$	2	0
$c$	$5/4$	$5/4$

For each integer $n\geq1$, let $\delta_n$ choose $u$ with probability $p_n=1/2+1/(4n)$, and choose $v$ otherwise. The benchmark $\delta_*$ always chooses $c$. At $\theta=0$, the risk is

R(0,\delta_n)=0p_n+2(1-p_n) =2\left(\frac12-\frac{1}{4n}\right) =1-\frac{1}{2n} \leq 1 < \frac54=R(0,\delta_*).

At $\theta=1$, it is

R(1,\delta_n)=2p_n+0(1-p_n) =2\left(\frac12+\frac{1}{4n}\right) =1+\frac{1}{2n} \leq \frac32.

That last bound is not enough to establish domination by $5/4$ for every $n$: at $n=1$, the risk is $3/2$. Thus this proposed sequence does not meet the theorem’s hypotheses. To obtain a valid example, instead set $p_n=1/2+1/(8n)$. Then $p_n\in[0,1]$, and the two risks are

R(0,\delta_n)=1-\frac{1}{4n}\leq 1<\frac54, \qquad R(1,\delta_n)=1+\frac{1}{4n}\leq\frac54,

where the second inequality follows from $1/(4n)\leq1/4$. Hence $\delta_n$ is no worse than $\delta_*$ at both parameter values. As $n\to\infty$, $p_n\to1/2$; the limiting rule chooses $u$ and $v$ with equal probabilities. Its risk is $1$ at each parameter value, so it also satisfies both benchmark inequalities.

Worked Example: One Randomized Rule Dominates Across an Interval

Let $\Theta=[0,1]$, use squared-error loss $L(\theta,a)=(a-\theta)^2$, and suppose there is no informative observation. The benchmark randomizes equally between actions $0.3$ and $0.7$. For each $n\geq1$, define $\delta_n$ to randomize equally between $a_{n,-}=0.5-0.2(1-1/n)$ and $a_{n,+}=0.5+0.2(1-1/n)$. All these actions lie in $[0,1]$.

For any random action $A$, squared-error risk is $E[(A-\theta)^2]=\operatorname{Var}(A)+(E[A]-\theta)^2$. The benchmark has mean $0.5$ and variance $0.2^2=0.04$. Rule $\delta_n$ has mean $0.5$ and variance $0.04(1-1/n)^2$. Therefore, for every $\theta\in[0,1]$,

R(\theta,\delta_n) =(0.5-\theta)^2+0.04(1-1/n)^2 \leq (0.5-\theta)^2+0.04 =R(\theta,\delta_*).

The two support points converge to $0.3$ and $0.7$, so the limiting rule is exactly the benchmark rule and retains the risk bound at every $\theta$. This calculation illustrates that a single sequence of inequalities can hold over an uncountable parameter space; no separate uniform-convergence argument is needed.

Worked Example: Weak Convergence and a Continuous Loss

Consider one parameter value, no informative observation, action space $[0,1]$, and loss $L(a)=a^2$. Let $U$ have the uniform distribution on $[0,1]$. For $n\geq2$, define the randomized rule by the action $A_n=(1-1/n)U$. Let the benchmark action be $A_*=U$. Using $E[U^2]=\int_0^1 u^2\,du=1/3$, the risks are

R(\delta_n)=E[A_n^2] =(1-1/n)^2E[U^2] =\frac{(1-1/n)^2}{3} \leq\frac13 =E[A_*^2] =R(\delta_*).

Under this coupling, $A_n\to U$ almost surely, so the action distributions converge weakly to the benchmark distribution. The limiting risk is $E[U^2]=1/3$, and the bound is retained. Here the action loss is continuous and bounded on $[0,1]$, so risk in fact converges; the theorem needs only the weaker lower-semicontinuity inequality.

Uniform Bounds and What the Theorem Does Not Say

Corollary: If $R(\theta,\delta)\leq\liminf_nR(\theta,\delta_n)$ for every $\theta$, and $\sup_{\theta\in\Theta}R(\theta,\delta_n)\leq C$ for every $n$, where $C<\infty$, then $\sup_{\theta\in\Theta}R(\theta,\delta)\leq C$.

Proof. For each fixed $\theta$, every term $R(\theta,\delta_n)$ is at most $C$, so its lower limit is at most $C$. The assumed inequality gives $R(\theta,\delta)\leq C$. Taking the supremum over $\theta$ proves the claim. $\square$

Neither result says that the limiting rule strictly improves on the benchmark, or that its risk is the limit of the sequence of risks. Lower semicontinuity gives a comparison with the lower limit, which is exactly the direction needed for domination. A common error is to infer that weak convergence alone preserves expectations for any loss. Weak convergence guarantees convergence of integrals for bounded continuous functions, not arbitrary discontinuous losses.

Worked Example: Failure When Loss Is Not Lower Semicontinuous

There is one parameter value, no observation, action space $\mathbb R$, and loss $L(a)=1$ if $a=0$, and $L(a)=0$ otherwise. Let $\delta_n$ choose action $1/n$ deterministically, and let the benchmark $\delta_*$ choose action $1$ deterministically. Every sequence risk and the benchmark risk is zero: $R(\delta_n)=L(1/n)=0= L(1)=R(\delta_*)$. The actions converge to $0$, so the limiting rule chooses action $0$ and has risk $L(0)=1$, which exceeds the benchmark risk $0$. The loss is not lower semicontinuous at $0$, because $L(0)=1>\liminf_n L(1/n)=0$. Thus the risk inequality required by the theorem fails, and the bound does not pass to the limit.

Using the Transfer Correctly

In a complete-class proof, the transfer step should be separated from the construction of the convergent sequence. First establish that the proposed limit is a valid rule and that risk is lower semicontinuous along the chosen convergence. Then record the benchmark inequality for every member of the sequence, at each parameter value. The theorem combines those facts without requiring the risks themselves to converge.

Fix a parameter value.
Use the same arbitrary $\theta$ in the limit inequality and in the sequence’s benchmark bounds.

Take the lower limit.
From $R(\theta,\delta_n)\leq B(\theta)$ for all $n$, conclude $\liminf_nR(\theta,\delta_n)\leq B(\theta)$.

Apply lower semicontinuity.
Combine $R(\theta,\delta)\leq\liminf_nR(\theta,\delta_n)$ with the preceding bound, then use the arbitrariness of $\theta$.

The order of the inequalities matters. Replacing lower semicontinuity with an unsupported assertion that risks converge, or checking the sequence bound only approximately at some parameter values, leaves a gap. When the hypotheses do hold, this simple coordinatewise argument is what allows domination to survive the passage from approximating rules to a limiting rule.

Check Your Understanding

For each question, identify the inequality or hypothesis that matters.

If $R(\theta,\delta)\leq\liminf_nR(\theta,\delta_n)$ and each sequence risk is at most $B(\theta)$, which two inequalities give the limit bound?
Does the transfer theorem require $\Theta$ to be countable? Explain why or why not.
In the interval example, calculate the benchmark risk at $\theta=0$ and the risk of $\delta_n$ there.
Why does weak convergence suffice for the uniform-action example, and which property of its loss is relevant?
In the discontinuous-loss example, which lower-semicontinuity inequality fails at the limit action?

Action	\(\theta=0\)	\(\theta=1\)
\(u\)	0	2
\(v\)	2	0
\(c\)	\(5/4\)	\(5/4\)