Unbounded Losses and Failure of Compactness

Compact Rules Do Not Guarantee Controlled Risks

In complete-class arguments, compactness supplies convergent subsequences of decision rules. To use such a subsequence, one must also know what happens to its risks. Bounded loss makes this step manageable: the contribution from any event with small probability is uniformly small. With unbounded loss, a sequence of rules can place its risk on rarer and rarer observations, so the rules converge while their risks do not converge.

The distinction matters. Under suitable lower-semicontinuity assumptions, a nonnegative unbounded loss can still give a lower-semicontinuous risk, which is enough to pass certain pointwise risk upper bounds to a limit. What unboundedness does not provide is uniform control of tail contributions, or continuity of risk along convergent rules. The examples below isolate that failure without relying on a noncompact action space.

Definition: A family of nonnegative random variables $\{Y_\alpha\}$ is uniformly integrable if $$ \lim_{M\to\infty}\ \sup_\alpha E\!\left[Y_\alpha \mathbf{1}_{\{Y_\alpha>M\}}\right]=0. $$ Uniform integrability says that large values make a uniformly negligible contribution to expectations. A uniform bound on $E[Y_\alpha]$ alone does not imply this condition.

For a rule $d$, the risk at parameter $\theta$ is $R(\theta,d)=E_\theta[L(\theta,d(X))]$, where $L$ is the loss. The compactness issue considered here is not whether a sequence of rules has a convergent subsequence; it is whether the risks along that sequence have the control needed by the proof. In particular, replacing an unbounded loss by bounded truncations is useful only if the truncation error is small uniformly over the rules under consideration.

A Compact Rule Sequence with Persistent Risk

Take a single parameter value and an observation $X$ on the positive integers, with $P(X=k)=2^{-k}$ for $k=1,2,\ldots$. These probabilities sum to one. The action space is the compact finite set $\{0,1\}$, and the loss is $L(x,a)=2^x a$. It is nonnegative and finite at every observation-action pair, but unbounded as $x$ increases.

Worked Example: Risk Escapes to Rare Observations

For each integer $n\geq 1$, define the deterministic rule $d_n(x)=1$ if $x=n$, and $d_n(x)=0$ otherwise. For every fixed $x$, $d_n(x)$ is eventually zero, so the rules converge pointwise to $d(x)=0$. With a finite action space, this is also pointwise convergence of the corresponding probability kernels.

The risk of the limit rule is zero because it always takes action 0: $R(d)=E[2^X d(X)]=0$. For each rule in the sequence, only the observation $X=n$ incurs loss. Thus $$ R(d_n)=E[2^X d_n(X)] =P(X=n)2^n =2^{-n}2^n =1. $$ Consequently, $R(d_n)=1$ for every $n$, while $R(d)=0$. The rules converge, and their risks are bounded, but their risks do not converge to the risk of the limit rule.

The contribution of the risk sits at an observation whose probability tends to zero, while the loss there grows at exactly the rate needed to keep the expectation equal to one. Bounded risks have not prevented this migration into the tail.

The example also identifies the failed uniform step in a truncation argument. Write $L_M(x,a)=\min\{L(x,a),M\}$. For any $n$ with $2^n>M$, the truncation error under $d_n$ is $$ E[L(X,d_n(X))-L_M(X,d_n(X))] =2^{-n}(2^n-M) =1-M2^{-n}. $$ For every fixed $M$, these errors approach one as $n$ increases. Therefore, bounded truncations approximate the loss for each fixed rule, but not uniformly over this sequence of rules. One cannot first choose a truncation level that works for all the rules and then apply a bounded-loss argument.

What Lower-Semicontinuity Does—and Does Not—Give

It would be too strong to conclude that unbounded losses always invalidate the risk step in a complete-class proof. For nonnegative lower-semicontinuous losses, appropriate convergence of the decision rules and the usual measurability and integration conditions can still yield lower semicontinuity of risk. The basic mechanism is Fatou's lemma: when the losses converge pointwise, the expectation of their pointwise lower limit cannot exceed the lower limit of their expectations. More general weak-convergence versions require the corresponding lower-semicontinuity hypotheses.

This one-sided inequality has an important use. Suppose $R(\theta,d_n)\leq c(\theta)$ for every $n$, for a fixed parameter $\theta$, and lower semicontinuity gives $R(\theta,d)\leq\liminf_n R(\theta,d_n)$. Since every term is at most $c(\theta)$, the lower limit is also at most $c(\theta)$, and hence $R(\theta,d)\leq c(\theta)$. Thus pointwise risk domination can survive even when risk convergence fails. Lower semicontinuity does not, however, establish that the risks converge, nor does it by itself supply a bound on truncation errors that is uniform over a family of rules.

Worked Example: Bounded Risks Are Not Uniform Integrability

Return to the geometric observation model and the rules $d_n$. Let $Y_n=L(X,d_n(X))$, the loss as a random variable under rule $d_n$. It equals $2^n$ when $X=n$, an event of probability $2^{-n}$, and equals zero otherwise. Its expectation is $E[Y_n]=1$ for every $n$.

Fix any $M>0$. For all sufficiently large $n$, $2^n>M$, and then $$ E[Y_n\mathbf{1}_{\{Y_n>M\}}] =2^{-n}2^n =1. $$ The supremum of these tail expectations is therefore at least one for every $M$. It does not approach zero as $M$ increases, so $\{Y_n\}$ is not uniformly integrable, despite its uniformly bounded expectations.

A moment calculation exposes the same problem. For any $\delta>0$, $E[Y_n^{1+\delta}]=2^{-n}(2^n)^{1+\delta}=2^{n\delta}$, which diverges with $n$. The sequence has no uniform bound on any moment of order greater than one.

Conditions That Restore Risk Convergence

A standard sufficient condition is uniform integrability of the losses, together with convergence of those losses in probability. The following result is stated for a fixed parameter; its expectation is then the risk at that parameter. The convergence-in-probability condition is separate from uniform integrability: it describes the behavior of the losses themselves, while uniform integrability controls their tails.

Theorem: Suppose $Y_n\geq0$, $Y_n\xrightarrow{p}Y$, and $\{Y_n:n\geq1\}$ is uniformly integrable. Then $E[Y]<\infty$ and $E[Y_n]\to E[Y]$.

Proof. First, convergence in probability allows a subsequence of any subsequence of $Y_n$ to converge to $Y$ almost surely. Fatou's lemma applied to such a subsequence gives $E[Y]\leq\liminf E[Y_{n_j}]$. Uniform integrability implies $\sup_n E[Y_n]<\infty$: choose $M$ so the tail expectations are at most one, and use $E[Y_n]\leq M+1$. The subsequence argument therefore shows $E[Y]<\infty$.

For $M>0$, put $Y_n^{(M)}=\min(Y_n,M)$ and $Y^{(M)}=\min(Y,M)$. The map $y\mapsto\min(y,M)$ is bounded and continuous on $[0,\infty)$, so convergence in probability implies $E[Y_n^{(M)}]\to E[Y^{(M)}]$. The truncation errors for the sequence satisfy $0\leq E[Y_n]-E[Y_n^{(M)}]\leq E[Y_n\mathbf{1}_{\{Y_n>M\}}]$, uniformly in $n$; their supremum tends to zero by uniform integrability. Also $E[Y-Y^{(M)}]\to0$ as $M\to\infty$, by integrability of $Y$. Taking $n\to\infty$ at fixed $M$, then $M\to\infty$, proves $E[Y_n]\to E[Y]$. $\square$

In a decision problem, this result applies at a fixed $\theta$ when the loss random variables $L(\theta,d_n(X))$ converge in probability to $L(\theta,d(X))$ and are uniformly integrable under $P_\theta$. For convergence uniformly over a parameter set, these conditions must be strengthened: tail expectations must be controlled uniformly over both the rules and the parameters, and the bounded, truncated risks must also converge uniformly over the parameters. Pointwise conditions alone do not justify a uniform conclusion.

Worked Example: A Gaussian Moment Bound Controls the Tail

Let $X\sim N(0,1)$, let the action space again be $\{0,1\}$, and take $L(x,a)=x^2a$. Define $d_n(x)=\mathbf{1}_{\{x>n\}}$, for positive integers $n$. For every fixed real $x$, $d_n(x)\to0$, and the loss under $d_n$ is $Y_n=X^2\mathbf{1}_{\{X>n\}}$. Since $0\leq Y_n\leq X^2$ and $E[X^2]=1$, the entire sequence is dominated by an integrable random variable. In particular, its tails are uniformly controlled: $$ E[Y_n\mathbf{1}_{\{Y_n>M\}}] \leq E[X^2\mathbf{1}_{\{X^2>M\}}]\longrightarrow 0. $$ The limit rule is zero and has risk zero. Directly, writing $\phi$ and $\Phi$ for the standard normal density and distribution function, integration by parts gives $$ R(d_n)=\int_n^\infty x^2\phi(x)\,dx =n\phi(n)+1-\Phi(n)\longrightarrow 0. $$ Here the risks do converge: the normal second moment supplies an integrable envelope, ruling out the persistent tail contribution in the geometric example.

Common Pitfall: Confusing a Risk Bound with Tail Control

A common but incorrect argument is: “The risks $E[Y_n]$ are all at most one, so the tails of the losses must be uniformly negligible; therefore the risks converge when the rules converge.” The geometric example disproves the first implication: every risk equals one, yet for every truncation level the tail contribution remains one along sufficiently large $n$. A bounded first moment does not imply uniform integrability.

The correction is to identify exactly what the proof needs. If the goal is only to pass pointwise domination to a limit, nonnegativity and lower semicontinuity may be enough. If the goal requires convergence of risks or uniform approximation by bounded losses, supply a tail condition, such as uniform integrability. A useful sufficient moment condition is $\sup_n E[Y_n^{1+\delta}]<\infty$ for some $\delta>0$, which implies $$ E[Y_n\mathbf{1}_{\{Y_n>M\}}] \leq \frac{E[Y_n^{1+\delta}]}{M^\delta}. $$ The right-hand side tends to zero uniformly in $n$. In applications, one must verify that the moment bound holds under the relevant sampling distributions and, when needed, uniformly over the parameter set.

Key takeaway: Compactness of decision rules and control of their risks are separate requirements. With unbounded loss, bounded risks alone do not control tail contributions. Lower semicontinuity can still preserve pointwise risk bounds, while risk convergence and uniform truncation arguments require additional tail control, commonly uniform integrability or a suitable higher-moment bound.

Check Your Understanding

Use the distinction between compactness, lower semicontinuity, and tail control in your answers.

In the geometric-observation example, why do the rules converge pointwise even though their risks do not converge to the limit risk?
For $Y_n$ in the second worked example, calculate $E[Y_n]$ and $E[Y_n\mathbf{1}_{\{Y_n>M\}}]$ when $2^n>M$. What does this show?
Why can lower semicontinuity still pass a pointwise risk upper bound to a limit without implying convergence of risks?
State a moment condition sufficient for uniform integrability and give the resulting upper bound on the tail expectation.
What extra uniformity is needed before pointwise-in-$\theta$ risk convergence can support a conclusion uniform over a parameter set?