Risk Truncation for Unbounded Losses

Why Truncate an Unbounded Loss?

Minimax analysis compares procedures by their worst-case risks. When the loss is unbounded, a small amount of probability on very large losses can contribute substantially to risk. Replacing the loss by a finite cap makes each loss bounded, but also changes the risk and potentially the minimax value. The key question is not merely whether the capped loss approaches the original loss pointwise; it is how much expected loss lies above the cap.

We make that comparison directly. The results below require no existence of a minimax rule. They apply to any fixed decision class, provided the loss is measurable and nonnegative; risks and minimax values may initially be extended-real numbers.

Truncated Loss and Tail Excess

Definition: Let \(L(\theta,a)\geq 0\) be a measurable loss, where \(\theta\) is the parameter and \(a\) is the action. For a finite threshold \(M>0\), define the capped loss by \(L_M(\theta,a)=\min\{L(\theta,a),M\}\). For a decision rule \(\delta\), its risks under the original and capped losses are \(R(\theta,\delta)=E_\theta[L(\theta,\delta(X))]\) and \(R_M(\theta,\delta)=E_\theta[L_M(\theta,\delta(X))]\). The tail excess risk is \(\tau_M(\theta,\delta)=E_\theta[(L(\theta,\delta(X))-M)_+]\), where \(x_+=\max\{x,0\}\).

All expectations here are under the sampling distribution at \(\theta\); the loss can be random through the observation and the rule. The pointwise identity \(L-L_M=(L-M)_+\) gives the exact risk decomposition \(R(\theta,\delta)=R_M(\theta,\delta)+\tau_M(\theta,\delta)\), including when an expectation is infinite. In particular, capping never increases risk. Also, \((L-M)_+\leq L\,\mathbf{1}_{\{L>M\}}\), so an upper bound on the expected loss beyond the threshold also bounds the tail excess.

Worked Example: Exact Tail Excess for an Exponential Loss

Suppose the loss incurred by a particular rule is a random variable \(Z\sim\operatorname{Exp}(1)\), with density \(f(z)=e^{-z}\) for \(z\geq 0\). Use the cap \(M=2\) to calculate the original expected loss, the capped expected loss, and the tail excess. Here \(\operatorname{Exp}(1)\) denotes the exponential distribution with rate 1.

The original expectation is \(E[Z]=\int_0^\infty ze^{-z}\,dz=1\). The capped loss equals \(z\) for \(0\leq z\leq2\) and 2 for \(z>2\), so

E[\min\{Z,2\}] =\int_0^2 ze^{-z}\,dz+2\int_2^\infty e^{-z}\,dz =\bigl[ -(z+1)e^{-z}\bigr]_0^2+2e^{-2} =1-e^{-2}.

The excess is zero when \(Z\leq2\), and is \(Z-2\) otherwise. Therefore

E[(Z-2)_+] =\int_2^\infty (z-2)e^{-z}\,dz =e^{-2}\int_0^\infty u e^{-u}\,du =e^{-2}.

Thus \(E[Z]=1=(1-e^{-2})+e^{-2}\), exactly as the risk decomposition requires. The probability \(P(Z>2)=e^{-2}\) happens to equal the excess here, but that equality is not general: a tail probability alone does not measure how large the losses above the cap are.

Minimax Bounds

For a fixed class \(\mathcal{D}\) of measurable decision rules, define the original and truncated minimax values by \(V=\inf_{\delta\in\mathcal{D}}\sup_{\theta\in\Theta}R(\theta,\delta)\) and \(V_M=\inf_{\delta\in\mathcal{D}}\sup_{\theta\in\Theta}R_M(\theta,\delta)\). The same parameter space and rule class are used in both definitions.

Theorem: For every \(M>0\), \(V_M\leq V\). For each rule \(\delta\), let \(\tau_M(\delta)=\sup_{\theta\in\Theta}\tau_M(\theta,\delta)\). Then \[ \sup_{\theta\in\Theta}R(\theta,\delta) \leq \sup_{\theta\in\Theta}R_M(\theta,\delta)+\tau_M(\delta). \] Consequently, if \(\tau_M(\delta)\leq\varepsilon_M\) for every \(\delta\in\mathcal{D}\), where \(\varepsilon_M<\infty\), then \(V_M\leq V\leq V_M+\varepsilon_M\).

The first inequality is unconditional: it follows from capping the loss, not from any tail assumption. The upper bound on the minimax value requires control of the excess in a form that applies across the rule class, or at least to rules close to optimal for the truncated problem.

Proof. Since \(L_M\leq L\), taking expectations gives \(R_M(\theta,\delta)\leq R(\theta,\delta)\) for every \(\theta\) and \(\delta\). Taking suprema over \(\theta\), then infima over \(\delta\), proves \(V_M\leq V\).

For a fixed rule, the exact risk decomposition gives \(R(\theta,\delta)=R_M(\theta,\delta)+\tau_M(\theta,\delta)\). By definition, \(\tau_M(\theta,\delta)\leq\tau_M(\delta)\). Taking the supremum over \(\theta\) therefore yields \(\sup_\theta R(\theta,\delta)\leq\sup_\theta R_M(\theta,\delta)+\tau_M(\delta)\). Under the stated uniform bound, this is at most \(\sup_\theta R_M(\theta,\delta)+\varepsilon_M\). Taking the infimum over \(\delta\) gives \(V\leq V_M+\varepsilon_M\), because adding the same finite constant commutes with the infimum. Together with the lower bound, this proves the claim. \(\square\)

Worked Example: Exact Values in a Two-State Game

There are two parameter states, \(\theta_1,\theta_2\), and two actions, \(a,b\). Loss is deterministic given the state and action, with table

State	Action \(a\)	Action \(b\)
\(\theta_1\)	0	12
\(\theta_2\)	12	0

Allow randomized rules and let \(p\) be the probability of choosing \(a\). At \(\theta_1\), the risk is \(12(1-p)\); at \(\theta_2\), it is \(12p\). The worst-case risk is \(12\max\{p,1-p\}\), minimized at \(p=1/2\), so \(V=6\).

With cap \(M=5\), each loss of 12 becomes 5. The two capped risks are \(5(1-p)\) and \(5p\), so the truncated worst-case risk is \(5\max\{p,1-p\}\), also minimized at \(p=1/2\). Hence \(V_M=2.5\).

For the rule \(p=1/2\), at either state the loss is 12 with probability \(1/2\) and zero otherwise. Its tail excess is therefore \(\tau_M(\theta_j,\delta)=\tfrac12(12-5)=3.5\) at each state. The rule-specific bound gives \(V\leq 2.5+3.5=6\), which is exact. By contrast, the largest tail excess over all rules is 7, so the uniform bound would give the valid but looser inequality \(V\leq2.5+7=9.5\).

Rule-Specific Control Without a Uniform Bound

A bound over every rule is sufficient but not always necessary. If a truncated minimax value is finite, the definition of an infimum guarantees rules with truncated worst-case risk arbitrarily close to it, even when no rule attains the infimum. Tail control for those near-optimal rules alone can then bound the original minimax value.

Corollary: Suppose \(V_M<\infty\). For every \(\eta>0\), there exists a rule \(\delta_\eta\in\mathcal{D}\) such that \[ V_M\leq V\leq V_M+\eta+\tau_M(\delta_\eta). \] In particular, if such a rule can be chosen with \(\tau_M(\delta_\eta)\leq b_M\), then \(V\leq V_M+\eta+b_M\).

Proof. Since \(V_M\) is the infimum of the truncated worst-case risks and is finite, for each \(\eta>0\) there exists \(\delta_\eta\) with \(\sup_\theta R_M(\theta,\delta_\eta)<V_M+\eta\). The rule-specific bound gives \(V\leq\sup_\theta R(\theta,\delta_\eta)\leq\sup_\theta R_M(\theta,\delta_\eta)+\tau_M(\delta_\eta)\). Combining these inequalities proves the displayed bound (with a weak inequality after replacing the strict bound by a weak one). \(\square\)

Worked Example: Using a Near-Optimal Rule's Tail Bound

Suppose \(V_M=4\) for a chosen threshold, but no truncated minimizer is known to exist. For every \(\eta>0\), choose a rule \(\delta_\eta\) with truncated worst-case risk less than \(4+\eta\). Assume that for these chosen rules the worst-case tail excess is at most \(0.3\). Then

V\leq \sup_\theta R(\theta,\delta_\eta) <4+\eta+0.3.

Because this holds for every \(\eta>0\), it follows that \(V\leq4.3\): if \(V>4.3\), choosing \(\eta\) smaller than \(V-4.3\) would contradict the displayed bound. Also \(V\geq V_M=4\). Thus \(4\leq V\leq4.3\), without assuming that any rule attains \(V_M\), and without requiring tail control for irrelevant rules.

Why Pointwise Approximation Can Fail

Worked Example: Truncated Values Stay Bounded While the Original Value Is Infinite

Let the parameter space be the positive integers \(\Theta=\{1,2,\ldots\}\), and suppose every rule incurs deterministic loss \(L(\theta,\delta)=\theta\), regardless of the observation or action. For every fixed \(\theta\), \(L_M(\theta,\delta)=\min\{\theta,M\}\) increases to \(\theta\) as \(M\) increases. But the original worst-case risk is \(\sup_{\theta\geq1}\theta=\infty\), while the truncated worst-case risk is \(\sup_{\theta\geq1}\min\{\theta,M\}=M\). Thus \(V=\infty\) and \(V_M=M\) for every rule class containing at least one rule.

For each fixed parameter, truncation recovers the loss as the cap grows, yet no finite cap controls the original worst case. Indeed, \(\sup_\theta \tau_M(\theta,\delta)=\infty\) for every finite \(M\). This example shows why pointwise convergence alone cannot supply a minimax bound: the bound concerns a supremum over parameters and requires suitable control of tail excess at that level.

Common Mistake and Practical Use

A tempting but incorrect argument is: “For each fixed parameter, the capped loss approaches the original loss as \(M\) grows; therefore the minimax values must approach each other.” The example above disproves this inference. Taking a supremum over parameters and taking a limit need not preserve pointwise convergence. To justify a quantitative comparison, identify a bound on \(\sup_\theta E_\theta[(L-M)_+]\) for the rules being considered, or establish such a bound uniformly over the rule class.

Do not substitute the tail probability \(P_\theta(L>M)\) for the tail excess without an additional bound on the size of losses above \(M\). The excess is \((L-M)_+\), so it depends both on how often the loss exceeds the cap and on how far it exceeds it. A useful practical sequence is to choose a cap, calculate or bound the excess expectation for candidate near-minimax rules, and then report the resulting interval \(V_M\leq V\leq V_M+\text{tail bound}\). If only a rule-specific bound is available, state clearly which rule it applies to.

Check Your Understanding

Use the definitions and bounds above to answer each question.

Why is \(V_M\leq V\) valid without any tail assumption?
State the tail-excess identity relating \(R(\theta,\delta)\) and \(R_M(\theta,\delta)\).
For a fixed rule, explain why a bound on \(P_\theta(L>M)\) alone need not bound the tail excess.
When does a uniform bound \(\tau_M(\delta)\leq\varepsilon_M\) imply \(V\leq V_M+\varepsilon_M\)?
How can a rule-specific tail bound be used if the truncated minimax infimum is not attained?
What fails in the argument that pointwise convergence of capped losses guarantees convergence of minimax values?