Tail Control Under Unbounded Losses

Why Large Losses Need Separate Control

Integrated-risk arguments often approximate a prior by restricting it to a large region, or approximate an unbounded loss by a bounded one. Either operation can be useful, but neither is justified merely because the region expands or the loss is finite almost surely. A small amount of probability can still carry a large amount of expected loss. The relevant question is whether that contribution is uniformly controlled.

We make this issue explicit by treating the loss as a random variable under the joint distribution of the parameter and the data. Moment assumptions then give quantitative bounds for the part of integrated risk contributed by large losses or by parameter values outside a chosen region.

Definition: Let $\pi$ be a probability measure on the parameter space, let $P_\theta$ be the sampling distribution, and let $\delta$ be a measurable decision rule. Assume the loss $L(\theta,a)$ is nonnegative and jointly measurable. Under the joint distribution $Q_\pi(d\theta,dx)=\pi(d\theta)P_\theta(dx)$, define $Y_\delta=L(\theta,\delta(x))$. The integrated risk is $$ R_\pi(\delta)=\int R(\theta,\delta)\,\pi(d\theta) =E_{Q_\pi}[Y_\delta], $$ where $R(\theta,\delta)=E_\theta[L(\theta,\delta(X))]$. Either expectation may be infinite unless an integrability condition is imposed.

The joint-distribution notation makes the target precise: truncation and convergence questions about integrated risk are expectation questions about $Y_\delta$. We first give a bound that applies directly to a single rule.

Truncating the Loss: A Moment Bound

For a cutoff $K>0$, the bounded loss $Y_\delta\wedge K=\min(Y_\delta,K)$ has integrated risk $E_{Q_\pi}[Y_\delta\wedge K]$. The difference from the original integrated risk is nonnegative. A finite moment of order greater than one bounds this difference at a rate that improves as $K$ grows.

Theorem: Suppose $q>1$ and $E_{Q_\pi}[Y_\delta^q]\le M<\infty$. Then $$ 0\le R_\pi(\delta)-E_{Q_\pi}[Y_\delta\wedge K] \le \frac{M}{K^{q-1}},\qquad K>0. $$ In particular, the integrated risk is finite.

Proof. For every $y\ge 0$, $$ 0\le y-(y\wedge K)=(y-K)_+\le y\,\mathbf{1}_{\{y>K\}}. $$ On the event $y>K$, $y\le y^q/K^{q-1}$. Therefore $$ 0\le Y_\delta-(Y_\delta\wedge K) \le \frac{Y_\delta^q}{K^{q-1}}. $$ Taking expectations gives the stated bound. Also $E[Y_\delta]\le 1+E[Y_\delta^q]<\infty$, since $y\le 1+y^q$ for $y\ge0$. $\square$

Worked Example: Exponential Loss and an Explicit Tail Error

Suppose the loss under $Q_\pi$ has an exponential distribution with mean $2$, so its density is $f(y)=\tfrac12 e^{-y/2}$ for $y\ge0$. Its second moment is $E[Y^2]=8$. Thus the theorem with $q=2$ gives, for any $K>0$, $$ 0\le E[Y]-E[Y\wedge K]\le \frac{8}{K}. $$

The exact truncation error can also be computed. Since $(Y-K)_+=0$ for $Y\le K$, $$ E[(Y-K)_+] =\int_K^\infty (y-K)\frac12e^{-y/2}\,dy =2e^{-K/2}. $$ For $K=10$, the moment bound is $8/10=0.8$, while the exact error is $2e^{-5}$, approximately $0.0135$. The bound is conservative, but it guarantees a vanishing error without requiring an exact tail calculation.

A related bound controls restriction to a part of the parameter space. Write $A$ for a measurable parameter set and $R_\pi(A^c;\delta)=\int_{A^c}R(\theta,\delta)\,\pi(d\theta)$ for the risk contributed by its complement. If the $q$th moment is integrable under the joint distribution, Hölder's inequality gives $$ R_\pi(A^c;\delta) \le \pi(A^c)^{1-1/q} \left(\int_{A^c}E_\theta[L(\theta,\delta(X))^q]\,\pi(d\theta)\right)^{1/q}. $$

Indeed, apply Hölder's inequality on the parameter space to the functions $\mathbf{1}_{A^c}(\theta)$ and $E_\theta[L(\theta,\delta(X))]$, and then use Jensen's inequality to bound the $q$th power of the conditional mean by the conditional $q$th moment. In particular, if $E_\theta[L(\theta,\delta(X))^q]\le C$ for every $\theta$, then $$ R_\pi(A^c;\delta)\le C^{1/q}\pi(A^c). $$ Thus a small prior mass outside $A$ controls its integrated-risk contribution when the moment bound is uniform. Without uniformity, small prior mass alone need not suffice.

Worked Example: A Pareto Loss and the Moment Threshold

Let $Y$ have density $f(y)=3y^{-4}$ for $y\ge1$. This is a probability density because $\int_1^\infty 3y^{-4}\,dy=1$. Its mean is $$ E[Y]=\int_1^\infty 3y^{-3}\,dy=\frac32, $$ and its second moment is $$ E[Y^2]=\int_1^\infty 3y^{-2}\,dy=3. $$ The theorem with $q=2$ therefore bounds the truncation error by $3/K$ for $K\ge1$.

For comparison, the exact error is $$ E[(Y-K)_+] =\int_K^\infty P(Y>t)\,dt =\int_K^\infty t^{-3}\,dt =\frac{1}{2K^2},\qquad K\ge1. $$ The loss has a finite second moment, so the moment theorem applies. If instead the density were proportional to $y^{-\alpha-1}$ with $1<\alpha\le2$, the mean would remain finite but the second moment would not. In that case a $q=2$ bound would be unavailable; the moment order must be chosen to match the tail.

When Convergence of Integrated Risks Follows

In sequences of integrated-risk comparisons, one may have losses $Y_j$ converging to a limiting loss $Y$ under a fixed joint distribution $Q_\pi$. Convergence in probability alone does not guarantee convergence of expectations. A uniform moment bound supplies the missing tail control.

Theorem: Let $Y_j\ge0$ and $Y\ge0$ be random variables on a probability space. Suppose $Y_j\xrightarrow{p}Y$, and for some $q>1$, $$ \sup_j E[Y_j^q]\le M<\infty \quad\text{and}\quad E[Y^q]\le M. $$ Then $E[Y_j]\to E[Y]$.

Proof. Fix $K>0$. The truncated variables $Y_j\wedge K$ converge in probability to $Y\wedge K$, because the map $y\mapsto y\wedge K$ is continuous. Their absolute difference is bounded by $K$. To see that its expectation tends to zero, fix $\varepsilon>0$ and write $$ E[|(Y_j\wedge K)-(Y\wedge K)|] \le \varepsilon+K\,P\bigl(|(Y_j\wedge K)-(Y\wedge K)|>\varepsilon\bigr). $$ The probability on the right tends to zero. Taking the limit superior and then letting $\varepsilon$ decrease to zero proves convergence of the truncated expectations.

The moment theorem, applied to each $Y_j$ and to $Y$, also gives $$ |E[Y_j]-E[Y_j\wedge K]|\le \frac{M}{K^{q-1}}, \qquad |E[Y]-E[Y\wedge K]|\le \frac{M}{K^{q-1}}. $$ Consequently, $$ |E[Y_j]-E[Y]| \le \frac{2M}{K^{q-1}} +|E[Y_j\wedge K]-E[Y\wedge K]|. $$ For fixed $K$, the final term tends to zero. Taking a limit superior in $j$ leaves a bound of $2M/K^{q-1}$; letting $K$ tend to infinity proves $E[Y_j]\to E[Y]$. $\square$

For integrated risks, this result applies when all losses are defined under the same joint prior-sampling distribution, converge there in probability, and have uniformly bounded moments of some order greater than one. A useful practical point is that the moment bound is on the losses under the joint distribution—not merely on the prior or on the data separately.

Worked Example: Squared Error Under a Normal Model

Let the prior put all its mass at $\theta=0$, let $X\sim N(0,1)$, and use squared-error loss with decision rule $\delta(X)=X$. Then $Y=(\delta(X)-\theta)^2=X^2$. The integrated risk is $E[X^2]=1$, and the second moment of the loss is $$ E[Y^2]=E[X^4]=3. $$ Here the last equality follows from the fourth moment of a standard normal random variable. Truncating at $K=9$ gives the guarantee $$ 0\le E[X^2]-E[X^2\wedge 9]\le \frac{3}{9}=\frac13. $$ The calculation does not require evaluating the normal tail integral. It certifies that the bounded-loss approximation is within one third of the integrated risk.

Worked Example: Convergence Without Moment Control Can Fail

Let $X\sim\operatorname{Uniform}(0,1)$, use action space $[0,\infty)$, and define the loss by $L(\theta,a)=a$, independent of $\theta$. For each positive integer $m$, use the rule $$ \delta_m(x)=m\,\mathbf{1}_{\{x\le 1/m\}}. $$ For every $x>0$, $\delta_m(x)$ is eventually zero, so $\delta_m(X)\to0$ almost surely and in probability. The limiting rule has risk zero. But every rule in the sequence has risk $$ E[\delta_m(X)]=m\,P(X\le1/m)=m\cdot\frac1m=1. $$ Thus the integrated risks do not converge to the limiting risk. The second moments are $E[\delta_m(X)^2]=m^2/m=m$, which are unbounded. The rare event on which the loss is large prevents convergence of expectations.

Using the Bounds in Risk Comparisons

Suppose two rules have finite $q$th integrated-loss moments, each at most $M$. Replacing each loss by its truncation at $K$ changes each integrated risk by at most $M/K^{q-1}$. Therefore, the difference between their true integrated risks and their truncated integrated-risk difference is at most $$ \frac{2M}{K^{q-1}}. $$ This makes bounded-loss comparisons useful: if a truncated comparison has a margin larger than this error, it also determines the sign of the true comparison. When the margin is smaller, the bound does not settle the comparison, and a sharper tail estimate or a larger cutoff may be needed.

Parameter truncation and loss truncation address different tails. Restricting the prior to $A$ discards parameter values in $A^c$; replacing $L$ by $L\wedge K$ discards the excess loss above $K$ throughout the parameter space. Both errors must be controlled if both approximations are used. Moment assumptions can control the loss tail, while the parameter-tail bound additionally requires suitable control of prior mass and moments outside the retained region.

Common Pitfall

Wrong reasoning: “The losses converge pointwise, so their integrated risks converge.” The rare-event example shows why this fails: increasingly large losses can occur on increasingly small sets, leaving the expectations unchanged.
Correction: Establish tail control, for example with a uniform moment bound of order $q>1$, before passing from convergence of losses to convergence of integrated risks.
Wrong reasoning: “The discarded parameter region has small prior mass, so its risk contribution is small.” This is not valid if conditional risks there can be arbitrarily large.
Correction: Bound the risk contribution on that region using a moment envelope or another explicit tail estimate.

Key takeaway: For nonnegative unbounded losses, integrated-risk truncation is justified by controlling the part that truncation removes. A finite higher moment gives a quantitative loss-tail bound, and a uniform higher-moment bound turns convergence in probability into convergence of integrated risks.

Check Your Understanding

Use the definitions and bounds above to answer the following questions.

If $E[Y^3]\le 16$, what bound does the moment theorem give for the truncation error at cutoff $K$?
Why does a uniform bound on $E[Y_j^q]$ control the error between $E[Y_j]$ and $E[Y_j\wedge K]$ uniformly over $j$?
What additional condition, beyond small prior mass of $A^c$, can bound the integrated risk contributed by $A^c$?
In the uniform-data example, which moment condition fails to hold uniformly over the sequence of rules?
When approximating an integrated-risk comparison by restricting the prior and truncating the loss, what are the two distinct errors that must be assessed?