Compact Risk Sets and Attained Replacements

Why Attainment Is a Separate Question

A decision rule can have risks that approach a desirable limit without any rule actually achieving that limit. This distinction matters whenever an argument claims that one rule can replace another: an infimum of risk is not, by itself, an attainable risk. In a finite-parameter problem, compactness of the attainable risk set supplies a precise existence guarantee.

The claim concerns the set of risk vectors, not necessarily the set of decision rules. There may be many rules with the same risks, and the rule space itself may be large. What matters here is whether the risks the rules attain form a compact subset of a finite-dimensional Euclidean space.

Definition: Let \(\Theta=\{\theta_1,\ldots,\theta_m\}\) be a finite parameter set, and let \(\mathcal D\) be a class of decision rules. The risk vector of \(\delta\in\mathcal D\) is \(R(\delta)=(R(\theta_1,\delta),\ldots,R(\theta_m,\delta))\in\mathbb R^m\). The attainable risk set is \(\mathcal A=\{R(\delta):\delta\in\mathcal D\}\). It is compact when it is compact in the usual Euclidean topology on \(\mathbb R^m\).

A risk vector is attainable exactly when at least one rule in the specified class has that vector. In finite dimensions, compactness is equivalent to being both closed and bounded. Closedness prevents limits of attainable vectors from disappearing at unattainable boundary points; boundedness rules out sequences escaping indefinitely. Convexity is not part of the definition and is not required for the existence results below.

Continuous Criteria Have Attained Minima

Often the goal is to minimize a scalar summary of the risks. Examples include a weighted sum of risks and the maximum risk over the finite parameter set. These criteria compare risk vectors, and their continuity is what allows compactness to deliver an actual minimizing vector.

Theorem: Let \(\mathcal A\) be a nonempty compact attainable risk set in \(\mathbb R^m\), and let \(\phi:\mathcal A\to\mathbb R\) be continuous. Then there is an \(r^*\in\mathcal A\) such that \(\phi(r^*)=\min_{r\in\mathcal A}\phi(r)\). Consequently, at least one decision rule in \(\mathcal D\) attains this minimum.

Proof. Since \(\mathcal A\) is compact and \(\phi\) is continuous, \(\phi(\mathcal A)\) is a compact subset of \(\mathbb R\). It is therefore nonempty and contains its infimum. Choose \(r^*\in\mathcal A\) whose image is that minimum. Because \(r^*\) belongs to the attainable risk set, the definition of \(\mathcal A\) guarantees a rule \(\delta^*\in\mathcal D\) with \(R(\delta^*)=r^*\). Thus the minimum is achieved by a rule, not merely approached by a sequence. \(\square\)

For fixed weights \(w_i\geq 0\), the criterion \(\phi(r)=\sum_{i=1}^m w_i r_i\) is continuous. The criterion \(\phi(r)=\max_{1\leq i\leq m}r_i\), which evaluates the largest risk across the listed parameter values, is continuous as well. The theorem applies to either criterion whenever the attainable risk set is nonempty and compact.

Worked Example: Minimizing a Weighted Risk

Suppose three decision rules have risk vectors \(r_A=(1,4)\), \(r_B=(2,1)\), and \(r_C=(3,0.5)\) at two parameter values. The attainable set consists of these three vectors. Minimize the weighted risk \(2r_1+r_2\).

A finite set in \(\mathbb R^2\) is compact, so the existence theorem applies. Calculate the criterion at each attainable vector:

\begin{aligned} \phi(r_A)&=2(1)+4=6,\\ \phi(r_B)&=2(2)+1=5,\\ \phi(r_C)&=2(3)+0.5=6.5. \end{aligned}

The minimum is \(5\), achieved at \(r_B\). Since \(r_B\) is the risk vector of rule \(B\), rule \(B\) is an attained minimizer. The calculation does not claim that \(r_B\) is best for every possible criterion; it is best for the specified weights.

Worked Example: Attaining the Minimum Maximum Risk

Let the attainable risk set be \(\mathcal A=\{(t,1-t):0\leq t\leq 1\}\). This is a closed, bounded line segment in \(\mathbb R^2\), hence compact. Find the rule risks minimizing the maximum of the two coordinates.

For \(0\leq t\leq 1\), the maximum risk is \(\max(t,1-t)\). If \(t\geq 1/2\), this equals \(t\), which is at least \(1/2\). If \(t\leq 1/2\), it equals \(1-t\), which is also at least \(1/2\). At \(t=1/2\), both risks equal \(1/2\), so the minimum possible maximum risk is \(1/2\) and is attained at \((1/2,1/2)\).

Every point on the segment is attainable by the way the set is defined. Therefore a decision rule realizing \((1/2,1/2)\) exists. The example also shows why the criterion matters: a different objective could select a different point on the same compact set.

Finding an Undominated Replacement

Compactness can also ensure a replacement that is no worse at any of the listed parameter values and cannot itself be improved by another attainable risk vector. Write \(s\leq r\) when \(s_i\leq r_i\) for every coordinate. Say that \(s\) strictly dominates \(r\) if \(s\leq r\) and \(s_j<r_j\) for at least one coordinate \(j\).

Theorem: Let \(\mathcal A\subset\mathbb R^m\) be compact, and fix any \(r_0\in\mathcal A\). There is an \(r^*\in\mathcal A\) such that \(r^*\leq r_0\) and no vector in \(\mathcal A\) strictly dominates \(r^*\). Thus \(r^*\) is an attainable, undominated replacement for \(r_0\).

Proof. Consider the subset

B=\{r\in\mathcal A:r_i\leq (r_0)_i\text{ for every }i\}.

It is nonempty because \(r_0\in B\). It is closed relative to \(\mathcal A\), since each condition \(r_i\leq (r_0)_i\) defines a closed subset of \(\mathbb R^m\). Hence \(B\) is compact. The function \(S(r)=\sum_{i=1}^m r_i\) is continuous, so it attains a minimum on \(B\); call a minimizing vector \(r^*\). By construction, \(r^*\leq r_0\). If an attainable vector \(s\) strictly dominated \(r^*\), then \(s\leq r^*\leq r_0\), so \(s\in B\), while strict inequality in at least one coordinate would give

S(s)=\sum_{i=1}^m s_i < \sum_{i=1}^m r_i^*=S(r^*).

This contradicts the minimizing property of \(r^*\) on \(B\). Therefore no attainable vector strictly dominates \(r^*\). Because \(r^*\) is in \(\mathcal A\), some rule attains it. \(\square\)

This result uses the coordinatewise order, not a chosen weighted sum over the whole risk set. The sum is used only to select a point from the compact set of vectors no worse than \(r_0\). The proof does not require the attainable set to be convex.

Worked Example: Replacing a Dominated Risk Vector

Consider the finite attainable set \(\mathcal A=\{(4,3),(3,3),(3,2),(2,4)\}\), and take \(r_0=(4,3)\). Find an undominated attainable vector no worse than \(r_0\).

The vectors in \(\mathcal A\) that are coordinatewise no worse than \(r_0\) are \((4,3)\), \((3,3)\), and \((3,2)\). Their coordinate sums are \(7\), \(6\), and \(5\), respectively. The remaining vector \((2,4)\) is not eligible because its second coordinate \(4\) exceeds the second coordinate of \(r_0\), which is \(3\).

The smallest sum among eligible vectors is \(5\), achieved at \((3,2)\). No eligible vector can strictly dominate \((3,2)\): the other eligible vectors each have a coordinate larger than the corresponding coordinate of \((3,2)\). Any vector that dominates it would also be no worse than \(r_0\), so it would have been among the eligible vectors. Thus \((3,2)\) is an undominated replacement for \((4,3)\), and a rule attaining \((3,2)\) exists because the vector is in \(\mathcal A\).

What Can Fail Without Compactness

If the attainable risk set is not compact, a continuous criterion can have an infimum that is not a minimum. A sequence of attainable risks can converge to a boundary point that is absent from the set. It is then incorrect to treat the limiting vector as the risk of some rule.

Worked Example: A Risk Infimum That No Rule Attains

Consider a decision problem with no informative observation, two parameter values, and actions \(a\in(0,1)\). Define the losses by \(L(\theta_1,a)=a\) and \(L(\theta_2,a)=1-a\). Each action is a deterministic rule, and its risk vector is therefore \((a,1-a)\). The attainable set is the open line segment \(\mathcal A=\{(a,1-a):0<a<1\}\), which is bounded but not closed and hence not compact.

Minimize the risk at \(\theta_1\), that is, minimize the first coordinate. Every attainable first coordinate is positive, while the sequence \(a_n=1/n\), for integers \(n\geq 2\), gives

R(a_n)=\left(\frac{1}{n},1-\frac{1}{n}\right) \xrightarrow[n\to\infty]{}(0,1).

The infimum of the first-coordinate risk is \(0\), but no action \(a\in(0,1)\) has first-coordinate risk \(0\). The limit \((0,1)\) belongs to the closure of the risk set, not to the attainable risk set. It cannot serve as an attained replacement. Notice that this does not say every criterion fails: for example, the maximum of the two risks is minimized at \(a=1/2\). What fails is the general guarantee that a continuous criterion's infimum is attained.

Reading the Compactness Assumption Correctly

The relevant hypothesis is compactness of the set over which the optimization is performed. Compactness of the rule space alone is not the statement used in the proofs, and a compact closure of the risk set is not enough if that closure includes unattainable vectors. In finite-dimensional risk-vector problems, checking compactness means checking both boundedness and closedness of the actual attainable set.

Compactness is sufficient, not necessary. A noncompact set may still contain a minimizer for a particular criterion, as the maximum-risk calculation in the preceding example demonstrates. Conversely, boundedness alone does not guarantee attainment. When an argument uses a risk limit as a decision rule's risk, verify that the limit lies in the attainable set; otherwise the argument has established only approximation.

Key takeaway: On a nonempty compact attainable risk set, continuous risk criteria attain their minima, and every attainable vector has an undominated attainable replacement no worse in any coordinate. Without compactness, an infimum or a boundary limit may fail to correspond to any decision rule.

Check Your Understanding

Answer each question using the distinction between attainable risk vectors and their limits.

Why does a minimum of a continuous criterion on a compact risk set correspond to at least one decision rule?
Which hypotheses allow the weighted-risk criterion \(\sum_i w_i r_i\) to have an attained minimum?
In the undominated-replacement proof, why is the subset of vectors no worse than \(r_0\) compact?
For the open segment \(\{(a,1-a):0<a<1\}\), which risk value is an unattained infimum when minimizing the first coordinate?
Does a noncompact attainable risk set imply that every continuous criterion fails to attain its minimum? Explain using an example from the tutorial.