When Better Risks Have No Best Rule
In a decision problem, it is possible to keep improving a rule’s risk without ever reaching a rule that is best. This happens, for example, when the available actions approach a boundary that is excluded from the action space. The resulting sequence of risks has a limit, but that limit need not be the risk of any allowed rule.
The distinction is important: a limit of attainable risks is not necessarily attainable. In particular, a risk infimum describes the best value that can be approached, whereas a minimizing rule must actually have that risk. We will make this distinction precise using simple decision problems whose calculations can be checked directly.
A scalar criterion can be formed by evaluating risk at one parameter value, or by combining risks across several parameter values. Its infimum is a number. A rule attains that infimum only if its own criterion value equals the infimum. Similarly, a risk function or vector is attained only if some allowed rule has exactly that risk function or vector. These are separate questions from whether a sequence of attainable risks converges.
A General Boundary Principle
The following elementary principle identifies a common source of nonattainment. It concerns optimization over actions; it applies directly to decision problems with no informative data, and to a scalar risk criterion whenever that criterion is a continuous function of the chosen action.
Proof. Since \(A\) is dense in its closure \(\overline A\), for each \(b\in\overline A\) there is a sequence \(a_m\in A\) with \(a_m\to b\). Continuity gives \(q(a_m)\to q(b)\), so \(\inf_{a\in A}q(a)\leq q(b)\). Taking the minimum over \(b\in\overline A\), which exists because \(\overline A\) is compact and \(q\) is continuous, gives \(\inf_{a\in A}q(a)\leq\min_{b\in\overline A}q(b)\). The reverse inequality follows from \(A\subseteq\overline A\): every \(q(a)\), \(a\in A\), is at least this minimum. Thus the two values are equal. If a minimizer \(b\) on \(\overline A\) belongs to \(A\), the common value is attained there. Conversely, if some \(a\in A\) attains the infimum, then \(q(a)=\min_{\overline A}q\), so \(a\) is a minimizer in \(A\). \(\square\)
The proposition does not say that every optimization problem has this form. It gives a precise conclusion under stated compactness and continuity conditions: the closure can supply a minimizer even when the original action set does not. The next examples show what this means for risks.
Worked Example: A Scalar Risk Infimum at an Excluded Endpoint
There is one parameter value, \(\theta=0\), no informative observation, and the action space is \(A=(0,1)\). The loss is squared error, \(L(0,a)=a^2\). Find the infimum risk and determine whether any rule attains it.
A deterministic rule selects one fixed \(a\in(0,1)\), so its risk is \(R(0,a)=a^2\). Since \(a^2>0\) for every allowed action, no rule has risk zero. On the other hand, for each integer \(k\geq2\), the action \(a_k=1/k\) is allowed and has risk \[ R(0,a_k)=(1/k)^2=1/k^2\longrightarrow 0. \] Therefore \(\inf_{a\in(0,1)}R(0,a)=0\), but the infimum is not attained. The continuous loss extends to the compact closure \([0,1]\), where its minimum occurs at the excluded action \(0\).
Allowing randomization does not change the conclusion. If \(A\) is a random action with \(P(0<A<1)=1\), then \(E[A^2]>0\): a nonnegative random variable has expectation zero only if it is zero almost surely, which would contradict \(A>0\) almost surely. The deterministic sequence above still makes the infimum zero.
More generally, if a sequence of rules \(\delta_k\) has scalar criterion values \(C(\delta_k)\to c\), then \(\inf_\delta C(\delta)\leq c\). If \(c\) is already known to be the infimum, this proves that the sequence approaches the best possible value; it does not prove that any rule has value \(c\). Attainment requires a separate equality check.
Risk Vectors: A Limit Need Not Be a Rule’s Risk
For a finite parameter set, it is useful to list the risks as a vector with one coordinate per parameter value. Convergence of these vectors means coordinatewise convergence. A vector can be in the closure of the attainable risk vectors without being one of them.
Worked Example: A Boundary Point in a Two-State Risk Set
Let \(\Theta=\{0,1\}\), let \(A=(0,1)\), and suppose there is no informative observation. Use squared-error loss \(L(\theta,a)=(a-\theta)^2\). Calculate a risk-vector limit and check whether it is attainable.
For a deterministic action \(a\), the two risks are \(R(0,a)=a^2\) and \(R(1,a)=(1-a)^2\). Thus its risk vector is \((a^2,(1-a)^2)\). Choose \(a_k=1/k\), for integers \(k\geq2\). Then \[ \bigl(R(0,a_k),R(1,a_k)\bigr) =\bigl(1/k^2,(1-1/k)^2\bigr) \longrightarrow(0,1). \] The vector \((0,1)\) is therefore in the closure of the attainable risk vectors.
It is not attained by a deterministic rule: its first coordinate would require \(a^2=0\), hence \(a=0\), which is not allowed. Nor can randomization attain it. For a random action \(A\in(0,1)\) almost surely, the first risk is \(E[A^2]>0\), so it cannot equal the first coordinate of \((0,1)\). The boundary vector is a genuine limit, not the risk vector of an available rule.
This example concerns a specific limit vector; it does not claim that the sequence minimizes every possible summary of risk. For instance, the maximum of the two risks can be optimized separately. To see the distinction, for deterministic \(a\in(0,1)\), the maximum risk is \(\max\{a^2,(1-a)^2\}\), which is minimized at \(a=1/2\), with value \(1/4\). A boundary limit of risk vectors and an attained minimum of a chosen scalar criterion are different objects.
Justification. In finite-dimensional Euclidean space, coordinatewise convergence is ordinary convergence: for every \(\varepsilon>0\), each of the finitely many coordinate differences is eventually less than \(\varepsilon/\sqrt m\), and hence the Euclidean distance from \(r_k\) to \(r\) is eventually less than \(\varepsilon\). Every open ball around \(r\) therefore contains attainable vectors, which is exactly the definition of membership in the closure. Attainment would additionally require \(r\) to be in the attainable set. Closure membership alone does not imply this. \(\square\)
Pointwise and Uniform Limits of Risk Functions
When the parameter space is infinite, risk functions can approach a limit in different senses. Pointwise convergence checks each fixed parameter value separately. Uniform convergence requires one bound to control the error over all parameter values. Neither mode of convergence, by itself, asserts that the limit is the risk function of an allowed rule.
Worked Example: A Uniform Limit That Is Not Attained
Let \(\Theta=[0,1]\), \(A=(0,1)\), and again suppose there is no informative observation. Take absolute-error loss \(L(\theta,a)=|a-\theta|\). For the actions \(a_k=1/k\), \(k\geq2\), show that the risk functions converge uniformly, and determine whether their limit is attainable.
The risk function for action \(a_k\) is \(R_k(\theta)=|1/k-\theta|\). Consider the candidate limit \(g(\theta)=\theta\), since \(\theta\geq0\). If \(0\leq\theta\leq a_k\), then \[ |R_k(\theta)-g(\theta)| =|(a_k-\theta)-\theta| =|a_k-2\theta| \leq a_k. \] If \(a_k\leq\theta\leq1\), then \(R_k(\theta)=\theta-a_k\), and \[ |R_k(\theta)-g(\theta)|=|(\theta-a_k)-\theta|=a_k. \] Equality in the first region occurs at \(\theta=0\), so \[ \sup_{\theta\in[0,1]}|R_k(\theta)-g(\theta)|=a_k=1/k\longrightarrow0. \] Thus the convergence to \(g\) is uniform.
No allowed deterministic action \(a\in(0,1)\) has risk function \(g\), because at \(\theta=0\) its risk is \(|a-0|=a>0\), whereas \(g(0)=0\). Randomization cannot attain \(g\) either: for a random action \(A\in(0,1)\) almost surely, the risk at \(\theta=0\) is \(E[A]>0\). The limit function is uniformly approximated by attainable risks but is not itself attainable.
What Nonattainment Does—and Does Not—Tell Us
A boundary limit can be useful even when no rule realizes it. It can identify the best achievable value of a risk criterion, reveal which parameter values create a limiting trade-off, and show how closely a proposed class of rules can approximate a desired risk function. But replacing “can be approached” with “is achieved” changes the mathematical claim. A proof of approximation does not supply an optimal rule.
Keep three statements distinct:
- Infimum: no rule has a criterion value below \(c\), and values can be found arbitrarily close to \(c\).
- Attainment: at least one allowed rule has criterion value exactly \(c\).
- Risk closure: a risk function or vector is a limit of attainable risks in a specified topology.
For the scalar example, the infimum is zero and the sequence \(1/k\) approaches it, but no rule attains it. For the two-state example, the vector \((0,1)\) is in the risk-vector closure but not in the attainable set. For the last example, convergence is uniform, yet no rule has the limit risk function. These conclusions are compatible: approximation can be arbitrarily strong while exact attainment still fails.
Common Mistake
Incorrect reasoning: “The risks of \(\delta_k\) converge to \(r\), so the optimal rule has risk \(r\).” This treats a limit of rules’ risks as though it were automatically the risk of some rule.
Correction: Convergence shows that \(r\) is a limit in the chosen risk space. To claim attainment, one must identify an allowed rule \(\delta\) and verify \(R(\theta,\delta)=r(\theta)\) for every relevant \(\theta\), or verify the corresponding equality for the risk vector or scalar criterion. If no such rule exists, report an unattained infimum or a closure point, not an optimal rule.
When a boundary is suspected, check the action space carefully: the point minimizing a continuous extension may be excluded. Then verify both parts of the conclusion directly: exhibit allowed actions approaching the boundary value, and show that every allowed rule has a different value. This keeps approximation and optimization claims logically separate.
Check Your Understanding
For each question, distinguish what follows from a limiting sequence from what would require an attaining rule.
- In the one-state squared-error example, why is the infimum zero even though every allowed deterministic action has strictly positive risk?
- For the two-state example, which coordinate proves that no randomized action can attain the vector \((0,1)\)?
- What is the uniform distance between the risk function for action \(a\in(0,1)\) and the function \(g(\theta)=\theta\) in the absolute-error example?
- State the additional condition needed to conclude that a scalar risk infimum is attained.
- How can a risk function belong to the closure of the attainable risk set without being the risk function of any allowed rule?