Risk Approximation in a Countable Parameter Problem

Pointwise Approximation Is Not Domination

On a countable parameter space, a sequence of risk functions can approach a target risk at every fixed parameter value. That convergence is useful, but it does not by itself provide a single rule whose risk is no greater than the target risk everywhere. Essential completeness is a domination property: it asks for an actual replacement rule, not merely a sequence whose risks get close.

The distinction is especially visible when the parameter value at which a rule performs poorly moves farther out as the sequence progresses. At each fixed parameter, that poor performance eventually disappears; across the whole parameter space, it never disappears.

The example below uses a single observation value, so a deterministic rule is simply an action. Randomized rules, if allowed, assign probabilities to actions. This removes sampling details and makes the risk comparison explicit.

A Countable-Parameter Construction

Let \(\Theta=\{1,2,3,\ldots\}\), and let the observation space contain only one possible observation. The action space consists of a target action \(b\) and actions \(a_k\), one for each positive integer \(k\). Define the loss by

Here \(\mathbf{1}\{\theta=k\}\) is 1 when \(\theta=k\) and 0 otherwise. Since the observation is constant, the risk of a deterministic action equals its loss. Write \(\delta_b\) for the rule choosing \(b\), and \(\delta_k\) for the rule choosing \(a_k\). Their risks are

Consider the subclass \(\mathcal C=\{\delta_1,\delta_2,\ldots\}\), which contains the sequence but not \(\delta_b\). The constant 1 is the target risk function. For any fixed \(\theta\), once \(k>\theta\), the indicator is zero, so the risk difference is exactly \(1/k\) and tends to zero.

Proof. Fix \(\theta\). For every \(k>\theta\), the indicator \(\mathbf{1}\{\theta=k\}\) equals zero. Hence

This proves pointwise convergence. But for each particular \(k\), evaluate the risk at \(\theta=k\):

Thus \(\delta_k\) fails to weakly dominate \(\delta_b\) at parameter \(k\). Since this applies to every \(k\), no member of \(\mathcal C\) dominates \(\delta_b\).

Now let a randomized rule choose \(a_j\) with probabilities \(q_j\), where \(q_j\geq 0\) and \(\sum_{j=1}^{\infty}q_j=1\). At parameter \(\theta\), its risk is

At least one \(q_j\) is positive, so \(\sum_j q_j/j>0\). The randomized rule therefore has risk strictly greater than 1 at every parameter, and cannot dominate \(\delta_b\). The class is not essentially complete relative to the full rule class, which includes \(\delta_b\). \(\square\)

What Finite Checks Can and Cannot Show

Pointwise convergence does imply good approximation on any fixed finite subset of parameter values. In this example, the rule index can be chosen beyond every parameter in that subset, removing all the indicator terms at once. But no one fixed index works for the entire infinite parameter space.

Proof. If \(F\) is nonempty, choose \(k>\max F\) and \(1/k<\varepsilon\). For every \(\theta\in F\), \(\theta\neq k\), so the risk difference is \(1/k<\varepsilon\). If \(F\) is empty, the condition holds vacuously. On the other hand, at a general \(\theta\) the difference is \(1/k+\mathbf{1}\{\theta=k\}\). Its supremum over \(\Theta\) is \(1+1/k\), attained at \(\theta=k\). Since \(1+1/k\) does not tend to zero, convergence is not uniform. \(\square\)

A Weighted Risk Can Also Converge

A countable parameter space may also be summarized using a probability distribution on the parameter values. Such a weighted average can converge even though it does not establish pointwise domination. For instance, assign prior probability \(\pi_\theta=2^{-\theta}\) to \(\theta\in\Theta\). These probabilities sum to 1 because \(\sum_{\theta=1}^{\infty}2^{-\theta}=1\).

Why the Approximation Does Not Establish Essential Completeness

Essential completeness requires a rule in the proposed subclass that weakly dominates each rule under consideration. In this construction, the target \(\delta_b\) has risk 1 everywhere, while every deterministic rule in \(\mathcal C\) has risk greater than 1 everywhere. Randomizing among those rules cannot fix the problem: the positive term \(\sum_j q_j/j\) remains in the risk at every parameter.

The pointwise limit is therefore a statement about the behavior of a sequence at each fixed coordinate, not a statement that any sequence member is a valid replacement. Even convergence of a weighted average risk does not supply the required coordinate-by-coordinate inequalities. To establish essential completeness from an approximation argument, one needs additional structure that produces an actual rule in the class with the required inequalities; convergence alone does not do so.

Common Mistake

• Wrong reasoning: “The risks of \(\delta_k\) converge to the target risk at every parameter, so some sufficiently large \(k\) must have risk no greater than the target everywhere.”
• Correction: Pointwise convergence allows the parameter that violates the inequality to depend on \(k\). Here, for every chosen \(k\), the violation occurs at \(\theta=k\). No single index avoids all such parameter values.

A useful check is to write down the quantifiers. Pointwise convergence says that for each fixed \(\theta\), the error becomes small as \(k\) grows. Domination asks for one rule index \(k\) such that the risk inequality holds for every \(\theta\). Those are different claims, and the first does not imply the second.