Essential Completeness and Pointwise Risk Limits

Why Approximation Changes the Comparison

A class of decision rules can fail to contain a rule that dominates a particular procedure, yet still contain rules whose risks approach that procedure's risk at every parameter value. This distinction matters when a useful procedure lies at the edge of what a class can attain: exact replacement may be impossible even though arbitrarily close risk performance is available in the pointwise sense.

The comparison is made coordinate by coordinate over the parameter space. It does not ask that one rule in the class improve on the target at every parameter. Instead, a sequence of rules may have risks that approach a function no larger than the target risk. The limiting function need not itself be the risk function of a rule in the class.

Definition: Essential Completeness

Let \(\mathcal D\) be a collection of decision rules for a specified statistical decision problem, and let \(\mathcal C\subseteq\mathcal D\). For a rule \(\delta\), write \(R(\theta,\delta)\) for its risk at parameter value \(\theta\). Risks may be extended nonnegative real numbers; the comparisons below are interpreted in that order.

Definition: The class \(\mathcal C\) is essentially complete relative to \(\mathcal D\) if, for every \(\delta\in\mathcal D\), there is a sequence \((\delta_m)_{m\geq 1}\) with each \(\delta_m\in\mathcal C\) such that \[ \limsup_{m\to\infty} R(\theta,\delta_m)\leq R(\theta,\delta) \quad\text{for every }\theta\in\Theta. \] If the sequence of risks converges at every \(\theta\), this says that its pointwise limit is no greater than the risk of \(\delta\) at every parameter.

The pointwise limit superior form allows the risks to oscillate, provided their eventual upper behavior is no worse than the target risk at each fixed parameter. When an example supplies an actual pointwise limit, that limit provides a particularly transparent witness. No uniformity over \(\theta\) is required: the index after which a bound holds may depend on \(\theta\).

The comparison is relative to \(\mathcal D\). A class may be essentially complete within one chosen universe of rules and fail to be essentially complete in a larger universe. As usual, risks are defined from the model and loss: for a measurable rule \(\delta\) and measurable loss \(L\), \(R(\theta,\delta)=E_\theta[L(\theta,\delta(X))]\), with the expectation allowed to be infinite. The examples below use a one-point sample space, so all rules and losses there are measurable.

Essential Completeness Versus Exact Domination

Exact domination is a single-rule condition. A class \(\mathcal C\) is complete relative to \(\mathcal D\) if, for every \(\delta\in\mathcal D\), there is some \(\delta^\ast\in\mathcal C\) such that \(R(\theta,\delta^\ast)\leq R(\theta,\delta)\) for every \(\theta\). Essential completeness weakens “there is some rule” to “there is a sequence of rules whose limit superior risks satisfy the comparison.”

Theorem: Every complete class is essentially complete.

Proof. Fix any \(\delta\in\mathcal D\). By completeness, choose \(\delta^\ast\in\mathcal C\) with \(R(\theta,\delta^\ast)\leq R(\theta,\delta)\) for every \(\theta\). Take the constant sequence \(\delta_m=\delta^\ast\) for every \(m\). Then \[ \limsup_{m\to\infty}R(\theta,\delta_m) =R(\theta,\delta^\ast) \leq R(\theta,\delta) \] for every \(\theta\). This is exactly the essential-completeness condition. \(\square\)

The reverse implication need not hold. A sequence can improve toward the target without any one member of the sequence achieving the comparison. The following example makes this distinction explicit.

Worked Example: A Class That Is Essentially Complete but Not Complete

Consider a decision problem with parameter space \(\Theta=\{0,1\}\) and a sample space containing one point, observed with probability one under either parameter. The action set is \(\{b,a_1,a_2,\ldots\}\). Define the loss at each parameter by the following risk values; because there is no data variation, each risk equals the corresponding loss.

R(0,b)=R(1,b)=0,\qquad R(0,a_n)=\frac{1}{n},\quad R(1,a_n)=0.

Let \(\mathcal D\) contain all these actions and let \(\mathcal C=\{a_1,a_2,\ldots\}\). For target rule \(b\), choose the sequence \(a_m\). At parameter \(0\), \(R(0,a_m)=1/m\to 0=R(0,b)\); at parameter \(1\), \(R(1,a_m)=0=R(1,b)\). Thus its risks converge pointwise to the risk of \(b\). For any target \(a_k\), the constant sequence \(a_k,a_k,\ldots\) works. Therefore \(\mathcal C\) is essentially complete relative to \(\mathcal D\).

But no action in \(\mathcal C\) dominates \(b\): for every \(n\), \(R(0,a_n)=1/n>0=R(0,b)\). Hence \(\mathcal C\) is not complete. The sequence approaches the target's risk without providing an exact dominating rule in the class.

Reading the Pointwise Condition Precisely

For finite \(R(\theta,\delta)\), the limit superior condition has a useful equivalent form: for every \(\varepsilon>0\), there is an index \(M\), possibly depending on \(\theta\) and \(\varepsilon\), such that \[ R(\theta,\delta_m)\leq R(\theta,\delta)+\varepsilon \quad\text{for every }m\geq M. \] This is an eventual, coordinate-by-coordinate bound, not a claim that one index works for all parameter values.

Result: For a real sequence \((x_m)\) and finite \(c\), \(\limsup_{m\to\infty}x_m\leq c\) if and only if, for every \(\varepsilon>0\), \(x_m\leq c+\varepsilon\) for all sufficiently large \(m\).

Proof. Suppose first that \(\limsup_m x_m\leq c\). By the definition \(\limsup_m x_m=\inf_N\sup_{m\geq N}x_m\), for every \(\varepsilon>0\) there is \(N\) such that \(\sup_{m\geq N}x_m<c+\varepsilon\), and therefore \(x_m\leq c+\varepsilon\) for all \(m\geq N\). Conversely, suppose that for every \(\varepsilon>0\) this eventual bound holds. Then \(\sup_{m\geq N}x_m\leq c+\varepsilon\) for some \(N\), so \(\limsup_m x_m\leq c+\varepsilon\). Since this is true for every \(\varepsilon>0\), \(\limsup_m x_m\leq c\). \(\square\)

When risks converge pointwise to \(r(\theta)\), essential completeness is witnessed whenever \(r(\theta)\leq R(\theta,\delta)\) for every \(\theta\). It does not require \(r\) to equal \(R(\theta,\delta^\ast)\) for any \(\delta^\ast\in\mathcal C\). That difference between a limiting risk function and an attained risk function is central to the examples that follow.

Worked Example: A Trade-Off That Disappears Only in the Limit

Use the same two-parameter, one-point-sample setup, but now let the actions be \(b,a_1,a_2,\ldots\), with \[ R(0,b)=R(1,b)=1,\qquad R(0,a_n)=1+\frac{1}{n},\quad R(1,a_n)=1-\frac{1}{n}. \] Let \(\mathcal C=\{a_n:n\geq1\}\) and \(\mathcal D=\mathcal C\cup\{b\}\).

For target \(b\), the risks of \(a_n\) converge to its risks at both parameter values: \[ \lim_{n\to\infty}R(0,a_n)=\lim_{n\to\infty}\left(1+\frac1n\right)=1, \qquad \lim_{n\to\infty}R(1,a_n)=\lim_{n\to\infty}\left(1-\frac1n\right)=1. \] For any target \(a_k\), the constant sequence at \(a_k\) satisfies the condition. Thus \(\mathcal C\) is essentially complete relative to \(\mathcal D\).

Yet no \(a_n\) dominates \(b\), because at parameter \(0\), \(1+1/n>1\). Each action improves on \(b\) at parameter \(1\) but is worse at parameter \(0\). The pointwise limiting comparison succeeds even though the class contains no exact replacement for \(b\).

Pointwise Does Not Mean Uniform

With an infinite parameter space, pointwise approximation can be especially different from uniform approximation. At each fixed \(\theta\), the risk sequence may eventually be close to its limit, while for every index \(n\) there remains some parameter value at which the discrepancy is large. The definition of essential completeness asks only for the first property.

Worked Example: Pointwise Approximation Without Uniform Approximation

Let \(\Theta=(0,1]\), again with a one-point sample space. Let the actions be \(b,a_1,a_2,\ldots\), and define \[ R(\theta,b)=1,\qquad R(\theta,a_n)=1+\mathbf{1}\{\theta<1/n\}, \] where \(\mathbf{1}\{A\}\) equals one when \(A\) holds and zero otherwise. Take \(\mathcal C=\{a_n:n\geq1\}\) and \(\mathcal D=\mathcal C\cup\{b\}\).

Fix any \(\theta>0\). Once \(n\geq 1/\theta\), we have \(1/n\leq\theta\), so \(\theta<1/n\) is false and \(R(\theta,a_n)=1\). Therefore \(R(\theta,a_n)\to1=R(\theta,b)\) for every fixed \(\theta\). For a target \(a_k\), the constant sequence \(a_k\) works, so \(\mathcal C\) is essentially complete relative to \(\mathcal D\).

No \(a_n\) dominates \(b\): for example, at \(\theta=1/(2n)\), \(\theta<1/n\), and hence \(R(\theta,a_n)=2>1=R(\theta,b)\). Moreover, \[ \sup_{\theta\in(0,1]}\left|R(\theta,a_n)-R(\theta,b)\right|=1 \] for every \(n\). Thus the risks converge pointwise but not uniformly. The index after which the risks equal one depends on the fixed parameter \(\theta\).

Sequential Pointwise Limits and a Topological Caution

It is tempting to describe essential completeness by saying that certain risk functions belong to “the pointwise closure” of the risks generated by \(\mathcal C\). Without further conditions on \(\Theta\), that wording can be misleading. The definition above is explicitly sequential: it requires a sequence of rules whose risks satisfy the limit-superior comparison. In general, topological closure in the pointwise topology is not the same thing as sequential closure.

For example, let \(\Theta\) be uncountable and consider indicator functions of finite subsets of \(\Theta\). The all-ones function belongs to their pointwise-topology closure: any basic neighborhood constrains only finitely many coordinates, and a finite subset can include all of them. But no sequence of these indicator functions converges pointwise to the all-ones function. The union of the finite supports in a sequence is countable, so some coordinate is absent from every support and remains zero. This distinction is why the sequential condition, rather than unqualified pointwise-closure language, is the safe formulation here.

Common Mistakes and Why the Distinction Matters

Demanding a single dominating member. That is completeness, not essential completeness. Essential completeness permits a sequence, as long as its limit superior risk is no greater than the target risk at every parameter.
Assuming a limiting risk is attainable. A pointwise limit of risk functions need not be the risk function of any rule in \(\mathcal C\). The first example shows that the class can approximate \(b\)'s risk without containing a rule that dominates \(b\).
Replacing pointwise convergence with uniform convergence. The definition allows the needed index to depend on \(\theta\). The continuous-parameter example has pointwise convergence but a supremum discrepancy equal to one at every index.
Using “pointwise closure” without qualification. For arbitrary parameter spaces, topological closure may contain functions that no sequence of class risks approaches. Refer to a sequential pointwise limit when a sequence is what the argument establishes.

Essential completeness is useful because it records when a rule class can approximate the risk performance of every available procedure, even if exact risk improvement is unavailable. It is a weaker guarantee than complete-class domination: it does not produce a single replacement rule, and it does not promise uniform closeness or an attained limiting risk. Keeping those conclusions separate prevents an asymptotic comparison from being mistaken for an exact decision-theoretic improvement.

Check Your Understanding

Use the definitions and examples above to answer these questions.

State the essential-completeness condition using a limit superior, and explain which quantities may depend on the parameter value.
Why does a complete class automatically satisfy the essential-completeness condition?
In the first worked example, which parameter value prevents any \(a_n\) from dominating \(b\), and what happens to the risk at that parameter along the sequence?
In the continuous-parameter example, why does pointwise convergence hold even though the supremum risk discrepancy does not decrease?
Why is “sequential pointwise closure” more precise than unqualified “pointwise closure” for an arbitrary parameter space?