Uniform Risk Closure and Weighted Metrics

Why the Metric on Risks Matters

A sequence of decision rules can have risks that approach a target risk in one sense but not another. The distinction is important when a complete-class argument uses approximating risks: a small error at every parameter value is stronger than a small error after parameter-specific weights have been applied. On a finite parameter space, strictly positive weights cannot hide a large error indefinitely. On an infinite space, weights that become small can make substantial errors in the tail count little in the weighted distance.

For a decision rule \(\delta\), its risk at parameter \(\theta\) is \(R(\theta,\delta)=E_\theta[L(\theta,\delta(X))]\), when the loss is measurable and this expectation exists and is finite. We treat a risk function as the map \(\theta\mapsto R(\theta,\delta)\). A set of rules therefore produces a set of risk functions, and a choice of metric specifies what it means for one of those functions to approximate another.

Uniform and Weighted Distances

Definition: For two bounded real-valued functions \(r,s\) on a parameter space \(\Theta\), their uniform distance is \(d_\infty(r,s)=\sup_{\theta\in\Theta}|r(\theta)-s(\theta)|\). Choose a weight function \(w:\Theta\to(0,\infty)\) with \(\sup_{\theta}w(\theta)<\infty\). Their weighted sup-distance is \(d_w(r,s)=\sup_{\theta\in\Theta}w(\theta)|r(\theta)-s(\theta)|\). More generally, \(d_w\) is defined for real-valued functions whenever this supremum is finite. Strictly positive weights ensure that \(d_w(r,s)=0\) only when \(r=s\).

The uniform distance controls the largest absolute error, wherever it occurs. The weighted distance controls the largest weighted error. Scaling all weights by a fixed positive constant changes the numerical distance but not which sequences converge. We may therefore take weights bounded above by \(1\) when convenient.

d_w(r,s)\leq \left(\sup_{\theta\in\Theta}w(\theta)\right)d_\infty(r,s).

Thus uniform convergence implies weighted convergence whenever the uniform distances are finite. The reverse implication requires a lower bound on the weights, as the next result shows.

When the Two Metrics Agree

Theorem: Suppose \(\Theta\) is finite and every weight \(w(\theta)\) is strictly positive. Then uniform and weighted distances are equivalent: there are constants \(0<m\leq M<\infty\) such that, for all \(r,s\), \(m d_\infty(r,s)\leq d_w(r,s)\leq M d_\infty(r,s)\). Consequently, the two metrics have the same convergent sequences and the same closures.

Proof. Since \(\Theta\) is finite and all its weights are positive, their minimum \(m=\min_{\theta\in\Theta}w(\theta)\) is positive, and their maximum \(M=\max_{\theta\in\Theta}w(\theta)\) is finite. For every \(\theta\), \(w(\theta)|r(\theta)-s(\theta)|\leq M d_\infty(r,s)\); taking the supremum gives \(d_w(r,s)\leq M d_\infty(r,s)\). Also, for every \(\theta\), \(w(\theta)|r(\theta)-s(\theta)|\geq m|r(\theta)-s(\theta)|\). Taking suprema gives \(d_w(r,s)\geq m d_\infty(r,s)\). The inequalities show that a sequence has distance tending to zero in one metric exactly when it does in the other. They also give the same open sets and hence the same closures. \(\square\)

Worked Example: Comparing Distances on a Finite Parameter Space

Let \(\Theta=\{0,1,2\}\), with weights \(w(0)=1\), \(w(1)=1/2\), and \(w(2)=1/4\). Consider risk functions \(r=(0,2,1)\) and \(s=(1,1,3)\), listed in parameter order.

The coordinatewise absolute errors are \(1,1,2\). Therefore \(d_\infty(r,s)=2\). The weighted errors are \(1,1/2,1/2\), so \(d_w(r,s)=1\). Here \(m=1/4\) and \(M=1\), giving \( (1/4)d_\infty(r,s)=1/2\leq 1\leq 2=M d_\infty(r,s)\).

A weighted distance can be smaller than the uniform distance, but the positive minimum weight prevents it from overlooking the error at parameter \(2\). The theorem guarantees this for every pair of risk functions on this finite space.

The same conclusion applies on an infinite parameter space if the weights are bounded below by a positive constant. Specifically, if \(w(\theta)\geq m>0\) for every \(\theta\), then \(m d_\infty(r,s)\leq d_w(r,s)\); together with the upper bound, this makes the metrics equivalent. If the weights have no positive lower bound, the reverse inequality need not hold. That possibility makes weighted closure potentially larger than uniform closure.

Theorem: Let \(\mathcal A\) be a set of bounded risk functions on \(\Theta\), and suppose \(w\) is positive and bounded above. Its uniform closure is contained in its weighted closure. If, in addition, \(\inf_{\theta\in\Theta}w(\theta)>0\), the two closures are equal. If \(\Theta\) is finite, they are equal for every strictly positive weight function.

Proof. For any \(r,s\), the upper bound \(d_w(r,s)\leq M d_\infty(r,s)\), where \(M=\sup_\theta w(\theta)<\infty\), shows that every uniformly convergent sequence in \(\mathcal A\) converges in weighted distance to the same limit. Thus every point in the uniform closure belongs to the weighted closure. If \(w(\theta)\geq m>0\), then \(d_\infty(r,s)\leq d_w(r,s)/m\). Weighted convergence then implies uniform convergence, giving the reverse inclusion of closures. On a finite space, strict positivity implies such a lower bound by taking the minimum of the finitely many weights. \(\square\)

What Shrinking Weights Permit

Worked Example: A Tail Error Hidden by Weights

Take \(\Theta=\{1,2,\ldots\}\), \(w(k)=2^{-k}\), and define \(r_n(k)=1\) when \(k\geq n\), and \(r_n(k)=0\) when \(k<n\). Let the target risk function be \(r(k)=0\) for every \(k\).

For every \(n\), there is at least one \(k\geq n\) with \(|r_n(k)-r(k)|=1\), so \(d_\infty(r_n,r)=1\). In particular, \(r_n\) does not approach \(r\) uniformly. For the weighted distance, the error is zero when \(k<n\), and is \(2^{-k}\) when \(k\geq n\). Its largest value is at \(k=n\), giving \(d_w(r_n,r)=2^{-n}\), which tends to zero.

These functions can be realized as risks in a decision problem with no observation: use actions \(a_n\) and losses \(L(k,a_n)=\mathbf{1}\{k\geq n\}\). The rule choosing \(a_n\) has risk \(r_n\). The example shows that weighted closure can include a target that uniform closure excludes. It does not say that every risk set has different closures when weights shrink; it shows that the difference is possible.

Worked Example: Approximation of Unbounded Risks

Let \(\Theta=\{1,2,\ldots\}\) and \(w(k)=1/k\). Consider a decision problem with no observation, actions \(a\in[0,1]\), and loss \(L(k,a)=ak\). The constant rule choosing \(a\) has risk \(r_a(k)=ak\). Compare the target \(r_1(k)=k\) with \(r_{1-1/n}(k)=(1-1/n)k\), for \(n\geq2\).

At every \(k\), the absolute difference is \(k/n\). Consequently, \(d_w(r_1,r_{1-1/n})=\sup_{k\geq1}(1/k)(k/n)=1/n\), which tends to zero. But the unweighted difference has supremum \(\sup_{k\geq1}k/n=\infty\) for every \(n\). Thus the sequence converges in the weighted metric, while it is not uniformly close to the target at any stage.

This example uses unbounded risks, for which the usual uniform distance between these functions is infinite. It illustrates why weighted distances can be useful beyond bounded risk functions: they can measure approximation relative to the scale of the parameter-specific errors.

Interpreting Risk Closure Correctly

For a risk set \(\mathcal R=\{R(\cdot,\delta):\delta\in\mathcal D\}\), membership of a function \(r\) in the closure of \(\mathcal R\) means that risks of rules in \(\mathcal D\) can approach \(r\) under the chosen metric. Closure alone does not assert that any rule has risk exactly \(r\). Nor does it establish that replacing a rule by an approximating rule preserves a particular dominance relation. Those conclusions require their own arguments about the loss, the ordering of risks, and the approximation error.

On a finite parameter space, changing from uniform to a positive weighted metric changes numerical error bounds but not which risk functions are limits. On an infinite space, a weight that becomes arbitrarily small can discount errors at particular parameter values. Uniform convergence still implies weighted convergence when the weights are bounded above, but weighted convergence need not imply uniform convergence. Weighted convergence also implies pointwise convergence at each fixed \(\theta\), since \(|r_n(\theta)-r(\theta)|\leq d_w(r_n,r)/w(\theta)\) and \(w(\theta)>0\). The bound is specific to each fixed coordinate; it need not be uniform over \(\Theta\).

Common Pitfall

A common error is to reason that because every weight is positive, convergence in weighted distance must imply uniform convergence. Positivity gives control at each fixed parameter, but on an infinite space the weights may approach zero. For example, in the tail example, \(w(k)>0\) for every \(k\), yet the uniform error remains \(1\) while the weighted error tends to zero. The correction is to check whether the weights have a positive lower bound. If they do, weighted convergence controls uniform error; if they do not, the two closures can differ, and the risk set itself must be examined.

Key takeaway: Uniform closure requires approximation in the largest absolute risk error. A weighted metric can discount errors where weights are small. Positive weights give equivalent topologies on finite parameter spaces, but on infinite spaces equivalence requires a positive lower bound on the weights.

Check Your Understanding

Use the definitions and results above to answer the following questions.

Why do strictly positive weights on a finite parameter space have a positive minimum?
If \(w(\theta)\leq 1\) for all \(\theta\), which convergence always implies the other: uniform or weighted convergence?
For \(\Theta=\{1,2,\ldots\}\) and \(w(k)=2^{-k}\), what are the uniform and weighted distances from \(r_n(k)=\mathbf{1}\{k\geq n\}\) to the zero function?
What additional condition on the weights guarantees that weighted convergence implies uniform convergence?
Does membership in the weighted closure of a risk set guarantee that the limiting risk is attained by a rule? Explain.