Pointwise Closure of Attainable Risk Functions

Why Pointwise Closure Matters

A sequence of decision rules can have risk functions that approach a limit at every fixed parameter value, even when no rule in the class has that limiting risk. The distinction matters in complete-class arguments: an attainable risk is the risk of an actual rule, whereas a pointwise limit is initially only a function. Conflating them can turn a valid approximation into an unjustified claim that a replacement rule exists.

We make this distinction precise by viewing the risks of a class of rules as a set of functions on the parameter space. Pointwise convergence compares the risk values separately at each parameter; it does not require a single bound on the approximation error across all parameter values.

Risk Functions and Pointwise Convergence

Definition: Let $\Theta$ be a parameter space, let $\mathcal D$ be a class of decision rules, and let $R(\theta,\delta)$ denote the risk of rule $\delta\in\mathcal D$ at parameter $\theta$. The attainable risk set is $$ \mathcal R(\mathcal D)=\{R_\delta:\delta\in\mathcal D\},\qquad R_\delta(\theta)=R(\theta,\delta). $$ A sequence of risk functions $R_{\delta_m}$ converges pointwise to a function $r:\Theta\to\mathbb R$ if, for every fixed $\theta\in\Theta$, $$ \lim_{m\to\infty}R_{\delta_m}(\theta)=r(\theta). $$ The pointwise closure of $\mathcal R(\mathcal D)$ consists of the functions that can be approximated by attainable risks to arbitrary accuracy on every finite collection of parameter values.

For clarity, assume throughout that the risks and candidate limit functions are finite-valued. The same discussion can be adapted to extended nonnegative risks, with the corresponding extended-real topology. Pointwise convergence is a property of a sequence of functions; pointwise closure is a property of the whole set of functions. The distinction is important when $\Theta$ is uncountable, because not every point in a pointwise closure must be the limit of a sequence from the set.

A Finite-Coordinate Characterization

Theorem: A function $r:\Theta\to\mathbb R$ belongs to the pointwise closure of $\mathcal R(\mathcal D)$ if and only if, for every finite set $F\subseteq\Theta$ and every $\varepsilon>0$, there is a rule $\delta\in\mathcal D$ such that $$ |R_\delta(\theta)-r(\theta)|<\varepsilon\quad\text{for every }\theta\in F. $$

Proof. In the pointwise topology, a basic neighborhood of $r$ is specified by choosing finitely many parameter values $\theta_1,\ldots,\theta_k$ and positive tolerances $\varepsilon_1,\ldots,\varepsilon_k$. It consists of the functions $g$ satisfying $|g(\theta_i)-r(\theta_i)|<\varepsilon_i$ for every $i$. By definition, $r$ is in the closure of $\mathcal R(\mathcal D)$ precisely when every such neighborhood contains an attainable risk.

If the finite-set condition holds, apply it with $F=\{\theta_1,\ldots,\theta_k\}$ and $\varepsilon=\min_i\varepsilon_i$. The resulting attainable risk lies in the specified neighborhood, so every basic neighborhood meets $\mathcal R(\mathcal D)$, and $r$ is in its closure. Conversely, suppose $r$ is in the closure. For any finite $F$ and $\varepsilon>0$, the set of functions satisfying $|g(\theta)-r(\theta)|<\varepsilon$ for all $\theta\in F$ is a neighborhood of $r$. It must contain an attainable risk, which gives the required rule. $\square$

This characterization needs only finitely many coordinates at a time. If $\Theta$ is countable, it also yields a sequential characterization.

Corollary: If $\Theta=\{\theta_1,\theta_2,\ldots\}$ is countable, then $r$ is in the pointwise closure of $\mathcal R(\mathcal D)$ if and only if there exists a sequence $\delta_m\in\mathcal D$ for which $R_{\delta_m}(\theta)\to r(\theta)$ at every $\theta\in\Theta$.

Proof. A pointwise limit of attainable risks is in the closure: any neighborhood restricts finitely many coordinates, and convergence makes all those restrictions hold for sufficiently large $m$. For the reverse direction, use the theorem with $F_m=\{\theta_1,\ldots,\theta_m\}$ and $\varepsilon=1/m$. For each $m$, choose $\delta_m$ such that $|R_{\delta_m}(\theta_i)-r(\theta_i)|<1/m$ for $i\le m$. Fix $i$. For every $m\ge i$, this difference is less than $1/m$, which tends to zero. Thus $R_{\delta_m}(\theta_i)\to r(\theta_i)$, as required. $\square$

Worked Examples

Worked Example: A Finite Risk Set with an Attained Limit

Suppose there is no informative observation, the parameter space is $\Theta=\{0,1\}$, and the available actions are $A$ and $B$. The loss of $A$ is 0 at parameter 0 and 2 at parameter 1; the loss of $B$ is 2 at parameter 0 and 0 at parameter 1. A randomized rule chooses $A$ with probability $p$ and $B$ with probability $1-p$, where $0\le p\le1$.

At parameter 0, only choosing $B$ incurs loss 2, so $R_p(0)=2(1-p)$. At parameter 1, only choosing $A$ incurs loss 2, so $R_p(1)=2p$. Take $p_m=1/2+1/(2m)$, which is in $[0,1]$ for every $m\ge1$. Then

R_{p_m}(0)=1-\frac{1}{m}\longrightarrow 1, \qquad R_{p_m}(1)=1+\frac{1}{m}\longrightarrow 1.

The pointwise limit is the function with risk 1 at both parameter values. It is attained: setting $p=1/2$ gives $R_p(0)=R_p(1)=1$. Thus pointwise limits need not be unattainable. This calculation also shows why the decision-rule class matters: randomized rules are included here, and the limiting risk is realized by one of them.

Worked Example: A Zero Limit That No Rule Attains

Let $\Theta=\{1,2,\ldots\}$, suppose there is no informative observation, and let the action space also be the positive integers. The loss at parameter $\theta$ from action $a$ is

L(\theta,a)=\mathbf 1\{a<\theta\}.

The deterministic rule that chooses action $m$ has risk $R_m(\theta)=\mathbf 1\{m<\theta\}$. Fix any $\theta$. Whenever $m\ge\theta$, the indicator is zero; hence $R_m(\theta)\to0$ as $m\to\infty$. The pointwise limit is the zero function.

No deterministic action has zero risk at every parameter: action $a$ has risk 1 at $\theta=a+1$. The same conclusion holds if randomized rules are allowed. Let $A$ be the random action of such a rule. Zero risk at every $\theta$ would require $P(A<\theta)=0$ for every positive integer $\theta$. In particular, taking $\theta=k+1$ gives $P(A\le k)=0$ for each $k$. But every positive-integer-valued action satisfies

P(A\in\{1,2,\ldots\}) =P\left(\bigcup_{k=1}^{\infty}\{A\le k\}\right) \le \sum_{k=1}^{\infty}P(A\le k)=0,

contradicting that $A$ takes a positive integer value with probability 1. Thus the zero function is in the pointwise closure of the attainable risks but is not itself an attainable risk, even when randomization is permitted.

Worked Example: A Limit Corresponding to an Excluded Action

Let $\Theta=(0,1)$, suppose there is no informative observation, and allow deterministic actions $a\in(0,1)$ under squared-error loss $L(\theta,a)=(a-\theta)^2$. For each integer $m\ge2$, choose $a_m=1/m$. Its risk at $\theta$ is

R_{a_m}(\theta)=\left(\frac{1}{m}-\theta\right)^2 =\theta^2-\frac{2\theta}{m}+\frac{1}{m^2} \longrightarrow \theta^2.

So the function $r(\theta)=\theta^2$ is a pointwise limit of attainable risks. It is not attained by any deterministic action $a\in(0,1)$. If it were, then $(a-\theta)^2=\theta^2$ for every $\theta\in(0,1)$, or $a^2-2a\theta=0$ for every such $\theta$. Subtracting this equation at two distinct parameter values forces $a=0$, which is excluded.

Randomization does not produce the limit either. If $A\in(0,1)$ is a random action, equality of its risk to $\theta^2$ for every $\theta$ would give $E[A^2]-2\theta E[A]=0$ for every $\theta$. Subtracting the equations for two distinct values of $\theta$ yields $E[A]=0$. Since $A>0$ almost surely, this is impossible: a nonnegative random variable with expectation zero must equal zero almost surely. The limit acts like the risk of the excluded action $a=0$, but excluding that action prevents attainment.

What Closure Does—and Does Not—Give

Pointwise closure records what can be approximated on any finite set of parameter values. This is useful when a sequence of rules improves or simplifies a risk function coordinate by coordinate, but the limit does not belong to the original rule class. The examples show two different reasons for nonattainment: the action needed for the limit may be absent, or no single action—or probability distribution over actions—may meet all the limiting risk requirements.

A common mistake is to reason, “The risks converge at every parameter, so the limiting function is the risk of a rule.” Convergence alone does not supply such a rule. In the countable example, every fixed parameter eventually has zero risk along the sequence, yet no randomized rule has zero risk at all parameters. The correction is to keep two claims separate: the limit belongs to the pointwise closure, and the limit is realized by a rule. The second requires a separate attainment argument.

A second pitfall is to treat pointwise approximation as though the same accuracy has been established over all parameter values at once. The finite-coordinate characterization only guarantees approximation on each chosen finite set. It does not, by itself, provide one uniform error bound over the whole parameter space. For complete-class reasoning, therefore, a pointwise closure statement can identify a limiting candidate, but it cannot silently replace that candidate with an attainable rule.

Check Your Understanding

Use the definitions and examples above to answer the following questions.

What does it mean for a sequence of risk functions to converge pointwise?
State the finite-coordinate condition for a function to belong to the pointwise closure of an attainable risk set.
In the countable-parameter example, why does zero risk at every parameter force a randomized action to have probability zero of being at most $k$, for every $k$?
Why is the limit in the squared-error example not attained by any allowed action?
What additional issue must be considered before claiming that a pointwise limit is the risk of a decision rule?