Noncompact Parameter Spaces and Uniformity Failure

Why Pointwise Risk Limits Can Mislead

A risk function describes a decision rule's expected loss at each parameter value. When a sequence of rules has risks converging at every fixed parameter, it is tempting to conclude that its worst-case performance also approaches the worst-case performance of the limit. On a noncompact parameter space, that conclusion can fail: the parameter values where the risks are largest can move farther away as the sequence progresses.

The key distinction is between checking the limit at each fixed \(\theta\) and controlling all \(\theta\) at once. A parameter value that witnesses a large risk for the \(n\)th rule may change with \(n\), so no single fixed value needs to retain that risk in the limit.

Definition: Let \(\Theta\) be a parameter space and let \(R_n:\Theta\to\mathbb{R}\) be a sequence of risk functions. The risks converge pointwise to \(R\) if, for every fixed \(\theta\in\Theta\), \(R_n(\theta)\to R(\theta)\). They converge uniformly if \(\sup_{\theta\in\Theta}|R_n(\theta)-R(\theta)|\to 0\). The worst-case risk of a rule with risk \(R\) is \(W=\sup_{\theta\in\Theta}R(\theta)\), possibly infinite.

Uniform convergence controls the error in risk simultaneously over the whole parameter space. Pointwise convergence does not. The difference is especially important when \(\Theta\) is noncompact, meaning that it cannot be covered by finitely many open sets from every open cover. For example, the real line is noncompact: risk peaks can keep escaping toward larger parameter values.

A Moving-Peak Construction

The basic mechanism can be seen directly in a sequence of bounded functions. Let \(\Theta=\mathbb{R}\), and define

r_n(\theta)=\max\{1-|\theta-n|,\,0\}.

Each function has a triangular peak of height \(1\) at \(\theta=n\), and vanishes whenever \(|\theta-n|\geq 1\). For any fixed \(\theta\), once \(n>|\theta|+1\), we have \(|\theta-n|>1\) and therefore \(r_n(\theta)=0\). Thus \(r_n(\theta)\to 0\) at every fixed \(\theta\). But \(\sup_{\theta\in\mathbb{R}}r_n(\theta)=r_n(n)=1\) for every \(n\). The pointwise limit is identically zero, while the worst-case risks never decrease.

Worked Example: Realizing the Moving Peak as a Risk

Consider an experiment in which the observation is deterministically \(X=\theta\), with parameter space \(\mathbb{R}\). The action space is \(\{0,1\}\), and the loss is \(L(\theta,a)=a\). Use a randomized rule \(\delta_n\) that chooses action \(1\) with probability \(r_n(X)\) and action \(0\) otherwise. Find its risk and compare its pointwise and worst-case limits.

At parameter \(\theta\), the observation equals \(\theta\), so the probability that the rule chooses action \(1\) is \(r_n(\theta)\). The loss is one for action \(1\) and zero for action \(0\). Consequently,

R(\theta,\delta_n) =E_\theta[L(\theta,\delta_n(X))] =P_\theta(\delta_n(X)=1) =r_n(\theta).

For each fixed \(\theta\), the risk is eventually zero, as shown above. However, at \(\theta=n\), the rule chooses action \(1\) with probability one, so \(R(n,\delta_n)=1\). Since all risks are at most one, \(\sup_{\theta}R(\theta,\delta_n)=1\) for every \(n\). This is a valid sequence of risks, not merely a sequence of arbitrary functions.

The deterministic observation in this construction makes the moving peak transparent. The same failure occurs in a familiar noisy model, even though the risk curves are smooth.

Worked Example: A Moving Alarm in a Normal Location Model

Suppose \(X\sim N(\theta,1)\), where \(\theta\in\mathbb{R}\). Let the action be either “alarm” or “no alarm,” with loss one for an alarm and zero otherwise. For integer \(n\), use the rule \(\delta_n\) that raises an alarm exactly when \(X\in[n-1,n+1]\). Compute its risk at a fixed parameter and its worst-case risk.

Write \(Z=X-\theta\), so \(Z\sim N(0,1)\). The risk is the alarm probability:

R(\theta,\delta_n) =P_\theta(n-1\leq X\leq n+1) =\Phi(n+1-\theta)-\Phi(n-1-\theta),

where \(\Phi\) is the standard normal distribution function. If \(\theta\) is fixed and \(n\) tends to infinity, both endpoints in this interval for \(Z\) tend to infinity. The probability therefore tends to zero. Equivalently, the event requires \(Z\geq n-1-\theta\), whose probability tends to zero.

To find the supremum, put \(t=\theta-n\). The risk becomes

g(t)=\Phi(1-t)-\Phi(-1-t).

Its derivative is \(g'(t)=\phi(1+t)-\phi(1-t)\), where \(\phi\) is the standard normal density. If \(t>0\), then \((1+t)^2>(1-t)^2\), so \(\phi(1+t)<\phi(1-t)\) and \(g'(t)<0\). If \(t<0\), the inequalities reverse and \(g'(t)>0\). Thus \(g\) is maximized at \(t=0\), or \(\theta=n\). The worst-case risk is

\sup_{\theta\in\mathbb{R}}R(\theta,\delta_n) =g(0)=\Phi(1)-\Phi(-1) =2\Phi(1)-1 \approx 0.6827.

So the risk tends to zero at every fixed parameter, but the worst-case risk stays at approximately \(0.6827\). For each \(n\), the largest risk is attained at a different parameter value, \(\theta=n\).

When Pointwise Convergence Does Control Risk Uniformly

Noncompactness makes uniform control harder, but it does not make it impossible. A useful contrast is that on a compact parameter space, pointwise convergence combined with a common continuity bound does imply uniform convergence. The common bound prevents narrow or sharply changing peaks from escaping detection between a finite set of parameter values.

Theorem: Let \((\Theta,d)\) be a compact metric space. Suppose \(f_n:\Theta\to\mathbb{R}\) converge pointwise to zero and are equicontinuous: for every \(\varepsilon>0\), there is a \(\delta>0\) such that \(d(\theta,\eta)<\delta\) implies \(|f_n(\theta)-f_n(\eta)|<\varepsilon\) for every \(n\). Then \(f_n\to 0\) uniformly on \(\Theta\).

In particular, the theorem applies to risk differences \(f_n(\theta)=R_n(\theta)-R(\theta)\) when these differences converge pointwise to zero and satisfy the stated common continuity condition.

Proof. Fix \(\varepsilon>0\). By equicontinuity, choose \(\delta>0\) such that \(d(\theta,\eta)<\delta\) implies \(|f_n(\theta)-f_n(\eta)|<\varepsilon/2\) for every \(n\). The open balls of radius \(\delta\), centered at points of \(\Theta\), cover \(\Theta\). Compactness gives a finite subcover with centers \(\theta_1,\ldots,\theta_m\). At each of these finitely many centers, pointwise convergence gives an index \(N_j\) such that \(|f_n(\theta_j)|<\varepsilon/2\) whenever \(n\geq N_j\). Let \(N=\max_{1\leq j\leq m}N_j\). For any \(\theta\in\Theta\), choose a center \(\theta_j\) with \(d(\theta,\theta_j)<\delta\). For \(n\geq N\), the triangle inequality gives

|f_n(\theta)| \leq |f_n(\theta)-f_n(\theta_j)|+|f_n(\theta_j)| <\frac{\varepsilon}{2}+\frac{\varepsilon}{2} =\varepsilon.

The bound holds for every \(\theta\), so \(\sup_{\theta\in\Theta}|f_n(\theta)|\leq\varepsilon\) for all \(n\geq N\). This is uniform convergence. \(\square\)

The finite subcover is the crucial step: it reduces control over all parameter values to control at finitely many centers, with equicontinuity transferring that control to nearby points. On a noncompact space there need not be a finite subcover. In the moving-peak examples, no fixed finite collection of parameter values can keep up with peaks moving farther out.

Worked Example: Uniform Convergence on Every Bounded Parameter Range

Return to the normal alarm rule, but restrict attention to a fixed range \([-M,M]\), where \(M>0\). Show that its risk converges uniformly to zero on this range, although it does not converge uniformly on all of \(\mathbb{R}\).

For \(n>M+1\), the alarm event \(X\in[n-1,n+1]\) implies \(X\geq n-1\). Since \(X=\theta+Z\) and \(\theta\leq M\), it follows that \(Z\geq n-1-M\). Hence, for every \(\theta\in[-M,M]\),

0\leq R(\theta,\delta_n) \leq P(Z\geq n-1-M) =1-\Phi(n-1-M).

The upper bound does not depend on \(\theta\), and it tends to zero as \(n\) increases. Therefore \(\sup_{|\theta|\leq M}R(\theta,\delta_n)\to 0\). But on the full parameter space, the supremum remains \(2\Phi(1)-1\). Uniformity on each fixed bounded range does not give uniformity over an unbounded union of such ranges.

Why Uniform Risk Control Matters

Worst-case comparisons ask about all parameter values, not just each parameter considered separately. A small risk error at every fixed \(\theta\), with the amount of error allowed to depend on \(\theta\), need not give a small worst-case error. To pass safely from risk functions to worst-case risks, a uniform bound is sufficient.

Result: If \(R_n\) and \(R\) have finite suprema and \(\sup_{\theta\in\Theta}|R_n(\theta)-R(\theta)|\leq \varepsilon_n\), then

\left|\sup_{\theta\in\Theta}R_n(\theta) -\sup_{\theta\in\Theta}R(\theta)\right| \leq \varepsilon_n.

Indeed, for every \(\theta\), \(R_n(\theta)\leq R(\theta)+\varepsilon_n\), so taking suprema gives \(\sup_\theta R_n(\theta)\leq\sup_\theta R(\theta)+\varepsilon_n\). Reversing the roles of \(R_n\) and \(R\) gives \(\sup_\theta R(\theta)\leq\sup_\theta R_n(\theta)+\varepsilon_n\). Combining the two inequalities proves the claim. Thus uniform convergence of risks controls the difference in worst-case risks; pointwise convergence alone does not provide this bound.

Common Mistake

Wrong reasoning: “For every fixed \(\theta\), the risk \(R_n(\theta)\) tends to zero, so eventually the risk is small for every parameter value.” The word “eventually” in pointwise convergence may depend on \(\theta\). There may be no single index after which the risk is small simultaneously for all \(\theta\).

Correction: To make a claim about worst-case risk, check a uniform bound such as \(\sup_\theta R_n(\theta)\to 0\), or use assumptions that genuinely imply uniform convergence. Compactness together with equicontinuity is one such route. On a noncompact parameter space, inspect whether large risks can occur at parameter values that escape every fixed bounded region.

Key takeaway: Pointwise convergence follows each fixed parameter; worst-case risk searches over the whole parameter space at every stage. On a noncompact space, moving peaks can make those two operations disagree. A uniform bound, not a pointwise limit alone, is what controls worst-case behavior.

Check Your Understanding

Use the definitions and arguments above to answer these questions.

For \(r_n(\theta)=\max\{1-|\theta-n|,0\}\), explain why \(r_n(\theta)\to 0\) for each fixed \(\theta\), but \(\sup_\theta r_n(\theta)\) does not tend to zero.
In the normal alarm example, which parameter value attains the largest risk for rule \(\delta_n\), and what is that risk?
Which step in the compactness theorem's proof cannot generally be carried out on a noncompact parameter space?
Why does uniform convergence of risk functions control the difference between their worst-case risks?
In the normal example, why does restricting \(\theta\) to a fixed bounded interval restore uniform convergence to zero?