Compact Exhaustion of Noncompact Parameter Spaces

Why Compact Restrictions Need a Tail Argument

A noncompact parameter space can be studied through an increasing sequence of compact subsets. On each compact set, one can ask for a rule with small maximum risk. But a rule that performs well on the current compact set may perform badly just beyond it. The central issue is therefore not only whether restricted minimax values converge, but whether the selected rules control risk outside the compact sets as they grow.

The transfer has two distinct parts: a bound on the compact region and a bound on its complement. The complement bound need not converge to the same value as the restricted bound. It is enough that it does not asymptotically exceed the limiting restricted minimax value.

Compact Exhaustions and Restricted Values

Definition: A compact exhaustion of a parameter space \(\Theta\) is a sequence of compact sets \(K_1\subseteq K_2\subseteq\cdots\) such that \(\bigcup_{m=1}^{\infty}K_m=\Theta\). For example, \(K_m=[-m,m]\) is a compact exhaustion of \(\mathbb{R}\).

Let \(\mathcal{D}\) be a common class of admissible decision rules, and let \(R(\theta,\delta)\) denote the risk of rule \(\delta\in\mathcal{D}\) at parameter value \(\theta\). Assume risks are nonnegative and measurable as needed to define them; they may be extended-real-valued. The full minimax value and the value restricted to \(K_m\) are

V=\inf_{\delta\in\mathcal{D}}\sup_{\theta\in\Theta}R(\theta,\delta), \qquad M_m=\inf_{\delta\in\mathcal{D}}\sup_{\theta\in K_m}R(\theta,\delta).

Because \(K_m\subseteq\Theta\), every rule's maximum risk on \(\Theta\) is at least its maximum risk on \(K_m\). Thus \(V\ge M_m\). Also, \(K_m\subseteq K_{m+1}\) implies \(M_m\le M_{m+1}\). In particular, if the \(M_m\) are bounded above, they converge to a limit \(M_*=\lim_{m\to\infty}M_m\), and \(V\ge M_*\). This lower bound alone does not give the reverse inequality: the rules used to obtain small restricted risks may have large risk outside \(K_m\).

A Tail-Control Transfer Theorem

For each \(m\), choose a rule \(\delta_m\) whose risk is nearly optimal on \(K_m\). Define its compact-region and tail bounds by

A_m=\sup_{\theta\in K_m}R(\theta,\delta_m), \qquad B_m=\sup_{\theta\in\Theta\setminus K_m}R(\theta,\delta_m).

If a complement is empty, take its supremum here to be \(0\), consistent with nonnegative risk. Since \(\Theta\) is the union of \(K_m\) and its complement, the full maximum risk of \(\delta_m\) is \(\max(A_m,B_m)\).

Theorem: Suppose \(K_m\) is a compact exhaustion, \(M_m\to M_*<\infty\), and for each \(m\) there is a rule \(\delta_m\in\mathcal{D}\) such that \(\sup_{\theta\in K_m}R(\theta,\delta_m)\le M_m+\alpha_m\), where \(\alpha_m\ge0\) and \(\alpha_m\to0\). If \(\limsup_{m\to\infty}B_m\le M_*\), then \(V=M_*\). Moreover, the full maximum risks of the selected rules converge to \(M_*\).

The condition on the tail is an upper bound on its eventual level, not a requirement that \(B_m\) converge to \(M_*\). The proof makes this distinction explicit.

Bound the full risk.
For every \(m\), \(C_m:=\sup_{\theta\in\Theta}R(\theta,\delta_m)=\max(A_m,B_m)\).

Control the limit superior.
The compact bound satisfies \(A_m\le M_m+\alpha_m\to M_*\), while \(\limsup_m B_m\le M_*\). Hence \(\limsup_m C_m\le M_*\).

Match the lower bound.
Since \(V\le C_m\) for every \(m\), \(V\le\liminf_m C_m\le M_*\). Since \(V\ge M_m\) for every \(m\), \(V\ge M_*\). Therefore \(V=M_*\).

Conclude convergence of full risks.
Now \(C_m\ge V=M_*\), and \(\limsup_m C_m\le M_*\). Thus \(C_m\to M_*\). \(\square\)

The key limit step uses \(\limsup_m\max(A_m,B_m)\le\max(\limsup_m A_m,\limsup_m B_m)\), with both limit superiors finite here. Since \(\limsup_m A_m\le M_*\) and \(\limsup_m B_m\le M_*\), the maximum has limit superior at most \(M_*\). In particular, the two bounds do not both have to converge to \(M_*\).

A practical consequence is that, for any \(\varepsilon>0\), sufficiently large \(m\) gives \(A_m\le M_*+\varepsilon\) and \(B_m\le M_*+\varepsilon\), so \(\delta_m\) is an \(\varepsilon\)-minimax rule on the full space. This conclusion uses both the near-optimal compact guarantee and the tail condition.

Worked Examples

Worked Example: A Discrete Model with Tail Risk Equal to the Limit

Let \(\Theta=\{1,2,\ldots\}\) with the discrete topology, and let the observation be constant, so it carries no information about \(\theta\). A randomized rule chooses an integer \(i\) according to probabilities \(q_i\), where \(q_i\ge0\) and \(\sum_{i=1}^{\infty}q_i=1\). Use zero-one loss: the loss is \(0\) if the chosen integer equals \(\theta\), and \(1\) otherwise. Then \(R(\theta,q)=1-q_\theta\). The finite sets \(K_m=\{1,\ldots,m\}\) form a compact exhaustion.

On \(K_m\), the uniform rule \(q_i=1/m\) for \(i\le m\), with \(q_i=0\) otherwise, has maximum risk \(1-1/m\). No rule can do better: if its maximum risk on \(K_m\) is \(r\), then \(q_i\ge1-r\) for every \(i\le m\), so \(1\ge\sum_{i=1}^m q_i\ge m(1-r)\). Therefore \(r\ge1-1/m\), and \(M_m=1-1/m\to1\).

For the selected uniform rule, the risk is \(1-1/m\) at each \(\theta\le m\), and it is \(1\) at every \(\theta>m\). Thus \(A_m=1-1/m\), \(B_m=1\), and the full maximum risk is \(1\). The tail bound does not fall below or converge to a smaller value: it equals \(M_*=1\). The theorem still applies, and indeed \(V=1\). To verify the full-space lower bound directly, any probability sequence \(q_i\) has \(q_i\to0\); otherwise infinitely many terms would be bounded below by some positive constant, contradicting \(\sum_iq_i=1\). Hence \(\sup_i(1-q_i)=1\) for every rule.

Worked Example: A Tail Bound Strictly Below the Limit

Let \(\Theta=\{1,2,\ldots\}\), and suppose the observation is deterministic: \(X=0\) if \(\theta=1\) or \(2\), and \(X=\theta\) if \(\theta\ge3\). The action is an integer estimate, with zero-one loss. Take \(K_m=\{1,\ldots,m\}\) for \(m\ge3\).

At \(X=0\), the observation cannot distinguish \(\theta=1\) from \(\theta=2\). If a rule assigns probabilities \(q_1\) and \(q_2\) to actions 1 and 2 after observing 0, then \(q_1+q_2\le1\). Its risks at the two parameter values are \(1-q_1\) and \(1-q_2\). At least one is at least \(1/2\): if both were less than \(1/2\), then \(q_1>1/2\) and \(q_2>1/2\), contradicting \(q_1+q_2\le1\). Thus \(M_m\ge1/2\).

Now use the rule that, after \(X=0\), selects 1 or 2 with probability \(1/2\) each, and, after \(X\ge3\), selects \(X\). Its risks at \(\theta=1,2\) are each \(1/2\), and its risk at every \(\theta\ge3\) is zero. It follows that \(M_m=1/2\) for all \(m\ge3\), and \(M_*=1/2\). For this same rule, \(B_m=0\) on \(\Theta\setminus K_m\). The full minimax value is \(1/2\): the compact lower bound gives \(V\ge1/2\), and the displayed rule gives \(V\le1/2\). Here the tail bound stays strictly below the exhausted value, which is entirely consistent with the theorem.

Worked Example: A Uniform Risk Bound in a Normal-Mean Model

Suppose \(X\sim N(\theta,1)\), where \(\theta\in\mathbb{R}\), and use squared-error loss \(L(\theta,a)=(a-\theta)^2\). Consider the rule \(\delta(X)=X\) and the compact exhaustion \(K_m=[-m,m]\). Write \(X=\theta+Z\), where \(Z\sim N(0,1)\). Then, for every \(\theta\),

R(\theta,\delta)=E_\theta[(X-\theta)^2] =E[Z^2] =\operatorname{Var}(Z)+(E[Z])^2 =1.

Consequently \(A_m=1\) and \(B_m=1\) for every \(m\): this particular rule has a full-space maximum risk of 1, regardless of how far the tail extends. This calculation supplies a global upper bound \(V\le1\). By itself it does not show that the restricted values \(M_m\) converge to 1 or that the rule is near-optimal on each \(K_m\); those are separate requirements if one wants to apply the transfer theorem to establish equality.

What Tail Control Can and Cannot Do

There are several ways to establish the required tail condition. The most direct is to calculate or bound \(\sup_{\theta\notin K_m}R(\theta,\delta_m)\) for the actual selected rules. Sometimes a single rule has a uniform full-space risk bound; sometimes the rules vary with \(m\), and their tail bounds must be tracked separately. In either case, the condition concerns the sequence of rules being used, not merely the model or the compact sets.

A common mistake is to infer a full-space guarantee from \(M_m\to M_*\) alone. Restricted values describe how well one can do on each \(K_m\), potentially using a different rule at every \(m\). They say nothing by themselves about the risk of those rules on \(\Theta\setminus K_m\). Another mistake is to demand \(B_m\to M_*\). That is stronger than needed: if \(A_m\to M_*\) and \(\limsup_m B_m\le M_*\), then \(\max(A_m,B_m)\to M_*\). For example, \(A_m=M_*+1/m\) and \(B_m=0\) have maximum converging to \(M_*\) whenever \(M_*>0\), even though \(B_m\) does not approach \(M_*\).

The discrete example also illustrates the boundary case \(B_m=M_*\) for every \(m\). The strictly smaller tail bound in the second example illustrates the other possibility. In both cases, it is the upper limit of the tail risk—compared with the limiting compact guarantee—that matters.

Key takeaway: Compact minimax values provide a lower bound on the full minimax value. To obtain the matching upper bound, select near-optimal rules on the compact sets and control their tail risks so that \(\limsup_m B_m\le M_*\). The tail bound may converge to \(M_*\), stay below it, or vary without converging.

Check Your Understanding

Use the definitions and transfer argument to answer the following questions.

Why does \(V\ge M_m\) hold for each \(m\), and what does this imply when \(M_m\to M_*\)?
State the two separate risk bounds needed to use the tail-control transfer theorem.
If \(A_m\to M_*\) and \(B_m\to c\) for some \(c<M_*\), what is the limit of \(\max(A_m,B_m)\)?
In the discrete zero-one-loss example, why must \(q_i\to0\) for any probability sequence on the positive integers?
Does a uniform full-space risk bound for a chosen rule alone prove that the rule is near-optimal on every \(K_m\)? Explain.