Countable Parameter Reduction in Complete-Class Proofs

Why a Countable Parameter Set Can Be Enough

Complete-class arguments often involve risk inequalities indexed by every parameter value in a potentially uncountable space. A useful reduction is possible when the parameter space has a countable dense subset and the relevant risks vary continuously: it is then enough to establish the inequalities on that subset. The key is that continuity controls the values between the points where the inequalities are checked.

This reduction is an analytic step, not a consequence of countability alone. Density supplies parameter values arbitrarily close to each point; continuity ensures the risk comparison persists in the limit. If risks can jump, inequalities on a dense set need not say anything about the points omitted from it.

Definition: Let \(\Theta\) be a metric parameter space. A subset \(D\subseteq\Theta\) is dense in \(\Theta\) if every nonempty open subset of \(\Theta\) intersects \(D\). It is countable dense if it is both countable and dense. For a decision rule \(\delta\), its risk at \(\theta\) is \(R(\theta,\delta)=E_\theta[L(\theta,\delta(X))]\), when this expectation is defined. A rule \(\delta_1\) weakly dominates \(\delta_0\) on \(\Theta\) if \(R(\theta,\delta_1)\leq R(\theta,\delta_0)\) for every \(\theta\in\Theta\).

For the main result, assume the two risks are finite real-valued functions. It is enough that their difference is continuous; continuity of both risks is a convenient, stronger condition that guarantees continuity of the difference. These conditions are alternatives, not equivalent statements: two identical discontinuous risks have a continuous difference, namely zero.

The Dense-Subset Extension

Theorem: Let \(\Theta\) be a metric space, let \(D\) be dense in \(\Theta\), and let \(g:\Theta\to\mathbb{R}\) be continuous. If \(g(\theta)\leq 0\) for every \(\theta\in D\), then \(g(\theta)\leq 0\) for every \(\theta\in\Theta\). In particular, if \(R(\cdot,\delta_1)-R(\cdot,\delta_0)\) is continuous and \(\delta_1\) has no greater risk than \(\delta_0\) on \(D\), then \(\delta_1\) has no greater risk than \(\delta_0\) on all of \(\Theta\).

Proof. Fix any \(\theta\in\Theta\). For each positive integer \(m\), density gives a point \(\theta_m\in D\) whose distance from \(\theta\) is less than \(1/m\). Thus \(\theta_m\to\theta\). The assumed inequalities give \(g(\theta_m)\leq 0\) for every \(m\), while continuity gives \(g(\theta_m)\to g(\theta)\). A limit of real numbers all at most zero is at most zero, so \(g(\theta)\leq 0\). Since \(\theta\) was arbitrary, the inequality holds throughout \(\Theta\). For the risk statement, take \(g(\theta)=R(\theta,\delta_1)-R(\theta,\delta_0)\). \(\square\)

The proof uses density to construct a sequence approaching each parameter. It therefore assumes a metric parameter space, as stated. In a general topological space, convergence along every sequence need not characterize continuity. One can formulate related results using nets, but the sequence proof above is the appropriate version for metric parameter spaces.

Worked Example: Extending a Risk Inequality in a Normal Model

Suppose \(X\sim N(\theta,1)\), with \(\theta\in[0,1]\), and use squared-error loss \(L(\theta,a)=(a-\theta)^2\). Compare the constant decision rules \(\delta_1(X)=0\) and \(\delta_0(X)=-1\). Verify the risk inequality on the countable dense set \(D=\mathbb{Q}\cap[0,1]\), then extend it.

The rules ignore \(X\), so their risks are simply their losses:

R(\theta,\delta_1)=\theta^2,\qquad R(\theta,\delta_0)=(\theta+1)^2.

Their risk difference is

g(\theta)=R(\theta,\delta_1)-R(\theta,\delta_0) =\theta^2-(\theta+1)^2=-2\theta-1.

For every \(\theta\in[0,1]\), including every rational \(\theta\) in that interval, \(-2\theta-1\leq -1<0\). Thus the inequality holds on \(D\); because \(g\) is continuous and \(D\) is dense, the theorem extends it to all of \([0,1]\). Directly, the risk difference is strictly negative everywhere, confirming the conclusion. This example is deliberately simple: the theorem’s value is that its proof does not require a separate direct calculation at every parameter.

The reduction applies to the risk difference, not necessarily to each risk separately. If both risks are continuous, their difference is continuous by the algebra of continuous functions. But it is sufficient for the difference alone to be continuous, even if the individual risks are not.

A Continuity Criterion for Risks

In applications, continuity of a risk may follow from continuity of the model and a domination condition on the integrand. Here is a useful version stated for a metric parameter space. The measure in the integral must be fixed as the parameter varies; otherwise the displayed dominated-convergence argument does not apply in this form.

Theorem: Let \(\Theta\) be a metric space with reference measure \(\nu\), and suppose \(R(\eta)=\int h_\eta(z)\,\nu(dz)\) is finite for parameters \(\eta\) near a fixed \(\theta\). Assume that for every sequence \(\theta_m\to\theta\), \(h_{\theta_m}(z)\to h_\theta(z)\) for \(\nu\)-almost every \(z\). Also assume there is a neighborhood \(U\) of \(\theta\) and a \(\nu\)-integrable function \(H\) such that \(|h_\eta(z)|\leq H(z)\) for every \(\eta\in U\), for \(\nu\)-almost every \(z\). Then \(R\) is continuous at \(\theta\).

Proof. Take any sequence \(\theta_m\to\theta\). Since \(U\) is a neighborhood, all sufficiently large \(m\) have \(\theta_m\in U\). For those indices, \(|h_{\theta_m}|\leq H\) almost everywhere, and \(H\) is integrable. The assumed almost-everywhere convergence and the dominated convergence theorem therefore give

R(\theta_m)=\int h_{\theta_m}\,d\nu \longrightarrow \int h_\theta\,d\nu=R(\theta).

The finitely many earlier terms do not affect convergence, so \(R(\theta_m)\to R(\theta)\) for every sequence converging to \(\theta\). In a metric space, sequential continuity is equivalent to continuity. Hence \(R\) is continuous at \(\theta\). \(\square\)

Worked Example: Continuous Risks in a Bernoulli Model

Let \(X\sim\operatorname{Bernoulli}(\theta)\), where \(\theta\in[0,1]\), and use squared-error loss. Compare \(\delta_1(X)=X/2\) with \(\delta_0(X)=0\). Calculate both risks and find where \(\delta_1\) has no greater risk.

Since \(X\) equals 1 with probability \(\theta\) and 0 with probability \(1-\theta\), the risk of \(\delta_1\) is

R(\theta,\delta_1) =\theta(1/2-\theta)^2+(1-\theta)(0-\theta)^2 =\theta(1/4-\theta+\theta^2)+(1-\theta)\theta^2 =\frac{\theta}{4}.

For the rule \(\delta_0(X)=0\), the squared error is always \(\theta^2\), so \(R(\theta,\delta_0)=\theta^2\). The difference is

R(\theta,\delta_1)-R(\theta,\delta_0) =\frac{\theta}{4}-\theta^2 =\theta\left(\frac14-\theta\right).

For \(\theta\in[1/4,1]\), this difference is at most zero, so \(\delta_1\) has no greater risk than \(\delta_0\) on that interval. In particular, the inequality holds on the countable dense subset \(\mathbb{Q}\cap[1/4,1]\) and extends throughout the interval. The explicit calculation also verifies the boundary case: at \(\theta=1/4\) the risks are equal, and for \(\theta>1/4\) the inequality is strict. The polynomial risk formulas are continuous, as the theorem requires.

Why Continuity Cannot Be Omitted

A dense subset can miss every point in a large set. If the risk comparison jumps at those omitted points, knowing the inequality on the dense subset cannot constrain the values at the jumps. The following example makes this failure explicit while keeping the model and risks well defined.

Worked Example: Dense-Set Domination Fails for Discontinuous Risks

Let \(\Theta=[0,1]\), and suppose the observation space contains a single point, so no data distinguish the parameter values. There are two actions, \(a\) and \(b\), with losses

L(\theta,a)=\mathbf{1}_{\{\theta\notin\mathbb{Q}\}}, \qquad L(\theta,b)=0.

Consider the rule \(\delta_a\) that always chooses \(a\) and the rule \(\delta_b\) that always chooses \(b\). Because the observation is deterministic and both rules are nonrandomized, their risks equal their losses:

R(\theta,\delta_a)=\mathbf{1}_{\{\theta\notin\mathbb{Q}\}}, \qquad R(\theta,\delta_b)=0.

On the countable dense set \(D=\mathbb{Q}\cap[0,1]\), both risks are zero, so \(R(\theta,\delta_a)\leq R(\theta,\delta_b)\) holds at every \(\theta\in D\). But at every irrational \(\theta\in[0,1]\), the risks are 1 and 0, respectively, and the inequality fails. The risk difference is discontinuous at every point: every neighborhood contains both rational and irrational parameters, where its values are 0 and 1. Thus the dense-subset theorem does not apply.

Scope, Use, and Common Pitfalls

In a complete-class proof, this reduction can replace an uncountable family of risk constraints by a countable family, provided the needed risk-difference continuity has already been established. Countability is useful for organizing the constraints—for example, when a construction handles one constraint at a time. It is not what proves the extension: the extension follows from density and continuity. Nor does the theorem create a countable dense subset; its existence is an assumption, as it is for separable metric parameter spaces.

The theorem transfers weak inequalities. If the risk inequality is strict at some point of \(D\), it remains true that the rules satisfy weak domination everywhere, and the strict inequality at that point also remains. Conversely, a strict inequality at a parameter outside \(D\) need not be strict at any particular nearby point of \(D\) without additional information; the theorem’s conclusion itself concerns only the weak inequality.

Dropping continuity. Density alone does not control a discontinuous risk difference; the rational-versus-irrational example shows the failure.
Checking the wrong function. The theorem needs continuity of the difference \(R(\cdot,\delta_1)-R(\cdot,\delta_0)\). Continuity of both risks guarantees this, but is stronger than necessary.
Calling conditions equivalent when they are not. A continuous difference does not imply that either risk is continuous. Identical discontinuous risks have difference zero.
Using sequences without a suitable space. The proof uses sequences and the equivalence of sequential continuity and continuity in metric spaces. Do not infer that equivalence for arbitrary topological spaces.
Forgetting the domain. The dense subset must be dense in the parameter space under consideration. A subset dense in a larger ambient space need not be dense in a restricted parameter space.

Key takeaway: In a metric parameter space, if the risk difference is continuous and is nonpositive on a dense subset, it is nonpositive everywhere. Countability makes the subset manageable; continuity is what carries the inequality to all parameter values.

Check Your Understanding

Use the hypotheses of the results above when answering.

State the metric-space theorem that extends a nonpositive continuous function from a dense subset to the full parameter space.
Why does continuity of both risks suffice for the risk-difference theorem, and why is continuity of the difference alone also sufficient?
In the Bernoulli example, at which parameter values does \(\delta_1(X)=X/2\) have strictly smaller risk than \(\delta_0(X)=0\)? Verify using the risk difference.
Which assumption in the dominated-convergence continuity criterion allows the proof to conclude continuity from convergence along every sequence?
In the rational-versus-irrational example, identify the dense subset, the parameter values where the claimed inequality fails, and the missing hypothesis that prevents extension.