Why Compactness Enters a Complete-Class Theorem
In a finite parameter problem, a procedure has a finite risk vector, and convex geometry can identify procedures that are Bayes for some prior. With an arbitrary parameter space, risks are functions, so controlling any one finite collection of parameter values does not by itself control every value. Wald’s compactness-based argument connects the two: it first finds Bayes procedures that improve risk on each finite collection, then uses compactness and lower semicontinuity to obtain one procedure that works across the whole parameter space.
The conclusion is about a closure of Bayes procedures, not necessarily about Bayes procedures alone. This distinction matters: the dominating procedure may be a limit of Bayes procedures rather than a Bayes procedure for any one prior used along the way.
Risk Functions and the Theorem’s Hypotheses
For clarity, use the pointwise risk topology: a net \(d_\alpha\) converges in risk to a risk function \(r\) when \(R(\theta,d_\alpha)\to r(\theta)\) for each fixed \(\theta\). A prior with finite support has the form \(\pi=\sum_{i=1}^m p_i\delta_{\theta_i}\), where \(p_i\geq0\) and \(\sum_i p_i=1\). A procedure is Bayes for \(\pi\) if it minimizes its integrated risk \(\sum_i p_iR(\theta_i,d)\) over \(\mathcal D\).
The convexity hypothesis means mixtures of procedures are available and their risks mix linearly. Lower semicontinuity means that if \(d_\alpha\to d\), then \(R(\theta,d)\leq\liminf_\alpha R(\theta,d_\alpha)\) for every \(\theta\). A common sufficient route to compactness of \(\mathcal R\) is compactness of \(\mathcal D\) together with continuity of each risk coordinate. The theorem below separates that compact-risk-set requirement from lower semicontinuity, which is used when passing back to a procedure.
A Finite-Dimensional Supporting-Prior Result
The key geometric step concerns only a finite set \(F=\{\theta_1,\ldots,\theta_m\}\). Project each risk function onto its coordinates in \(F\). The resulting set \(C\subset\mathbb R^m\) is compact and convex because it is the projection of \(\mathcal R\).
Proof. The set \(C_x=\{z\in C:z_i\leq x_i\text{ for all }i\}\) is nonempty, since \(x\in C_x\), and compact. Choose \(y\in C_x\) minimizing \(\sum_i y_i\). There cannot be \(z\in C\) with \(z_i\leq y_i\) for all \(i\) and strict inequality in at least one coordinate: such a \(z\) would belong to \(C_x\) and have a smaller coordinate sum.
Thus \(C\) is disjoint from the open convex set \(y-(0,\infty)^m\). The separating-hyperplane theorem gives a nonzero vector \(\lambda\) with \(\lambda\cdot z\geq\lambda\cdot u\) for every \(z\in C\) and \(u\in y-(0,\infty)^m\). Finiteness of the supremum over this open set forces every \(\lambda_i\geq0\). Its supremum is then \(\lambda\cdot y\), so \(\lambda\cdot z\geq\lambda\cdot y\) for every \(z\in C\). Normalize by \(\sum_i\lambda_i>0\) and set \(p_i=\lambda_i/\sum_j\lambda_j\). These weights make \(y\) a minimizer of the weighted risk, and \(y\leq x\) by construction. \(\square\)
Wald’s Compactness-Based Conclusion
Proof. Write \(r_0=R(\cdot,d_0)\). Direct the set of finite subsets \(F\subset\Theta\) by inclusion. For each \(F\), apply the lemma to the projection of \(\mathcal R\) onto \(F\), with \(x\) the projection of \(r_0\). The lemma gives a risk vector \(b_F\in\mathcal R\) that is no greater than \(r_0\) on \(F\), and a prior supported on \(F\) for which \(b_F\) minimizes integrated risk. Choose a procedure \(d_F\) having risk vector \(b_F\); it is Bayes for that finite-support prior.
Compactness of \(\mathcal R\) gives a convergent subnet of the net \((b_F)\), with limit \(b\in\mathcal R\). For any fixed \(\theta\), the finite sets in this subnet eventually contain \(\theta\). Hence \(b_F(\theta)\leq r_0(\theta)\) eventually, and coordinatewise convergence gives \(b(\theta)\leq r_0(\theta)\). By the definition of \(\mathcal R\), \(b=R(\cdot,\bar d)\) for some \(\bar d\in\mathcal D\).
To obtain a procedure in the closure of the Bayes procedures, take the corresponding subnet of \((d_F)\). Compactness of \(\mathcal D\) gives a further convergent subnet, say \(d_F\to d_*\). Its risks still converge pointwise to \(b\). Lower semicontinuity now gives, for each \(\theta\), $$ R(\theta,d_*)\leq\liminf_F R(\theta,d_F)=b(\theta)\leq R(\theta,d_0). $$ The convergent subnet consists of finite-prior Bayes procedures, so \(d_*\) belongs to their closure. This proves the asserted domination. \(\square\)
The argument has two different compactness roles. Compactness of the risk set supplies a limit risk function after the finite-coordinate improvements; compactness of the procedure space supplies a limiting procedure. Lower semicontinuity ensures that this procedure’s risk is no larger than the limiting risk function. Omitting any of these steps can leave a limit risk vector without a procedure that attains or improves it.
Worked Examples
Worked Example: A Randomized Bayes Improvement in Three States
There is no observation. The state is one of \(\theta_1,\theta_2,\theta_3\), and the loss table is:
| Action | \(\theta_1\) | \(\theta_2\) | \(\theta_3\) |
|---|---|---|---|
| A | 0 | 5 | 5 |
| B | 5 | 0 | 5 |
| C | 5 | 5 | 0 |
| D | 4 | 4 | 4 |
Under the uniform prior, each of A, B, and C has prior risk \((0+5+5)/3=10/3\). Action D has prior risk \((4+4+4)/3=4\), so it is not Bayes. The randomized rule choosing each of A, B, and C with probability \(1/3\) has risk at every state $$ \frac{0+5+5}{3}=\frac{10}{3}. $$ Its integrated risk is also \(10/3\), the minimum; thus it is Bayes under the uniform prior. Since \(10/3<4\), it strictly dominates D at all three states. Randomization supplies the convex risk vector used in the argument.
Worked Example: A Nonuniform Prior Selects a Dominating Rule
Consider two states, \(\theta_1,\theta_2\), and actions U, V, and W with risk vectors U \(=(0,8)\), V \(=(8,0)\), and W \(=(3,3)\). Add an action Z with risk vector \((7,5)\). Let the prior assign probability \(1/4\) to \(\theta_1\) and \(3/4\) to \(\theta_2\).
The integrated risks are $$ r_\pi(\mathrm U)=\tfrac14(0)+\tfrac34(8)=6,\quad r_\pi(\mathrm V)=\tfrac14(8)+\tfrac34(0)=2, $$ $$ r_\pi(\mathrm W)=\tfrac14(3)+\tfrac34(3)=3,\quad r_\pi(\mathrm Z)=\tfrac14(7)+\tfrac34(5)=\tfrac{11}{2}. $$ Thus V is Bayes for this prior. Its risk vector \((8,0)\) does not dominate Z, because \(8>7\) at \(\theta_1\). However, the randomized rule choosing V with probability \(1/2\) and W with probability \(1/2\) has risk vector \((11/2,3/2)\), not Bayes for this prior automatically. In fact its integrated risk is \((1/4)(11/2)+(3/4)(3/2)=11/8+9/8=20/8=5/2\), so it is not Bayes. This calculation illustrates why an arbitrary mixture cannot be called Bayes without checking its weighted risk.
For a valid domination here, W has risk \((3,3)\), which is no greater than Z’s \((7,5)\) in either state and strictly smaller in both. Under the uniform prior, the integrated risks of U, V, W, and Z are respectively \(4,4,3,6\), so W is Bayes and gives the claimed improvement. The prior used to expose a dominating Bayes rule need not be the first prior considered.
Worked Example: Finite Priors in a Countable Parameter Problem
Let \(\Theta=\{1,2,\ldots\}\), with no observation and two actions a and b. Suppose their risks are \(R(k,\mathrm a)=1\) and \(R(k,\mathrm b)=1-1/(k+1)\). For any finite-support prior \(\pi\), its support is a finite set \(F\), and $$ r_\pi(\mathrm a)=1,\qquad r_\pi(\mathrm b)=\sum_{k\in F}\pi(k)\left(1-\frac1{k+1}\right) =1-\sum_{k\in F}\frac{\pi(k)}{k+1}<1. $$ The strict inequality follows because \(F\) is finite, each \(k+1\) is positive, and the prior weights sum to one. Thus b is Bayes for every finite-support prior. Also \(R(k,\mathrm b)<R(k,\mathrm a)\) for every \(k\). This example is a direct case where one finite-prior Bayes procedure already dominates; the theorem is more useful when the finite subsets require different Bayes procedures and only a limit can control all parameter values together.
Common Pitfall and Why the Closure Matters
A common error is to reason: “For every finite subset of parameter values, some Bayes procedure improves the original one; therefore one Bayes procedure improves it at all parameter values.” The conclusion does not follow. The procedure selected for one finite subset may differ from the procedure selected for another, and there may be no single Bayes procedure among them that works everywhere. The theorem instead extracts a limit using compactness, then invokes lower semicontinuity to preserve the risk inequalities.
A second error is to infer that the limit must itself be Bayes for one of the finite priors. Its role is different: it belongs to the closure of finite-prior Bayes procedures and is guaranteed to dominate the original procedure. This is why the result establishes essential completeness of the closure, not necessarily completeness of the unclosed Bayes class. In applications, check that the risk set really is compact in the topology being used; compactness of the parameter space by itself does not provide that property.
Check Your Understanding
Use the theorem’s hypotheses and proof structure to answer these questions.
- Why does the finite-dimensional lemma need convexity of the risk set?
- What is the role of the finite-support prior in the theorem’s proof?
- Explain why lower semicontinuity gives an inequality in the useful direction when Bayes procedures converge.
- Does the theorem guarantee that the dominating procedure is Bayes for one particular prior? Explain.
- Which compactness assumption ensures that the limiting risk function is itself realized by a procedure?