Why Audit the Hypotheses?
A Wald-type complete-class result is not a general license to replace every decision rule by a Bayes rule. Its conclusions depend on properties of the procedures’ attainable risks: in particular, whether those risks form a compact convex set and whether limits of procedures behave well under risk. A useful audit asks not only whether a hypothesis appears in a theorem, but what step of its proof needs it.
The result below is deliberately stated in a finite-parameter setting with explicit regularity assumptions. It gives a precise reference point for auditing broader arguments. When a condition is relaxed, the examples identify which conclusion is no longer guaranteed; they do not assert that every failure must produce a counterexample.
A Finite-State Complete-Class Result
Let the parameter set be \(\Theta=\{\theta_1,\ldots,\theta_m\}\), and let the observation take values in a finite set \(\mathcal X\). Under \(\theta_i\), suppose \(X\) has probability mass function \(p_i(x)\). Let the action space \(A\) be a compact metric space, and let \(L_i(x,a)\) be the loss at parameter \(\theta_i\), observation \(x\), and action \(a\). Assume each \(L_i(x,\cdot)\) is continuous and bounded.
A randomized decision rule \(\delta\) assigns to each observation \(x\) a probability distribution \(\delta(\cdot\mid x)\) on \(A\). Its risk at \(\theta_i\) is \(R(\theta_i,\delta)=\sum_{x\in\mathcal X}p_i(x)\int_A L_i(x,a)\,\delta(da\mid x)\). A rule \(\delta'\) dominates \(\delta\) if \(R(\theta_i,\delta')\leq R(\theta_i,\delta)\) for every \(i\), with strict inequality for at least one \(i\). A rule is admissible if no rule in the specified class dominates it.
Here “Bayes” means minimizing prior-weighted risk over the specified class of randomized procedures. To see why the theorem’s geometry matters, write each rule’s risk vector as \(r(\delta)=(R(\theta_1,\delta),\ldots,R(\theta_m,\delta))\), and let \(\mathcal R\) be the set of all such vectors. Randomization makes \(\mathcal R\) convex: mixing two rules with probability \(\alpha\) gives risk vector \(\alpha r+(1-\alpha)s\). Also, the space of rules is a finite product of spaces of probability measures on compact \(A\); it is compact in the weak topology. Continuity and boundedness of the losses make each risk coordinate continuous on that space. Consequently \(\mathcal R\) is compact.
Proof: From Risk Geometry to Priors
First consider an admissible rule with risk vector \(r\). Form the upper risk set \(Q=\mathcal R+\mathbb R_+^m\), where \(\mathbb R_+^m\) is the set of vectors with nonnegative coordinates. Thus \(q\in Q\) means that some attainable risk vector is no larger than \(q\) in any coordinate.
The set \(Q\) is convex because both \(\mathcal R\) and \(\mathbb R_+^m\) are convex. It is closed as well: if \(q_k=r_k+c_k\) converges, compactness of \(\mathcal R\) gives a subsequence with \(r_k\to r_*\in\mathcal R\); then \(c_k=q_k-r_k\) converges to \(q-r_*\in\mathbb R_+^m\), so \(q\in Q\). Admissibility implies that \(r\) is a boundary point of \(Q\). Otherwise, for some \(\varepsilon>0\), the vector \(r-\varepsilon(1,\ldots,1)\) would be in \(Q\), giving an attainable risk vector strictly below \(r\) in every coordinate.
The supporting-hyperplane theorem for a closed convex set at a boundary point gives a nonzero vector \(w\) such that \(w\cdot(q-r)\geq0\) for all \(q\in Q\). Since \(r+t e_i\in Q\) for every \(t\geq0\), where \(e_i\) is the \(i\)th coordinate vector, this inequality forces \(w_i\geq0\) for every \(i\). As \(w\ne0\), \(\sum_i w_i>0\). Normalize by defining \(\pi_i=w_i/\sum_jw_j\). Then \(\pi=(\pi_1,\ldots,\pi_m)\) is a prior, and for every procedure with risk vector \(s\in\mathcal R\),
where \(\delta_0\) is the admissible rule with risk vector \(r\). Thus \(\delta_0\) is Bayes.
For the other claim, take any procedure with risk vector \(r\). The set \(S=\{s\in\mathcal R:s_i\leq r_i\text{ for all }i\}\) is nonempty and compact. Choose \(s_*\in S\) minimizing \(\sum_i s_i\). No attainable vector can dominate \(s_*\): if it did, it would belong to \(S\) and have a strictly smaller coordinate sum. Therefore \(s_*\) is admissible, and the argument just given shows that a procedure attaining \(s_*\) is Bayes for some prior. It weakly dominates the original procedure. This proves both conclusions. \(\square\)
Worked Example: A Convex Risk Set and Its Supporting Prior
Worked Example: A Convex Risk Set and Its Supporting Prior
Suppose there are two parameter values, one possible observation, and two actions, \(a\) and \(b\). Their risk vectors are \(r(a)=(0,5)\) and \(r(b)=(5,0)\). A randomized rule chooses \(a\) with probability \(t\), where \(0\leq t\leq1\).
Its risk vector is \(r(t)=t(0,5)+(1-t)(5,0)=(5(1-t),5t)\). If \(t\) increases, the first risk decreases and the second increases. Therefore no different \(t\) has risks no greater in both coordinates, and every such rule is admissible.
For the prior \(\pi=(\pi_1,\pi_2)\), the prior risk is \(5\pi_1(1-t)+5\pi_2t=5\pi_1+5t(\pi_2-\pi_1)\). If \(\pi_1>\pi_2\), this is minimized at \(t=1\); if \(\pi_2>\pi_1\), it is minimized at \(t=0\). If \(\pi_1=\pi_2\), every \(t\) minimizes it. Thus each risk vector is supported by a prior, as the theorem predicts.
Audit Convexity and Compactness Separately
Convexity and compactness do different jobs. Convexity supplies intermediate risks and makes a supporting-prior characterization possible. Compactness helps ensure that a limiting or minimizing risk is actually attained. Neither property should be silently substituted for the other.
Worked Example: Losing Convexity Under a Deterministic Restriction
Consider one observation, two parameter values, and a restricted class containing only two deterministic rules, with risk vectors \(A=(0,5)\) and \(B=(5,0)\). Now add a third available deterministic rule with risk vector \(C=(2,3.5)\). The set of three risk vectors is not convex.
The vector \(C\) is admissible in this restricted class: \(A\) has a smaller first coordinate but a larger second coordinate, while \(B\) has a larger first coordinate but a smaller second coordinate. Yet \(C\) is not Bayes for any prior. For weights \(w_1,w_2\geq0\), not both zero, Bayes optimality of \(C\) would require its weighted risk to be no larger than either alternative. Comparing it with \(A\) gives \(2w_1+3.5w_2\leq5w_2\), or \(w_1/w_2\leq0.75\) when \(w_2>0\). Comparing it with \(B\) gives \(2w_1+3.5w_2\leq5w_1\), or \(w_1/w_2\geq3.5/3\). These requirements are incompatible; the cases \(w_1=0\) or \(w_2=0\) also fail one of the comparisons.
The failure is substantive: \(C\) is admissible but unsupported by any prior in the restricted class. If randomization between \(A\) and \(B\) is restored, the mixture with weights \(0.6\) on \(A\) and \(0.4\) on \(B\) has risk \((2,3)\), which dominates \(C\). The example shows that changing the procedure class can change both convexity and admissibility.
Worked Example: Losing Compactness and an Attained Optimum
There is one parameter value and no informative observation. Let the action set be \(A=\{1,2,3,\ldots\}\), and let the loss at action \(a\) be \(L(a)=1/a\). The loss is bounded, but the action set is not compact and the risk infimum is not attained.
Every deterministic action \(a\) is dominated by action \(a+1\), because \(1/(a+1)<1/a\). Randomization does not restore an admissible rule: any probability distribution on the positive integers has positive expected loss, since \(1/a>0\) for every \(a\). Choose an integer \(N\) with \(1/N\) smaller than that expected loss. The constant action \(N\) then has strictly smaller risk. Thus no procedure is admissible, and the infimum risk \(0\) is not attained.
This example does not show that every noncompact problem lacks an admissible rule. It shows that compactness is a safeguard for attainment, not a cosmetic assumption that can be dropped without checking the argument.
Audit Boundedness, Measurability, and Risk Regularity
In the stated theorem, compactness of \(A\), continuity of the losses, and finiteness of the parameter and observation sets ensure bounded risks and continuous risk coordinates. Broader Wald-type arguments often need to check these properties separately, especially when using limits of procedures. The following examples isolate three distinct problems.
Worked Example: An Unbounded Loss Breaks Risk Convergence
Let \(X\) be uniformly distributed on \([0,1]\), and let a rule report an action in \([0,\infty)\). For each positive integer \(n\), define \(\delta_n(x)=n\) when \(0\leq x\leq1/n^2\), and \(\delta_n(x)=0\) otherwise. Let the loss be \(L(a)=a^2\).
For any \(\varepsilon>0\), once \(n>\varepsilon\), the probability that \(\delta_n(X)>\varepsilon\) is \(1/n^2\), which tends to zero. Thus \(\delta_n(X)\to0\) in probability. But the risks are \(E[L(\delta_n(X))]=n^2(1/n^2)=1\), whereas the limiting action \(0\) has risk \(L(0)=0\). Weak or probabilistic convergence of actions therefore does not by itself justify passing to the limit in risk when the loss is unbounded. A proof requiring that passage needs additional control, not just convergence of procedures.
Worked Example: A Nonmeasurable Rule Is Not a Statistical Procedure
Let \(X\) be uniform on \([0,1]\), and choose a non-Lebesgue-measurable subset \(B\) of \([0,1]\). The assignment \(\delta(x)=1\) for \(x\in B\) and \(\delta(x)=0\) otherwise is not a measurable decision rule. For the bounded loss \(L(a)=a\), the expression \(E[L(\delta(X))]\) is not defined as a Lebesgue expectation, because \(x\mapsto L(\delta(x))\) is nonmeasurable.
This is not an example of an admissible but non-Bayes procedure: the assignment is outside the class of valid procedures, so admissibility and risk are not defined for it. In a finite-observation theorem, measurability is automatic. In more general settings, measurable kernels and appropriate measurable-space conditions are needed to make risks and selections well-defined.
Worked Example: Failure of Lower Semicontinuity Prevents Minimization
Take one parameter value, no informative observation, and compact action space \(A=[0,1]\). Define the bounded measurable loss by \(L(0)=1\) and \(L(a)=a\) for \(a>0\). For \(a_n=1/n\), the risks satisfy \(L(a_n)=1/n\to0\), but the limiting action has risk \(L(0)=1\).
The infimum risk is \(0\), but no action attains it: \(L(a)>0\) for every \(a\in A\). In particular, the risk function is not lower semicontinuous at zero, since \(\liminf_n L(a_n)=0<L(0)\). Compactness alone does not guarantee a minimizing rule when risk regularity fails. Here there is no Bayes rule for the unique prior, because minimizing prior risk is exactly minimizing this loss.
A Practical Hypothesis Audit
Before applying a complete-class argument, identify the procedure class and check what each assumption is doing. The following distinctions help prevent a local failure from being mistaken for a universal impossibility.
| Condition to check | Role in the argument | What a failure can leave unproved |
|---|---|---|
| Convex attainable risks | Supports the risk-set separation argument. | An admissible risk vector may not be Bayes for any prior, as the deterministic example shows. |
| Compactness or another attainment condition | Helps ensure limiting or minimizing risks are attained. | An optimal or admissible procedure may fail to exist. |
| Boundedness or suitable integrability control | Can justify stability of risk under convergence. | Convergence of procedures may not imply convergence of risks. |
| Measurability | Makes a rule valid and its risk an expectation. | The proposed assignment may not define a statistical procedure at all. |
| Risk lower semicontinuity or continuity | Prevents limiting procedures from having unexpectedly larger risk. | A minimizing sequence may have no minimizing limit. |
These are not interchangeable requirements. For example, boundedness of a loss does not itself ensure lower semicontinuity, and compactness does not repair a nonmeasurable rule. Conversely, the failure of a sufficient hypothesis does not establish that the desired conclusion is false in every particular model; a separate argument may still prove it.
Common Pitfall: Treating a Sufficient Theorem as an Equivalence
A common mistake is to see that a Wald-type theorem assumes compactness, convexity, or continuity and infer that violating one of these conditions automatically makes every complete-class conclusion false. The correct inference is narrower: the cited proof can no longer use the step that depends on that condition. One must either verify a different sufficient condition or construct a counterexample to the particular conclusion being claimed.
A related mistake is to conclude that every Bayes rule is admissible. A prior with zero weight on some parameter values can leave room for strict improvement at those values without changing prior risk. The theorem above guarantees a Bayes representative that dominates any given rule, and that every admissible rule is Bayes for some prior. It does not reverse the second statement without extra conditions on the prior and the risk comparison.
Check Your Understanding
For each question, identify the relevant hypothesis and state exactly what conclusion is or is not justified.
- Why does randomization make the attainable risk set convex, and which part of the supporting-prior proof uses convexity?
- In the noncompact action example, why does boundedness of the loss not ensure that an optimal action exists?
- For the unbounded-loss example, calculate the probability that \(\delta_n(X)>\varepsilon\) for \(n>\varepsilon\), and compare it with the risk.
- Why is a nonmeasurable assignment not a counterexample involving an admissible rule?
- Does failure of a sufficient condition prove that a complete-class conclusion is false in every model? Explain the correct inference.