From Bayes Optimality to Admissibility
A Bayes rule minimizes risk averaged over a prior. Admissibility, by contrast, concerns risk at every parameter value: a rule is inadmissible if another rule does at least as well everywhere and strictly better somewhere. A connection between these two criteria is available when risks vary continuously with the parameter and the prior puts positive probability in every open neighborhood. Then a strict improvement cannot be confined to an isolated point: it persists nearby, where the prior assigns positive mass.
For a rule \(\delta\), its integrated risk under \(\pi\) is \(\int_\Theta R(\theta,\delta)\,\pi(d\theta)\). A Bayes rule minimizes this quantity over the decision rules being considered. In the result below, risks are finite and continuous on \(\Theta\), the prior is proper with full support, and the Bayes risk of \(\delta^*\) is finite. These conditions ensure the integrals used in the argument are well-defined; under pointwise dominance, the competing rule also has finite integrated risk.
The uniqueness condition is stronger than necessary for this conclusion: Bayes optimality alone suffices. Uniqueness can be useful for identifying the Bayes rule, but the contradiction establishing admissibility only needs its integrated-risk optimality.
Why a Strict Pointwise Improvement Matters to the Prior
The key step is local. If a rule \(\delta\) dominates \(\delta^*\), define the risk difference \(g(\theta)=R(\theta,\delta^*)-R(\theta,\delta)\). It is nonnegative everywhere. If dominance is strict at some \(\theta_0\), then \(g(\theta_0)>0\). Continuity makes \(g\) positive by a fixed amount in a neighborhood of \(\theta_0\), and full support gives that neighborhood positive prior probability.
Proof. Set \(a=g(\theta_0)>0\). By continuity, there is an open neighborhood \(U\) of \(\theta_0\) on which \(|g(\theta)-a|<a/2\). Thus \(g(\theta)>a/2\) for \(\theta\in U\). Full support gives \(\pi(U)>0\), so
Proof of the theorem. Suppose, to the contrary, that a rule \(\delta\) dominates \(\delta^*\). The continuous risk difference \(g(\theta)=R(\theta,\delta^*)-R(\theta,\delta)\) is nonnegative everywhere and positive at some \(\theta_0\). The lemma shows that its prior integral is strictly positive. Therefore
This contradicts the Bayes optimality of \(\delta^*\), which requires its integrated risk to be no greater than that of any competing rule. Hence no rule dominates \(\delta^*\), and it is admissible. \(\square\)
Worked Examples
Worked Example: A Bernoulli Estimator Under Squared Error
Let \(X\sim\operatorname{Bernoulli}(\theta)\), where \(\Theta=[0,1]\), and estimate \(\theta\) under squared-error loss \(L(\theta,a)=(a-\theta)^2\). Take the proper prior \(\theta\sim\operatorname{Beta}(2,2)\), whose density is \(6\theta(1-\theta)\) on \([0,1]\). This density is positive on \((0,1)\), so every nonempty relatively open subset of \([0,1]\) has positive prior probability.
For \(X=1\), the posterior is \(\operatorname{Beta}(3,2)\), with mean \(3/5\). For \(X=0\), it is \(\operatorname{Beta}(2,3)\), with mean \(2/5\). Under squared-error loss, the posterior expected loss is uniquely minimized by the posterior mean: for any posterior random parameter \(T\),
Thus the unique Bayes estimator is \(\delta^*(1)=3/5\), \(\delta^*(0)=2/5\). Its risk is
The two possible estimation errors are \(3/5-\theta\) and \(2/5-\theta\); they are not fixed in magnitude. The risk expression is finite and continuous on \([0,1]\). The theorem therefore implies that this unique Bayes estimator is admissible.
Worked Example: A Normal-Mean Estimator
Suppose \(X\sim N(\theta,1)\), with \(\theta\in\mathbb R\), and use squared-error loss. Put the proper prior \(N(0,1)\) on \(\theta\). Completing the square in the likelihood times prior gives the posterior distribution \(N(X/2,1/2)\), so the posterior mean \(X/2\) uniquely minimizes posterior expected squared error. Hence \(\delta^*(X)=X/2\) is the unique Bayes rule.
The normal prior has positive density at every real number, and therefore has full support. To check continuity of risk directly, write \(X=\theta+Z\), where \(Z\sim N(0,1)\). Then
This risk is finite and continuous for every \(\theta\in\mathbb R\). The theorem proves admissibility. The point is not that this risk is uniformly small; it grows with \(|\theta|\). The conclusion is that no other rule can have risk no greater at every parameter and strictly smaller at even one.
Worked Example: A Bayes Rule Under Absolute-Error Loss
Again let \(X\sim\operatorname{Bernoulli}(\theta)\), \(\theta\in[0,1]\), but now use absolute-error loss \(L(\theta,a)=|a-\theta|\) and the uniform prior \(\operatorname{Beta}(1,1)\). The prior is proper and has full support on \([0,1]\).
If \(X=0\), the posterior density is \(2(1-\theta)\), with distribution function \(2t-t^2\) for \(0\leq t\leq1\). Its median \(m_0\) solves \(2m_0-m_0^2=1/2\), yielding \(m_0=1-1/\sqrt2\). If \(X=1\), the posterior density is \(2\theta\), with distribution function \(t^2\), whose median is \(m_1=1/\sqrt2\). For a continuous posterior distribution with strictly increasing distribution function, the unique median uniquely minimizes expected absolute error. Thus the unique Bayes rule is \(\delta^*(0)=m_0\), \(\delta^*(1)=m_1\).
Its frequentist risk is
Each absolute-value term is continuous in \(\theta\), so the risk is finite and continuous on \([0,1]\). Full support and the theorem again imply admissibility.
Common Pitfall: A Point Is Not Necessarily Positive Prior Mass
A tempting but incomplete argument says, “The competitor is strictly better at \(\theta_0\), so its integrated risk is strictly better.” A continuous prior may assign probability zero to the singleton \(\{\theta_0\}\). For example, a normal prior gives zero mass to every individual point. The argument works because continuity extends the strict difference to a neighborhood, not because the point itself has positive mass.
Both ingredients matter. Full support ensures that the neighborhood has positive prior mass; continuity ensures there is a neighborhood on which the risk difference stays positive. If either condition fails, pointwise strict improvement need not create a strict integrated-risk improvement. The theorem is a sufficient admissibility criterion, not a claim that every Bayes rule under any prior is admissible.
Check Your Understanding
Use the definitions and argument above to answer the following.
- Why does a prior assigning zero probability to every singleton still assign positive probability to the neighborhood used in the theorem?
- In the theorem’s proof, where are continuity and full support used, respectively?
- Is uniqueness of the Bayes rule required for the admissibility conclusion? Explain.
- For the Bernoulli squared-error example, verify the risk at \(\theta=1/2\) from the displayed polynomial.
- What part of the argument fails if risks are discontinuous at the parameter where dominance is strict?