The Blyth Argument from Vanishing Bayes-Risk Gaps

Why Vanishing Bayes-Risk Gaps Matter

A decision rule can be difficult to establish as admissible by examining its risk at one parameter value at a time. Blyth’s argument instead compares integrated risks: if another rule improves on a candidate throughout a region that the priors continue to notice, then that improvement contributes a positive amount to integrated risk. A sequence of priors for which the candidate’s Bayes-risk gaps tend to zero cannot accommodate such a fixed positive improvement.

The logic depends on two separate facts. First, dominance bounds the candidate’s Bayes-risk gap below by the integrated improvement of the competing rule. Second, the priors must give enough weight to where that improvement occurs. If the priors lose that weight, the integrated improvement can vanish too, and there is no contradiction.

Integrated Risk and the Bayes-Risk Gap

Definition: Let \(\Theta\) be a parameter space, let \(\pi\) be a probability measure on \(\Theta\), and let \(R(\theta,\delta)\) be the risk of a decision rule \(\delta\). The integrated risk is \(r(\pi,\delta)=\int_\Theta R(\theta,\delta)\,\pi(d\theta)\). The Bayes risk is \(r(\pi)=\inf_{\gamma}r(\pi,\gamma)\), where the infimum is over the allowed decision rules. When \(r(\pi,\delta)<\infty\), define the Bayes-risk gap of \(\delta\) by \(D(\pi,\delta)=r(\pi,\delta)-r(\pi)\). In this case \(0\le r(\pi)\le r(\pi,\delta)<\infty\), so the subtraction is well-defined.

Throughout, assume risks are measurable and nonnegative, so their integrated risks are defined as extended nonnegative real numbers. The finite-integrated-risk condition in the definition is essential: if \(r(\pi,\delta)=\infty\), then \(r(\pi)\) may also be infinite, and the expression \(\infty-\infty\) is not a Bayes-risk gap.

A rule \(\delta'\) dominates \(\delta\) if \(R(\theta,\delta')\le R(\theta,\delta)\) for every \(\theta\), with strict inequality for at least one parameter value. For the argument below, pointwise domination is not enough by itself: the strict improvement must contribute positively under the priors being considered.

Worked Example: A Finite-Parameter Risk Gap

There are two parameter values, \(0\) and \(1\), no informative observation, and two available actions. Their risks are \(R(0,a_0)=1,\ R(1,a_0)=1\), and \(R(0,a_1)=0.8,\ R(1,a_1)=1\). Let \(\delta=a_0\) and \(\delta'=a_1\). Under the prior with \(\pi(\{0\})=0.25\) and \(\pi(\{1\})=0.75\), calculate the Bayes-risk gap of \(\delta\).

The integrated risk of \(\delta\) is \(r(\pi,\delta)=0.25(1)+0.75(1)=1\). The integrated risk of \(\delta'\) is \(r(\pi,\delta')=0.25(0.8)+0.75(1)=0.2+0.75=0.95\). Since \(a_1\) has risk no greater than \(a_0\) at either parameter, it minimizes integrated risk among these two actions. Thus \(r(\pi)=0.95\) and \(D(\pi,\delta)=1-0.95=0.05\).

The pointwise improvement is \(0.2\) at parameter \(0\) and \(0\) at parameter \(1\); its prior integral is \(0.25(0.2)=0.05\), exactly the gap in this example.

The Basic Inequality

Theorem: Let \(\pi\) be a probability measure and suppose \(r(\pi,\delta)<\infty\). If \(\delta'\) dominates \(\delta\), then \(r(\pi,\delta')<\infty\) and \(D(\pi,\delta)\ge \int_\Theta [R(\theta,\delta)-R(\theta,\delta')]\,\pi(d\theta)\).

Proof. Set \(g(\theta)=R(\theta,\delta)-R(\theta,\delta')\). By domination, \(g\ge0\). Also \(0\le R(\theta,\delta')\le R(\theta,\delta)\), so \(r(\pi,\delta')\le r(\pi,\delta)<\infty\). Both integrated risks are finite, and linearity of the integral gives \(r(\pi,\delta)-r(\pi,\delta')=\int_\Theta g\,d\pi\). By the definition of Bayes risk, \(r(\pi)\le r(\pi,\delta')\). Subtracting finite quantities therefore yields \(D(\pi,\delta)=r(\pi,\delta)-r(\pi)\ge r(\pi,\delta)-r(\pi,\delta')=\int_\Theta g\,d\pi\). \(\square\)

The finiteness condition is not cosmetic. It ensures that the gap and the difference of integrated risks in the proof are defined. The nonnegative integrand \(g\) also makes the comparison direct: a positive integrated improvement forces a positive lower bound on the gap.

From a Strict Improvement to a Contradiction

Here is a useful sufficient condition for that lower bound to persist. Let \(\mu\) be a measure on \(\Theta\), and suppose the priors \(\pi_n\) have densities \(q_n\) with respect to \(\mu\). Write \(g(\theta)=R(\theta,\delta)-R(\theta,\delta')\). Assume \(g\) is measurable and nonnegative, and there is a measurable set \(B\) such that \(\int_B g\,d\mu>0\). If, for some \(c>0\), \(q_n(\theta)\ge c\) for \(\mu\)-almost every \(\theta\in B\) for all sufficiently large \(n\), then \(\int g\,d\pi_n=\int gq_n\,d\mu\ge c\int_B g\,d\mu>0\) for all such \(n\).

For completeness, strict positivity on a set of positive \(\mu\)-measure implies a positive integral: if \(g>0\) on a set \(A\) with \(\mu(A)>0\), then \(A\) is the union of the sets \(A_k=\{\theta\in A:g(\theta)\ge 1/k\}\), \(k=1,2,\ldots\). At least one \(A_k\) has positive measure, since a countable union of measure-zero sets has measure zero. Consequently, \(\int_A g\,d\mu\ge \mu(A_k)/k>0\).

The Blyth contradiction: Suppose \(r(\pi_n,\delta)<\infty\) and \(D(\pi_n,\delta)\to0\). If a rule \(\delta'\) dominates \(\delta\) and its integrated improvement satisfies \(\liminf_{n\to\infty}\int_\Theta [R(\theta,\delta)-R(\theta,\delta')]\,\pi_n(d\theta)>0\), then a contradiction follows. The basic inequality makes every gap at least as large as that improvement, whereas the gaps tend to zero.

In applications, a density lower bound on a region of positive improvement is one way to verify the positive lower limit. The essential condition is the positive lower limit itself; the density bound is a convenient sufficient condition, not part of the logical core of the contradiction.

Worked Example: A Continuous-Parameter Lower Bound

Take \(\Theta=[0,1]\), with actions \(a_0,a_1\) and no informative observation. Let their risks be \(R(\theta,a_0)=1\) and \(R(\theta,a_1)=1-\theta/2\). Consider any prior with density \(q\) relative to Lebesgue measure such that \(q(\theta)\ge1/2\) on \([1/2,1]\). Find a lower bound for the gap of \(\delta=a_0\).

For every \(\theta\in[0,1]\), \(R(\theta,a_1)\le R(\theta,a_0)\). Thus \(a_1\) is Bayes, and the integrated improvement is \(\int_0^1 [R(\theta,a_0)-R(\theta,a_1)]q(\theta)\,d\theta =\int_0^1 (\theta/2)q(\theta)\,d\theta\).

The integrand is nonnegative, so restricting the integral to \([1/2,1]\) and using the density bound gives \(D(\pi,a_0)=\int_0^1(\theta/2)q(\theta)\,d\theta \ge\int_{1/2}^1(\theta/2)(1/2)\,d\theta =\frac14[\theta^2/2]_{1/2}^{1} =\frac14(3/8)=3/32\).

Therefore no sequence of priors satisfying this same lower bound on \([1/2,1]\) can have gaps for \(a_0\) tending to zero. The strict improvement is spread over a region to which every prior in the sequence assigns enough density.

Why the Priors’ Weight Matters

A dominating rule may improve strictly at some parameter values while a chosen prior sequence assigns progressively less mass to them. In that case, the integrated improvement can vanish. Vanishing gaps alone do not prove admissibility; the sequence must also detect the improvement that domination would create.

Worked Example: Vanishing Gaps When Prior Mass Escapes

Let \(\Theta=[0,1]\), \(A=[0,1/2]\), and define two actions by \(R(\theta,a_0)=1\) and \(R(\theta,a_1)=1-\mathbf{1}_A(\theta)\), where \(\mathbf{1}_A\) equals one on \(A\) and zero elsewhere. For each integer \(n\ge2\), define a prior density \(q_n(\theta)=2/n\) on \(A\) and \(q_n(\theta)=2(1-1/n)\) on \((1/2,1]\).

This is a probability density: its integral over \(A\) is \((1/2)(2/n)=1/n\), and its integral over the complement is \((1/2)2(1-1/n)=1-1/n\); the two masses sum to one. Action \(a_1\) dominates \(a_0\), with improvement one on \(A\) and zero outside. Hence its integrated risk is \(1-\pi_n(A)=1-1/n\), while \(r(\pi_n,a_0)=1\). Since \(a_1\) is pointwise no worse, it is Bayes, and \(D(\pi_n,a_0)=1-(1-1/n)=1/n\to0\).

There is no contradiction: \(\pi_n(A)=1/n\to0\), so the priors lose mass exactly where the improvement occurs. This example shows why a strict improvement at some parameter values does not by itself yield the positive integrated lower bound needed in Blyth’s argument.

Common Pitfalls

Subtracting two infinite integrated risks. Writing \(D(\pi,\delta)=r(\pi,\delta)-r(\pi)\) without ensuring finiteness can produce the undefined expression \(\infty-\infty\). Define the gap only when \(r(\pi,\delta)<\infty\); then \(r(\pi)\le r(\pi,\delta)<\infty\) as well.
Treating strict improvement at one point as a positive integral. A singleton can have measure zero under a continuous prior. The argument needs positive integrated improvement, often established by improvement on a positive-measure set together with adequate prior density there.
Using vanishing gaps without checking what the priors cover. The gaps can vanish because the priors move away from the region of improvement. Verify a positive lower bound on integrated improvement before claiming a contradiction.

The method matters because it converts a global admissibility question into an explicit comparison of integrated risks. Its conclusion is only as strong as its hypotheses: finite gaps make the algebra valid, and priors that continue to register the strict improvement make the contradiction possible.

Check Your Understanding

For each question, identify which part of the gap argument is being used.

Why does \(r(\pi,\delta)<\infty\) ensure that \(D(\pi,\delta)\) is well-defined?
If \(\delta'\) dominates \(\delta\), derive the inequality relating \(D(\pi,\delta)\) to their integrated risk difference.
Suppose \(g\ge0\) is measurable and positive on a set of positive \(\mu\)-measure. Why must \(\int g\,d\mu>0\)?
What additional condition on a sequence of priors makes a positive improvement on a region yield a contradiction with gaps tending to zero?
In the escaping-mass example, why do the Bayes-risk gaps vanish despite strict domination?