Why Quantitative Bounds Matter
In admissibility arguments, it is often useful to show that a candidate rule has integrated risk close to the Bayes risk for a proper prior. Saying that the two are “close” is not enough when the argument depends on how close: the estimate must identify what controls the gap. Two useful quantities are the amount of prior mass outside a region where the candidate behaves well, and a bound on the loss incurred there.
This tutorial develops a direct inequality for that situation. It also derives an exact posterior-regret identity, which can give sharper estimates when the candidate differs from a Bayes rule only on a small-probability set of observations. The bounds are elementary, but their direction matters: a small tail mass gives an upper bound on a deficit only when the candidate is suitably controlled on the high-mass region.
Integrated Risk and Its Deficit
Let \(X\) have distribution \(P_\theta\) at parameter \(\theta\in\Theta\), let \(a\in\mathcal A\) be an action, and let \(L(\theta,a)\) be a nonnegative measurable loss. A decision rule \(\delta\) is a measurable function of the observation taking values in \(\mathcal A\). Its risk is
Let \(\pi\) be a proper prior probability measure. The integrated risk is \(r(\pi,\delta)=\int_\Theta R(\theta,\delta)\,\pi(d\theta)\), and the Bayes risk is the infimum of integrated risk over the specified class of measurable decision rules.
Assume throughout the core-and-tail argument that \(0\leq L(\theta,a)\leq M\) for every parameter and action, where \(M<\infty\). Then \(0\leq R(\theta,\delta)\leq M\) for every rule. The nonnegativity is essential to the lower bound on Bayes risk below; a signed loss would require a different argument.
A Core-and-Tail Bound
Split the parameter space into a measurable core \(C\) and its complement \(T=\Theta\setminus C\), and write \(\alpha=\pi(T)\). Define the core integrated-risk infimum
This is an infimum under the restricted, generally subprobability, measure \(\pi\) on \(C\). No rule attaining the infimum is required. The candidate needs only to have core integrated risk within a specified tolerance of it.
Proof. For every rule \(\eta\), nonnegative risk on the tail gives \[ r(\pi,\eta) =\int_C R(\theta,\eta)\,\pi(d\theta)+\int_T R(\theta,\eta)\,\pi(d\theta) \geq \int_C R(\theta,\eta)\,\pi(d\theta)\geq b_C. \] Taking the infimum over \(\eta\) shows \(r_B(\pi)\geq b_C\). For the candidate, its assumed core bound and \(R(\theta,\delta)\leq M\) give \[ r(\pi,\delta) \leq b_C+\varepsilon+M\pi(T) =b_C+\varepsilon+M\alpha. \] Subtract \(r_B(\pi)\geq b_C\) to obtain \(\Delta_\pi(\delta)\leq\varepsilon+M\alpha\). Nonnegativity of the deficit follows from the definition of Bayes risk. \(\square\)
The inequality uses prior mass rather than the geometric size of \(T\). A region can be large in a parameter-space metric and still contribute little if it has small prior probability. Conversely, a narrow region can matter if it carries substantial prior mass. The bound also separates two sources of error: \(\varepsilon\) measures imperfect performance on the core, while \(M\alpha\) is the worst-case contribution from the tail.
Worked Example: A Sharp Finite-Parameter Bound
Consider a model with no informative observation, parameter space \(\Theta=\{\theta_1,\theta_2,\theta_3\}\), and two actions \(A,B\). The prior probabilities are \(0.45,0.45,0.10\), respectively. The loss table is
| Parameter | Prior probability | Loss for \(A\) | Loss for \(B\) |
|---|---|---|---|
| \(\theta_1\) | 0.45 | 0 | 1 |
| \(\theta_2\) | 0.45 | 1 | 0 |
| \(\theta_3\) | 0.10 | 0 | 1 |
Take \(C=\{\theta_1,\theta_2\}\) and \(T=\{\theta_3\}\). On the core, action \(A\) has integrated risk \(0.45\), as does action \(B\). Thus \(b_C=0.45\). Let \(\delta\) choose \(B\). Its core risk is exactly \(b_C\), so \(\varepsilon=0\). Since \(M=1\) and \(\alpha=0.10\), the theorem gives \(\Delta_\pi(\delta)\leq0.10\).
The full integrated risks are \(r(\pi,A)=0.45\) and \(r(\pi,B)=0.45+0.10=0.55\). Therefore \(r_B(\pi)=0.45\), and \(\Delta_\pi(\delta)=0.55-0.45=0.10\). The bound is attained exactly: in this example the tail-mass term cannot be reduced using only the stated loss bound and core optimality.
Posterior Regret Gives an Exact Identity
For some problems, a uniform loss bound over a parameter tail is crude. If a posterior Bayes rule is available, the integrated-risk deficit can instead be written as an average of posterior regrets. Assume a proper prior, a joint prior-predictive distribution for \((\theta,X)\), and a measurable Bayes rule \(\delta_B\) that minimizes posterior expected loss for almost every observation. Write \(Q\) for the prior-predictive distribution of \(X\), and define
Proof. By conditioning on \(X\), the integrated risk of any rule \(\eta\) is \[ r(\pi,\eta)=\int \rho(\eta(x)\mid x)\,Q(dx). \] For almost every \(x\), the Bayes property gives \(\rho(\delta_B(x)\mid x)\leq\rho(a\mid x)\) for every available action \(a\). Consequently, every rule has integrated risk at least \(r(\pi,\delta_B)\), and \(\delta_B\) attains the infimum, so \(r_B(\pi)=r(\pi,\delta_B)\). Subtracting the two conditional-risk integrals proves the identity. Each integrand is nonnegative almost everywhere. \(\square\)
In particular, if the posterior regret of \(\delta\) is at most \(c\) on an observation set \(E\), is zero outside \(E\), and \(Q(E)\leq q\), then \(\Delta_\pi(\delta)\leq cq\). More generally, if the regret is at most \(M\) on \(E\), the deficit is at most \(M Q(E)\). If regret is at least \(c\) on \(E\), the identity gives the lower bound \(\Delta_\pi(\delta)\geq cQ(E)\). Upper and lower bounds require opposite inequalities on regret; a small-set estimate alone does not establish both.
Worked Example: Correctly Signed Regret in a Binary Decision
Let \(\theta\in\{0,1\}\) have prior probabilities \(P(\theta=1)=0.6\) and \(P(\theta=0)=0.4\). The observation is \(X\in\{b,\bar b\}\), with \(P(b\mid\theta=1)=0.05\) and \(P(b\mid\theta=0)=0.0485\). Actions are 0 and 1, with loss 1 for an incorrect action and 0 for a correct one. The candidate rule chooses 0 at \(b\) and 1 at \(\bar b\).
At \(b\), the joint probabilities of \((\theta=1,b)\) and \((\theta=0,b)\) are \(0.6(0.05)=0.03\) and \(0.4(0.0485)=0.0194\). Hence \[ P(b)=0.03+0.0194=0.0494,\qquad P(\theta=1\mid b)=\frac{0.03}{0.0494}\approx0.6073. \] The posterior-optimal action at \(b\) is 1. Choosing action 0 there has posterior loss \(0.6073\), whereas action 1 has posterior loss \(1-0.6073\approx0.3927\). The regret of action 0 is therefore \[ 0.6073-0.3927 =2\left(\frac{0.03}{0.0494}\right)-1 \approx0.2146. \] The positive sign matters: reversing this subtraction would give a negative number, which cannot be regret.
At \(\bar b\), the joint probabilities are \(0.6(0.95)=0.57\) for \(\theta=1\) and \(0.4(0.9515)=0.3806\) for \(\theta=0\). Thus \(P(\theta=1\mid\bar b)=0.57/0.9506\approx0.5996\), so action 1 is also posterior-optimal there. The candidate differs from the Bayes rule only at \(b\). Its integrated deficit is exactly \[ P(b)\left\{2\left(\frac{0.03}{0.0494}\right)-1\right\} =0.0494\left(\frac{0.06}{0.0494}-1\right) =0.06-0.0494 =0.0106. \] For a direct check, the Bayes rule always chooses 1 and has integrated risk \(0.4\). The candidate’s integrated risk is \(0.6(0.05)+0.4(0.9515)=0.03+0.3806=0.4106\), giving the same deficit, \(0.4106-0.4=0.0106\).
Worked Example: Estimating a Tail-Mass Bound
Suppose there is no informative observation, \(\Theta=[0,1]\), and actions are 0 and 1, with losses \(L(\theta,0)=\theta\) and \(L(\theta,1)=1-\theta\). Both losses lie between 0 and 1, so \(M=1\). Let the prior density be 1 on \([0,0.8]\), 2 on \([0.9,1]\), and 0 elsewhere. Its total mass is \(0.8+2(0.1)=1\).
Take \(C=[0,0.8]\), so \(\alpha=\pi(T)=2(1-0.9)=0.2\). On \(C\), action 0 has integrated risk \(\int_0^{0.8}\theta\,d\theta=0.32\), while action 1 has integrated risk \(\int_0^{0.8}(1-\theta)\,d\theta=0.48\). Thus \(b_C=0.32\), attained by action 0. The core-and-tail bound for this candidate is \(\Delta_\pi(0)\leq 0+1(0.2)=0.2\).
Its full integrated risk is \(0.32+\int_{0.9}^1 2\theta\,d\theta=0.32+(1-0.81)=0.51\). Action 1 has risk \(0.48+\int_{0.9}^1 2(1-\theta)\,d\theta=0.48+0.01=0.49\). Since there are only two actions and no informative data, the Bayes risk is \(0.49\). The candidate deficit is \(0.51-0.49=0.02\), below the valid, but conservative, bound \(0.2\). This calculation shows how an explicit prior-mass estimate supplies a bound without requiring the tail contribution to be calculated exactly.
Common Mistakes and Practical Use
- Reversing the regret subtraction. Regret is candidate posterior loss minus Bayes posterior loss, so it is nonnegative. In the binary example the action-0 regret at \(b\) is \(2P(\theta=1\mid b)-1\), not \(1-2P(\theta=1\mid b)\).
- Using the tail bound without core control. Small \(\pi(T)\) does not by itself imply a small deficit. The candidate must also be within \(\varepsilon\) of the core infimum; otherwise its excess risk on \(C\) is uncontrolled.
- Subtracting an upper bound from an upper bound. To bound the deficit above, the proof needs an upper bound on the candidate’s full integrated risk and a lower bound on the Bayes risk. The core infimum supplies the latter because tail risks are nonnegative.
- Confusing parameter-tail mass with observation-event probability. The core-and-tail theorem uses \(\pi(T)\). The posterior-regret identity uses prior-predictive probability \(Q(E)\) of an observation set. These are different probabilities on different spaces.
A practical bound can be built in three steps: identify a core on which the candidate is optimal or nearly optimal, bound the prior mass outside that core, and multiply that mass by a valid uniform loss bound. If posterior regret is available, use its observation-level identity instead and bound regret on the set where the candidate differs from a posterior Bayes rule. In either approach, state clearly whether the estimate is an upper or a lower bound on the integrated-risk deficit.
Check Your Understanding
For each question, identify which bound or inequality justifies the conclusion.
- Under a proper prior, assume \(0\leq L\leq 3\), the tail has prior mass \(0.04\), and a candidate’s core integrated risk is within \(0.02\) of the core infimum. What upper bound follows for its deficit?
- Why does nonnegative risk imply that the full Bayes risk is at least the core integrated-risk infimum?
- In the binary example, compute the posterior regret at \(b\) by subtracting the posterior loss of action 1 from the posterior loss of action 0. Why must the result be nonnegative?
- If a rule has posterior regret at most \(0.5\) on an observation set with prior-predictive probability \(0.06\), and zero elsewhere, what upper bound follows from the posterior-regret identity?
- Explain why a small value of \(\pi(T)\) cannot control the deficit if the candidate is arbitrarily worse than core-optimal on \(C\).