Supporting Priors and Bayes Risk Hyperplanes

From a Risk Boundary to a Prior

A risk function records how a decision rule performs at every parameter value. When a convex set of such functions has a boundary point that cannot be improved in all coordinates, a supporting hyperplane can certify that fact. In decision theory, the important feature is that this hyperplane can be represented by averaging risk against a prior distribution. Its contact points then identify rules that minimize prior-weighted risk.

This connection requires care. A supporting hyperplane is initially a linear functional on risk functions; it is not automatically a prior. Positivity makes it correspond to a nonnegative measure, and normalization turns that measure into a probability distribution. Also, a particular rule's integrated risk is not the Bayes risk: the Bayes risk is the infimum of integrated risk over the specified class of rules.

Definition: Let \(\Theta\) be a compact Hausdorff parameter space, and let \(C(\Theta)\) be the space of real-valued continuous functions on \(\Theta\), with the supremum norm. For a decision rule \(\delta\), write its risk function as \(r_\delta\in C(\Theta)\). The attainable risk set is \(\mathcal R=\{r_\delta:\delta\in\mathcal D\}\). The positive cone is \(C_+(\Theta)=\{g\in C(\Theta):g(\theta)\geq 0\text{ for every }\theta\}\). The closed upper risk set is \(\mathcal U=\overline{\mathcal R+C_+(\Theta)}\), where the closure is in the supremum norm. The upper set includes risks that are no better than attainable risks.

Assume that \(\mathcal D\) includes randomized mixtures of its rules and that risk is affine under mixing. Then \(\mathcal R\), and hence \(\mathcal U\), is convex. The upper set is also upward closed: if \(u\in\mathcal U\) and \(g\in C_+(\Theta)\), then \(u+g\in\mathcal U\). The closure is useful because a risk limit can be relevant to complete-class arguments even when no rule attains it.

The Supporting-Hyperplane Result

For the following result, suppose \(\mathcal U\) has nonempty interior in \(C(\Theta)\). The supporting-hyperplane theorem for a closed convex set in a normed linear space gives a nonzero continuous linear functional at each boundary point. The next result explains why, for an upper risk set, this functional yields a prior.

Theorem: Let \(\mathcal U\) be a nonempty, closed, convex, upward-closed subset of \(C(\Theta)\) with nonempty interior. If \(r_0\in\partial\mathcal U\), there is a nonzero continuous linear functional \(\Lambda\) such that \(\Lambda(u)\geq\Lambda(r_0)\) for every \(u\in\mathcal U\), and \(\Lambda(g)\geq0\) for every \(g\in C_+(\Theta)\). There is a nonzero finite positive regular Borel measure \(\nu\) on \(\Theta\) such that \(\Lambda(f)=\int_\Theta f(\theta)\,\nu(d\theta)\) for all \(f\in C(\Theta)\). Consequently, \(\pi=\nu/\nu(\Theta)\) is a prior probability measure and \[ \int_\Theta u(\theta)\,\pi(d\theta)\ \geq\ \int_\Theta r_0(\theta)\,\pi(d\theta) \quad\text{for every }u\in\mathcal U. \]

Proof. By the supporting-hyperplane theorem, choose a nonzero continuous linear functional \(\Lambda\) supporting \(\mathcal U\) at \(r_0\), with the orientation \(\Lambda(u)\geq\Lambda(r_0)\). For any \(g\in C_+(\Theta)\) and \(t\geq0\), upward closure gives \(r_0+tg\in\mathcal U\). Thus \[ \Lambda(r_0)+t\Lambda(g)=\Lambda(r_0+tg)\geq\Lambda(r_0). \] Taking \(t>0\) shows \(\Lambda(g)\geq0\). So the functional is positive.

The Riesz representation theorem for positive continuous linear functionals on \(C(\Theta)\) gives a finite positive regular Borel measure \(\nu\) representing \(\Lambda\). Since \(\Lambda\) is nonzero, \(\nu\) is nonzero, so \(\nu(\Theta)>0\). Normalizing by its total mass gives a probability measure \(\pi\). Dividing the supporting inequality by \(\nu(\Theta)\) gives the asserted prior-weighted inequality. \(\square\)

The prior-weighted integral is the geometric meaning of the supporting functional. The hyperplane consists of functions having a fixed integral under \(\pi\); the risk set lies on one side of it. The prior need not assign positive mass, or positive density, everywhere. Positivity of the measure means only that it is nonnegative.

Why Contact Means Bayes

For a prior \(\pi\), the integrated risk, also called the prior risk, of a rule \(\delta\) is \(r_\pi(\delta)=\int_\Theta r_\delta(\theta)\,\pi(d\theta)\). The Bayes risk under \(\pi\), relative to the specified class \(\mathcal D\), is \(\inf_{\delta\in\mathcal D}r_\pi(\delta)\). A rule is a Bayes rule if it attains this infimum. The distinction between the integrated risk of one rule and the infimum over rules is essential.

Theorem: Under the preceding theorem's assumptions, let \(\pi\) be the normalized measure induced by a supporting functional at \(r_0\in\partial\mathcal U\). Then \[ \inf_{\delta\in\mathcal D} r_\pi(\delta) =\int_\Theta r_0(\theta)\,\pi(d\theta). \] Any rule \(\delta\) whose risk function satisfies \(r_\delta=r_0\) is a Bayes rule. More generally, a rule is a Bayes rule exactly when its risk lies on the supporting hyperplane: \[ r_\pi(\delta)=\int_\Theta r_0(\theta)\,\pi(d\theta). \]

Proof. Since every attainable risk belongs to \(\mathcal U\), the supporting inequality gives \(r_\pi(\delta)\geq\int r_0\,d\pi\) for every \(\delta\in\mathcal D\). Hence the infimum is at least the right-hand side. To establish the reverse inequality, use \(r_0\in\mathcal U=\overline{\mathcal R+C_+(\Theta)}\). There are \(r_{\delta_m}\in\mathcal R\) and \(g_m\in C_+(\Theta)\) such that \(r_{\delta_m}+g_m\) converges uniformly to \(r_0\). Since \(\pi\) is positive, \[ r_\pi(\delta_m)\leq\int_\Theta\{r_{\delta_m}(\theta)+g_m(\theta)\}\,\pi(d\theta). \] Uniform convergence implies that the right-hand side converges to \(\int r_0\,d\pi\). Therefore the infimum over rules is at most this value, proving equality. A rule attaining equality attains the infimum, so it is Bayes; conversely, every Bayes rule attains that value and its risk lies on the hyperplane. \(\square\)

There is an important qualification: the boundary point \(r_0\) need not itself be an attainable risk. The infimum integrated risk can therefore be approached without being attained. In that case the theorem identifies the Bayes-risk value, but it does not assert the existence of a Bayes rule. A contact rule is a rule whose risk actually attains equality.

Worked Examples of Supporting Priors

Worked Example: Uniform Prior and Squared-Error Loss

There is no observation. The parameter is \(\theta\in[0,1]\), an action is \(a\in[0,1]\), and the loss is \(L(\theta,a)=(a-\theta)^2\). Each action has risk \(r_a(\theta)=(a-\theta)^2\). Use the uniform prior to find the supporting value and contact action.

The prior-weighted risk of action \(a\) is \[ r_\pi(a)=\int_0^1(a-\theta)^2\,d\theta =a^2-2a\int_0^1\theta\,d\theta+\int_0^1\theta^2\,d\theta =a^2-a+\frac13. \] Completing the square gives \[ r_\pi(a)=\left(a-\frac12\right)^2+\frac1{12}. \] Thus the integrated risk is at least \(1/12\), with equality at \(a=1/2\). The corresponding risk function \(r_{1/2}(\theta)=(1/2-\theta)^2\) lies on the hyperplane \(\{f:\int_0^1 f(\theta)\,d\theta=1/12\}\). The uniform prior's functional supports the upper risk set there, and \(a=1/2\) is a Bayes rule. Here the Bayes risk is \(1/12\), while another action's integrated risk, for example \(a=0\), is \(1/3\).

Worked Example: A Nonuniform Prior Changes the Contact Rule

Keep the same parameter space, no-observation setup, actions, and squared-error loss, but use the prior density \(p(\theta)=2\theta\) on \([0,1]\). It is a probability density because \(\int_0^1 2\theta\,d\theta=1\). Find the contact action.

The first two prior moments are \[ E_\pi[\theta]=\int_0^1 2\theta^2\,d\theta=\frac23,\qquad E_\pi[\theta^2]=\int_0^1 2\theta^3\,d\theta=\frac12. \] Consequently, \[ r_\pi(a)=E_\pi[(a-\theta)^2] =a^2-2aE_\pi[\theta]+E_\pi[\theta^2] =a^2-\frac43a+\frac12 =\left(a-\frac23\right)^2+\frac1{18}. \] The integrated risk is minimized at \(a=2/3\), and the minimum, hence the Bayes risk, is \(1/18\). The risk function \(r_{2/3}\) contacts the supporting hyperplane defined by integration against this prior. For comparison, action \(a=0\) has integrated risk \(1/2\), so its gap above the Bayes risk is \(1/2-1/18=4/9\). This calculation shows how changing the prior changes both the supporting functional and its contact rule.

Worked Example: Absolute Loss and a Strictly Higher Risk

Again let \(\theta\in[0,1]\), with no observation and action \(a\in[0,1]\), but now use \(L(\theta,a)=|a-\theta|\) and the uniform prior. Calculate the supporting value and compare actions \(a=1/2\) and \(a=4/5\).

For any \(a\in[0,1]\), split the integral at \(a\): \[ r_\pi(a)=\int_0^a(a-\theta)\,d\theta+\int_a^1(\theta-a)\,d\theta =\frac{a^2}{2}+\frac{(1-a)^2}{2} =\left(a-\frac12\right)^2+\frac14. \] The minimum is \(1/4\), attained at \(a=1/2\). Its risk function contacts the hyperplane \(\{f:\int_0^1 f(\theta)\,d\theta=1/4\}\), so \(a=1/2\) is Bayes. At \(a=4/5\), the integrated risk is \[ \left(\frac45-\frac12\right)^2+\frac14 =\frac9{100}+\frac{25}{100} =\frac{17}{50}. \] Its excess over the Bayes risk is \(17/50-1/4=9/100\). Thus this risk lies strictly on the risk-set side of the hyperplane rather than on the contact face.

Using the Geometry Correctly

A supporting prior gives a weighted certificate: every rule has integrated risk at least the supporting value, and every contact rule attains the minimum. This is particularly useful when comparing risk functions that cannot be ordered pointwise. One rule may do better at some parameter values and worse at others; integration against the supporting prior balances those differences in a way that certifies optimality for that prior.

The result concerns Bayes optimality for the induced prior, not automatic admissibility. A Bayes rule need not be uniquely optimal, and a prior may fail to distinguish rules whose risks differ only where the prior puts no mass. Nor does a supporting hyperplane imply strict positivity of its weights. Strictly positive priors and conclusions about admissibility require additional arguments.

Pitfall: calling one rule's integral the Bayes risk. The incorrect reasoning is: “\(\delta\) has integrated risk \(c\), so \(c\) is the Bayes risk.” That only computes \(r_\pi(\delta)\). The correction is to compare it with every rule and establish \(\inf_{\gamma\in\mathcal D}r_\pi(\gamma)=c\); only then is it the Bayes risk.
Pitfall: assuming every boundary point is an attainable risk. A point in the closed upper risk set may be only a limit. The supporting value can be an infimum even if no rule attains it. To claim a Bayes rule exists, verify that some rule's risk actually contacts the hyperplane.
Pitfall: treating the prior as automatically full support. Positivity of the functional yields a nonnegative measure, not a measure that assigns positive mass to every open set. Claims requiring full support need that condition separately.

Key takeaway: For an upper risk set, a supporting functional is positive. On a compact parameter space it is integration against a finite positive measure, which can be normalized to a prior. The infimum integrated risk equals the supporting value, and precisely the rules attaining that value are the Bayes rules for the prior.

Check Your Understanding

Use the definitions and supporting-hyperplane argument to answer the following questions.

Why does upward closure of \(\mathcal U\) force a supporting functional to be nonnegative on every nonnegative continuous function?
What additional step turns the positive measure representing a functional into a prior probability measure?
Explain why the integrated risk of a specified rule need not equal the Bayes risk.
If a supporting point belongs only to the closure of attainable risks, what can be concluded about the Bayes-risk value, and what cannot be concluded about existence of a Bayes rule?
In the uniform squared-error example, compute the integrated risk at \(a=1\) and its difference from the Bayes risk.