Measurable Selection in Bayes Rule Constructions

Why Pointwise Minimization Is Not Enough

For each observed data value, it may be possible to find an action with the smallest posterior expected loss. But a statistical decision rule must choose an action measurably as the data vary. Without that condition, the pointwise choices need not define an allowable rule, and their risks cannot be used in a complete-class argument. The central task here is to turn pointwise minimization into a measurable function of the observation.

Definition: Let $X$ be an observation with values in a measurable space $(\mathcal X,\mathcal A)$, let $\mathcal{D}$ be an action space with its Borel sigma-algebra, and let $\Pi(\,\cdot\mid x)$ be a posterior probability distribution on the parameter space. For a loss $L(\theta,a)$, the posterior expected loss at observation $x$ and action $a$ is $$ q(x,a)=\int L(\theta,a)\,\Pi(d\theta\mid x). $$ A deterministic decision rule is a measurable function $\delta:\mathcal X\to\mathcal D$. It is a posterior-minimizing rule if $q(x,\delta(x))=\inf_{a\in\mathcal D}q(x,a)$ for every $x$ (or for almost every $x$, under the marginal distribution of the data).

The distinction is between two statements: for each fixed $x$, a minimizer exists; and there is one measurable function $\delta$ that chooses a minimizer at every $x$. The first is a pointwise optimization claim. The second is what is needed to construct a decision rule.

A Measurable-Minimization Result for Interval Actions

A convenient sufficient setting is a compact interval of actions and an objective continuous in the action. Assume $\mathcal X$ is a measurable space, $\mathcal D=[c,d]$ for finite $c\le d$, and $q:\mathcal X\times[c,d]\to\mathbb R$ has these properties:

For every fixed $a$, $x\mapsto q(x,a)$ is measurable.
For every fixed $x$, $a\mapsto q(x,a)$ is continuous.

These assumptions concern the posterior objective itself. In a Bayesian model, they can be checked from the posterior kernel and the loss. For example, bounded measurable loss that is continuous in $a$ for each parameter gives a continuous posterior objective by dominated convergence, provided the posterior is a measurable probability kernel.

Theorem: Under the two conditions above, there exists a measurable rule $\delta:\mathcal X\to[c,d]$ such that $$ q(x,\delta(x))=\min_{a\in[c,d]}q(x,a) $$ for every $x$. In fact, the rule can be chosen as the smallest minimizing action.

Proof. Continuity and compactness imply that the minimum $m(x)=\min_{a\in[c,d]}q(x,a)$ exists. Let $D$ be a countable dense subset of $[c,d]$ containing its endpoints. Continuity gives $$ m(x)=\inf_{a\in D}q(x,a). $$ The right-hand side is a countable infimum of measurable functions of $x$, so $m$ is measurable.

The set of minimizers is nonempty and closed in the compact interval, hence compact. It therefore has a smallest element; denote it by $\delta(x)$. To prove measurability, fix $r\in[c,d]$. Continuity also implies $$ m_r(x):=\min_{a\in[c,r]}q(x,a) =\inf_{a\in D_r}q(x,a), $$ where $D_r$ is any countable dense subset of $[c,r]$ containing its endpoints. Thus $m_r$ is measurable. The smallest minimizer is at most $r$ exactly when the minimum on $[c,r]$ equals the minimum on $[c,d]$: $$ \{x:\delta(x)\le r\}=\{x:m_r(x)=m(x)\}. $$ This is measurable because $m_r$ and $m$ are measurable. These sublevel sets, as $r$ varies, establish that $\delta$ is measurable. By its definition, it minimizes $q(x,\cdot)$ at every $x$. $\square$

This proof makes the role of tie-breaking explicit. A minimizer need not be unique, but choosing the smallest one gives a definite rule, and the sublevel-set argument verifies that this particular choice is measurable. Compactness and continuity are sufficient conditions here, not the only possible ones. More general measurable-selection results allow other action spaces and objectives, but require their own hypotheses.

Worked Posterior Minimizations

Worked Example: Squared-Error Action With a Binary Parameter

Let $\theta\in\{0,1\}$, use squared-error loss $L(\theta,a)=(a-\theta)^2$, and allow actions $a\in[0,1]$. Suppose the posterior probability of $\theta=1$, after observing $x$, is $p(x)$. The posterior expected loss is $$ q(x,a)=p(x)(a-1)^2+\{1-p(x)\}a^2. $$ Expanding and completing the square gives $$ q(x,a)=a^2-2p(x)a+p(x) =\{a-p(x)\}^2+p(x)\{1-p(x)\}. $$ Since $0\le p(x)\le1$, the unique minimizer over $[0,1]$ is $a=p(x)$.

For a numerical model, suppose the prior probabilities of $\theta=0$ and $\theta=1$ are both $1/2$, and $X$ is binary with $P(X=1\mid\theta=1)=0.8$ and $P(X=1\mid\theta=0)=0.2$. Bayes' formula gives $$ P(\theta=1\mid X=1)=\frac{0.8(0.5)}{0.8(0.5)+0.2(0.5)}=0.8, \qquad P(\theta=1\mid X=0)=\frac{0.2(0.5)}{0.2(0.5)+0.8(0.5)}=0.2. $$ Thus the rule chooses $0.8$ after $X=1$ and $0.2$ after $X=0$. The observation space is finite, so this rule is measurable. In a general observation space, the theorem above supplies measurability when $p(x)$ is measurable.

Worked Example: A Treatment Threshold From Asymmetric Costs

Let $\theta=1$ mean that a patient has a condition and $\theta=0$ mean that the patient does not. The available actions are $a=1$ (treat) and $a=0$ (do not treat). Suppose treating a patient without the condition costs 4, while failing to treat a patient with the condition costs 1; correct decisions have loss 0. Write $p(x)=P(\theta=1\mid x)$. The two posterior expected losses are $$ q(x,1)=4\{1-p(x)\}, \qquad q(x,0)=p(x). $$ Treat exactly when $4\{1-p(x)\}<p(x)$, which is equivalent to $p(x)>4/5$. At $p(x)=4/5$, the two actions tie.

For example, if $p(x)=1/(1+e^{-x})$, then $p(x)>4/5$ is equivalent to $e^{-x}<1/4$, and hence to $x>\log 4$. Choosing not to treat at a tie gives the rule $\delta(x)=1$ when $x>\log 4$ and $\delta(x)=0$ otherwise. This is measurable because the set $(\log 4,\infty)$ is Borel. The example also shows why minimizing posterior loss need not mean choosing the most probable state: the unequal costs change the threshold.

Worked Example: A Measurable Tie-Breaking Rule

Let the actions again be $[0,1]$, and suppose the posterior expected loss is $q(x,a)=|a-p(x)|$, where $p:\mathcal X\to[0,1]$ is measurable. For each fixed $x$, the unique minimizer is $a=p(x)$, with minimum 0, so the minimizing rule is measurable.

Now consider instead $q(x,a)=0$ for every $x$ and every $a\in[0,1]$. Every action minimizes at every observation. There is no unique action dictated by optimization, but the smallest-minimizer convention selects $\delta(x)=0$ for all $x$, which is measurable. This constant rule is a valid choice; the other pointwise minimizers are not needed to establish existence. Measurable tie-breaking is useful even when the decision criterion leaves a whole set of actions equally good.

From Posterior Minimization to Bayes Risk

A measurable pointwise minimizer gives an integrated risk comparison, not merely a collection of separate optimizations. Let $M$ denote the marginal distribution of the observation under the prior and sampling model. Assume the losses are nonnegative and measurable, so the relevant integrals are defined. For any measurable rule $\gamma$, posterior conditioning gives its prior-averaged loss as $$ \int q(x,\gamma(x))\,M(dx). $$ If $\delta$ minimizes $q(x,\cdot)$ at every $x$, then $q(x,\delta(x))\le q(x,\gamma(x))$ pointwise. Integrating this inequality yields $$ \int q(x,\delta(x))\,M(dx) \le \int q(x,\gamma(x))\,M(dx). $$ Thus $\delta$ is Bayes: it minimizes prior-averaged risk among the measurable rules under consideration. If losses are bounded, the integrals are finite; with nonnegative losses, the comparison remains valid even if an integral is infinite.

This is the link to complete-class reasoning. A pointwise posterior comparison can support a Bayes-risk comparison only after there is an admissible measurable rule realizing the minimizing actions. In arguments that replace a rule by a Bayes rule, measurable selection is what ensures the replacement belongs to the class of decision rules and has a well-defined risk.

When Measurability Fails

Pointwise existence alone does not guarantee a measurable choice. One way to see the issue is through a standard type of measurable-minimization counterexample: there are Borel subsets $C$ of products of Polish spaces whose sections $C_x=\{a:(x,a)\in C\}$ are all nonempty, but for which no Borel function $s$ satisfies $s(x)\in C_x$ for every $x$. Define $q(x,a)=0$ on $C$ and $q(x,a)=1$ outside $C$. This objective is Borel measurable and has minimum 0 at every $x$, yet a Borel minimizing rule would be exactly a Borel selector for $C$, which does not exist. The example relies on not imposing the compactness and continuity conditions of the theorem; it is not a claim that every such objective arises from a particular Bayesian experiment.

The complete-class consequence is direct. If one can identify a minimizing action separately for every observation but cannot select those actions measurably, the pointwise choices do not define a deterministic decision rule. Therefore they cannot, without further argument, be used to claim that a Bayes rule exists or to replace other rules by a Bayes rule in a risk comparison. A pointwise infimum may still be informative, but it is not by itself a measurable construction.

Common Pitfall

Wrong reasoning: “A minimum exists for every observed $x$, so choosing one at each $x$ gives a Bayes rule.” This skips the requirement that the choices form a measurable function.
Correction: Establish conditions that yield a measurable selector, such as a measurable objective, a compact interval of actions, and continuity in the action. Then verify that the selector is measurable and that its posterior-risk comparison can be integrated.
Another error: Treating a pointwise tie as a problem that must be resolved by an arbitrary choice. A fixed measurable tie-breaking convention, such as choosing the smallest minimizer under the interval conditions above, resolves ties without changing the minimum posterior loss.

Key takeaway: A posterior minimizer is useful in a decision-theoretic argument only when the minimizing actions can be chosen measurably. Under compact-interval and continuity conditions, the smallest minimizer is a measurable rule, and its pointwise posterior-risk advantage integrates to a Bayes-risk advantage.

Check Your Understanding

Use the definitions and conditions above to answer these questions.

Why does existence of a minimizing action for each fixed observation not, by itself, establish existence of a decision rule?
In the squared-error example, derive the posterior-minimizing action from the expanded expression for $q(x,a)$.
With false-positive cost 4 and false-negative cost 1, what action minimizes posterior expected loss when $p(x)=0.7$? Show the two losses.
In the interval-action theorem, why is $m(x)$ measurable when it is expressed as an infimum over a countable dense set?
What does the smallest-minimizer convention accomplish when the posterior objective has multiple minimizers?