Mixtures of Priors for Integrated-Risk Comparisons

Why Mixture Weights Matter

An integrated-risk comparison averages a decision rule’s risk over a prior distribution on the parameter. When that prior is a mixture, the average has a useful componentwise structure: each component contributes its own integrated risk, multiplied by its mixture weight. This lets us turn local comparisons into a quantified statement about the mixture as a whole.

The qualification is important. A rule can improve risk over one part of the parameter space and worsen it elsewhere. A mixture comparison depends on both the componentwise risk differences and the weights assigned to their components. A positive improvement under one component does not, by itself, imply a positive improvement under the mixture.

Integrated Risk Under a Prior Mixture

Consider a statistical decision problem with parameter space $\Theta$, a decision rule $\delta$, and risk function $R(\theta,\delta)$. For a probability measure $\pi$ on $\Theta$, the integrated risk is the average risk under $\pi$.

Definition: The integrated risk of $\delta$ under a prior probability measure $\pi$ is $$ r(\pi,\delta)=\int_{\Theta} R(\theta,\delta)\,\pi(d\theta), $$ when the integral is defined. A finite mixture of prior probability measures $\pi_1,\ldots,\pi_m$ has the form $$ \pi=\sum_{j=1}^{m}w_j\pi_j,\qquad w_j\geq 0,\qquad \sum_{j=1}^{m}w_j=1. $$ The numbers $w_j$ are the mixture weights.

For each component, write $r_j(\delta)=r(\pi_j,\delta)$. For two decision rules $\delta_0$ and $\delta_1$, define the componentwise integrated-risk gain of $\delta_1$ over $\delta_0$ by $G_j=r_j(\delta_0)-r_j(\delta_1)$. Thus $G_j>0$ means that $\delta_1$ has lower integrated risk under $\pi_j$; $G_j<0$ means it has higher integrated risk there.

For the comparisons below, assume the risks are measurable and the integrated risks of both rules are finite under every component with positive weight. These conditions ensure the risk differences are finite and subtraction is legitimate. Nonnegative risks can have infinite integrals, but in that case a difference such as $\infty-\infty$ is not defined; the finite-integral assumption avoids that ambiguity.

Worked Example: A Two-Point Parameter Space

Let $\Theta=\{0,1\}$, with an observation that carries no information about $\theta$. There are two possible actions, 0 and 1, and the loss is 0 for a correct action and 1 for an incorrect action. Rule $\delta_0$ always chooses action 0, and rule $\delta_1$ always chooses action 1. Their risks are $R(0,\delta_0)=0$, $R(1,\delta_0)=1$, $R(0,\delta_1)=1$, and $R(1,\delta_1)=0$.

Take $\pi_1$ to put all its mass on 0 and $\pi_2$ to put all its mass on 1. Then $r_1(\delta_0)=0$, $r_1(\delta_1)=1$, $r_2(\delta_0)=1$, and $r_2(\delta_1)=0$. The component gains are $G_1=-1$ and $G_2=1$.

With mixture weights $w_1=0.7$ and $w_2=0.3$, the prior assigns probabilities $0.7$ and $0.3$ to 0 and 1. Its integrated risks are $r(\pi,\delta_0)=0.7(0)+0.3(1)=0.3$ and $r(\pi,\delta_1)=0.7(1)+0.3(0)=0.7$. Therefore the mixture gain is $0.3-0.7=-0.4$. The weighted component gains give the same answer: $0.7(-1)+0.3(1)=-0.4$.

Although $\delta_1$ improves integrated risk under $\pi_2$, that improvement is outweighed by its loss under the more heavily weighted component $\pi_1$.

The Mixture Decomposition

Theorem: Let $\pi=\sum_{j=1}^{m}w_j\pi_j$ be a finite mixture of probability measures, and let $\delta$ have measurable, nonnegative risk with finite integrated risk under every component with positive weight. Then $$ r(\pi,\delta)=\sum_{j=1}^{m}w_jr(\pi_j,\delta). $$ If $\delta_0$ and $\delta_1$ both satisfy these conditions, their mixture gain is $$ r(\pi,\delta_0)-r(\pi,\delta_1) =\sum_{j=1}^{m}w_j\bigl[r(\pi_j,\delta_0)-r(\pi_j,\delta_1)\bigr] =\sum_{j=1}^{m}w_jG_j. $$

Proof. Integration is linear for a finite sum of finite measures when the integrals are finite. Using the definition of the mixture, $$ r(\pi,\delta)=\int_{\Theta}R(\theta,\delta)\left(\sum_{j=1}^{m}w_j\pi_j\right)(d\theta) =\sum_{j=1}^{m}w_j\int_{\Theta}R(\theta,\delta)\,\pi_j(d\theta) =\sum_{j=1}^{m}w_jr(\pi_j,\delta). $$ Apply this identity to each rule and subtract the two finite sums: $$ r(\pi,\delta_0)-r(\pi,\delta_1) =\sum_{j=1}^{m}w_jr(\pi_j,\delta_0)-\sum_{j=1}^{m}w_jr(\pi_j,\delta_1) =\sum_{j=1}^{m}w_jG_j. $$ This proves both identities. $\square$

The formula is exact, not an approximation. It says that the overall gain is the weighted average of the component gains. In particular, its sign depends on the complete weighted sum, not merely on whether any one component has a positive gain.

Combining Componentwise Bounds

Theorem: Suppose the component gains satisfy $G_j\geq b_j$ for $j=1,\ldots,m$, where each $b_j$ is a finite real number. Then $$ r(\pi,\delta_0)-r(\pi,\delta_1)=\sum_{j=1}^{m}w_jG_j \geq \sum_{j=1}^{m}w_jb_j. $$ In particular, if one component $k$ satisfies $G_k\geq c>0$, while each other component satisfies $G_j\geq-\varepsilon_j$ for some $\varepsilon_j\geq0$, then $$ r(\pi,\delta_0)-r(\pi,\delta_1) \geq w_kc-\sum_{j\ne k}w_j\varepsilon_j. $$

Proof. Since every weight is nonnegative, multiplying $G_j\geq b_j$ by $w_j$ preserves the inequality, so $w_jG_j\geq w_jb_j$. Adding these inequalities over the finite set of components and applying the mixture decomposition gives the first bound. For the second bound, use $b_k=c$ and $b_j=-\varepsilon_j$ for $j\ne k$. $\square$

A positive lower bound on the mixture gain follows whenever the displayed lower bound is strictly positive. This gives a practical way to choose weights: a component with a guaranteed gain must receive enough weight to offset the possible losses allowed by the other component bounds.

Worked Example: Finding the Weight Threshold

Suppose there are two components. Under $\pi_1$, a proposed rule has gain at least $0.2$; under $\pi_2$, its gain is at least $-0.05$. Let $w$ be the weight on $\pi_1$, so the weight on $\pi_2$ is $1-w$. The mixture gain is at least $$ 0.2w-0.05(1-w)=0.2w-0.05+0.05w=0.25w-0.05. $$ For this lower bound to be positive, we need $0.25w-0.05>0$, or $w>0.2$.

For example, with $w=0.4$, the guaranteed gain is $0.25(0.4)-0.05=0.10-0.05=0.05$, which is positive. With $w=0.2$, the bound is exactly zero; the stated component bounds then do not guarantee a strictly positive mixture gain. At $w=0.1$, the bound is $-0.025$, so these bounds alone cannot establish an improvement.

The threshold is a statement about what follows from the bounds, not necessarily about the exact gain. The actual component gains could be larger than their lower bounds.

Worked Example: A Continuous Parameter and Two Prior Components

Let $\Theta=[0,1]$, and consider a decision problem with no informative observation and actions $a_0$ and $a_1$. Define the losses, hence the risks, by $R(\theta,a_0)=0.5$ for every $\theta$, and $$ R(\theta,a_1)= \begin{cases} 0.3,&0\leq\theta\leq 0.25,\\ 0.6,&0.25<\theta\leq1. \end{cases} $$ Both risks are nonnegative and bounded. Let $\pi_1$ be uniform on $[0,0.25]$, and let $\pi_2$ be uniform on $(0.25,1]$.

Under $\pi_1$, the risks are constant over the support: $r_1(a_0)=0.5$ and $r_1(a_1)=0.3$, so $G_1=0.2$. Under $\pi_2$, $r_2(a_0)=0.5$ and $r_2(a_1)=0.6$, so $G_2=-0.1$.

Set $\pi=0.4\pi_1+0.6\pi_2$. The mixture integrated risks are $r(\pi,a_0)=0.4(0.5)+0.6(0.5)=0.5$ and $r(\pi,a_1)=0.4(0.3)+0.6(0.6)=0.12+0.36=0.48$. Thus the mixture gain is $0.5-0.48=0.02$. Directly combining component gains gives the same result: $0.4(0.2)+0.6(-0.1)=0.08-0.06=0.02$.

Here the local improvement on the first interval outweighs the local loss on the second because its weighted contribution, $0.08$, exceeds the loss contribution, $0.06$.

What the Comparison Does and Does Not Establish

Mixture decompositions are useful in admissibility arguments because they turn a comparison under a deliberately chosen prior into a sum of interpretable contributions. If a proposed improvement is known to be small or negative under some components, the bounds show exactly how much weight other components must receive for the overall comparison to remain favorable.

A mixture integrated-risk comparison is still a statement about an average under that prior. It is not, by itself, a pointwise comparison of risk over all $\theta$, nor does it alone establish admissibility or inadmissibility. Those conclusions require the relevant risk comparisons and hypotheses of the argument in which the mixture is used. In particular, a component concentrated where a rule improves contributes nothing if its mixture weight is zero.

The same decomposition also explains why it is essential to track signs. A positive component gain can be canceled by a sufficiently large weighted negative gain elsewhere. Conversely, if every component gain is nonnegative and at least one positive-gain component has positive weight, the mixture gain is positive.

Common Pitfall

Wrong reasoning: “The new rule has lower integrated risk under one component prior, so it must have lower integrated risk under their mixture.”

Correction: The mixture gain is $\sum_j w_jG_j$, not any single $G_j$. A positive component gain establishes a positive mixture gain only when the remaining weighted contributions cannot offset it. Check the other component gains or use valid lower bounds for them, then calculate the weighted sum.

Key takeaway: Under a finite prior mixture, integrated risk and integrated-risk differences decompose exactly into weighted component contributions. To draw a conclusion about the mixture, combine the component gains with their actual weights and account for every possible negative contribution.

Check Your Understanding

Use the component decomposition and keep track of the sign convention $G_j=r_j(\delta_0)-r_j(\delta_1)$.

For a two-component mixture with weights $0.25$ and $0.75$, the component gains are $0.4$ and $-0.1$. What is the mixture gain?
If $G_j\geq b_j$ for each component, why does multiplying by a mixture weight preserve the inequality?
A component prior gives a positive gain, but its mixture weight is zero. Does it affect the mixture comparison? Explain.
Suppose one component gain is at least $0.3$, another is at least $-0.1$, and the first component has weight $w$. What condition on $w$ makes the resulting lower bound strictly positive?
Why must finite integrated risks be assumed before subtracting the two mixture integrated risks in the stated theorem?