Why the Prior Can Change an Arm Decision
In a multi-arm trial, an observed treatment contrast may be compatible with both a small effect and a clinically useful effect, especially when its standard error is substantial. A skeptical prior expresses a belief that treatment effects are likely to be near zero. A robust mixture prior retains that skeptical component but also assigns some prior probability to a broader range of effects. The broader component lets sufficiently informative data move the posterior away from zero more readily than a narrow prior alone.
The issue is not merely whether a posterior mean changes. An arm decision may depend on the posterior probability that its effect exceeds a prespecified clinical threshold. A narrow prior can reduce that probability, while a mixture prior can partly protect against over-shrinkage when the observed effect is larger than the skeptical component anticipates. The result depends on the observed contrast, its uncertainty, the component scales and the mixture weights.
Normal Likelihood and Component Updating
For the calculations here, suppose an arm’s estimated contrast is \(y_j\), and the sampling model is \(y_j\mid\theta_j\sim N(\theta_j,s_j^2)\), with known standard error \(s_j\). This normal summary-data model is an assumption for the examples, not a universal model for treatment outcomes. We also assume the prior scales and mixture weight were specified before observing the arm’s contrast.
Under one normal prior component, \(\theta_j\sim N(0,\tau^2)\), conjugacy gives a normal posterior. Its variance is the reciprocal of the sum of prior and likelihood precisions, and its mean is that variance multiplied by the likelihood precision times the observed contrast:
For a mixture prior, the posterior is a mixture of these component-specific normal posteriors. Bayes’ rule also changes the component weights. If the prior probability of the skeptical component is \(w\), its posterior probability is proportional to \(w\) times the marginal density of the data under that component. For the normal model, that marginal density is \(N(y_j;0,s_j^2+\tau^2)\).
Here \(\phi(y;0,V)\) is the normal density at \(y\) with mean zero and variance \(V\). The posterior probability of the broad component is one minus the displayed probability. Because the component evidence depends on \(y_j\), different arms generally have different posterior mixture weights, even when they share the same prior.
Worked Example: Updating the Broad-Component Weight
Suppose a hypothetical arm has estimated effect \(y=0.8\) benefit units with standard error \(s=1\) benefit unit. Use a robust prior with weight \(w=0.8\) on \(N(0,0.25)\) and weight \(0.2\) on \(N(0,4)\). These variances correspond to prior standard deviations of \(0.5\) and \(2\) benefit units. Find the posterior probability of the broad component.
The sampling variance is \(s^2=1\). The marginal data variances under the narrow and broad components are \(1+0.25=1.25\) and \(1+4=5\). The ratio of their marginal densities is therefore \(\sqrt{5/1.25}\exp\{-0.8^2(1/1.25-1/5)/2\}=2\exp(-0.192)\).
The narrow-component evidence multiplied by its prior weight is \(0.8\times2\exp(-0.192)=1.3205\) (approximately). The broad-component evidence multiplied by its prior weight is \(0.2\), using the broad density as the reference. Thus the posterior broad-component probability is \(0.2/(1.3205+0.2)=0.1315\), or 13.15%.
Although the prior broad-component probability was 20%, this moderate estimate gives relatively more support to the narrow component after observing the data. The posterior weight is not the probability that the treatment effect is large; it is the probability assigned to one prior component, conditional on the model and the observed contrast.
Posterior Probability of Clinically Meaningful Benefit
Let \(\delta\) be the minimum effect considered clinically meaningful. Under a single normal posterior \(N(m,v)\), the probability of benefit beyond that threshold is \(P(\theta_j>\delta\mid y_j)=1-\Phi((\delta-m)/\sqrt{v})\), where \(\Phi\) is the standard normal cumulative distribution function. Under a mixture, calculate this probability within each component and average using the posterior component weights.
This is a posterior probability conditional on the stated likelihood and prior. It is not a frequentist error rate, and it does not by itself specify whether an arm should be selected: the decision also depends on the decision threshold and the clinical context.
Worked Example: One Arm Under Skeptical and Robust Priors
Continue with the hypothetical estimate \(y=0.8\), standard error \(s=1\), and priors from the previous example. Suppose clinically meaningful benefit is \(\delta=0.5\) benefit units. First calculate the posterior under the skeptical prior alone, \(N(0,0.25)\).
The posterior variance is \(v=1/(1/1+1/0.25)=0.2\), and the posterior mean is \(m=0.2(0.8)=0.16\). Thus \(P(\theta>0.5\mid y)=1-\Phi((0.5-0.16)/\sqrt{0.2})=1-\Phi(0.760)=0.224\), approximately.
For the robust prior, the narrow component has the same posterior probability of exceeding \(0.5\), about \(0.224\). Under the broad component, \(v=1/(1+1/4)=0.8\) and \(m=0.8(0.8)=0.64\), giving \(1-\Phi((0.5-0.64)/\sqrt{0.8})=1-\Phi(-0.157)=0.562\), approximately.
Using the broad-component posterior weight of \(0.1315\), the robust-mixture probability is \((1-0.1315)(0.224)+(0.1315)(0.562)=0.268\), approximately. In this example, the mixture raises the posterior probability of clinically meaningful benefit from about 22.4% to 26.8%. It does not make the evidence decisive: the broad component has only 13.15% posterior weight.
Worked Example: Comparing Three Hypothetical Arms
Suppose three hypothetical arms have estimated contrasts \(y_A=0.8\), \(y_B=1.5\), and \(y_C=0.2\) benefit units, each with standard error \(1\) benefit unit. Use \(\delta=0.5\), the skeptical prior \(N(0,0.25)\), and the robust prior \(0.8N(0,0.25)+0.2N(0,4)\). The common standard error and prior scales make the component posterior variances \(0.2\) and \(0.8\) for every arm. For each arm, the posterior means are \(0.2y_j\) and \(0.8y_j\).
For arm A, the skeptical posterior probability is \(1-\Phi((0.5-0.16)/\sqrt{0.2})=0.224\). Its broad-component posterior weight is \(0.2/[0.2+0.8(2\exp(-0.3(0.8)^2))]=0.1315\). Combining the narrow and broad probabilities \(0.224\) and \(0.562\) gives a robust probability of \(0.268\).
For arm B, the skeptical posterior mean is \(0.30\), so the skeptical probability is \(1-\Phi((0.5-0.30)/\sqrt{0.2})=0.327\). Its broad weight is \(0.2/[0.2+1.6\exp(-0.3(1.5)^2)]=0.1971\). The broad-component probability is \(1-\Phi((0.5-1.20)/\sqrt{0.8})=0.783\); the mixture probability is therefore \(0.803(0.327)+0.197(0.783)=0.417\), approximately.
For arm C, the skeptical mean is \(0.04\), giving \(1-\Phi((0.5-0.04)/\sqrt{0.2})=0.152\). Its broad weight is \(0.2/[0.2+1.6\exp(-0.3(0.2)^2)]=0.1123\). The broad-component probability is \(1-\Phi((0.5-0.16)/\sqrt{0.8})=0.352\), so the mixture probability is \(0.888(0.152)+0.112(0.352)=0.174\), approximately.
| Arm | Observed contrast (benefit units) | Skeptical-prior probability | Robust-mixture probability | Posterior broad weight |
|---|---|---|---|---|
| A | 0.8 | 0.224 | 0.268 | 0.1315 |
| B | 1.5 | 0.327 | 0.417 | 0.1971 |
| C | 0.2 | 0.152 | 0.174 | 0.1123 |
The mixture changes the probabilities by different amounts because the data update each arm’s component weight separately. Arm B has the largest observed contrast and the largest broad-component posterior weight. These calculations compare marginal probabilities; they do not give the probability that a particular arm is best among all three.
Why the Mixture Formula Works
The component-specific normal posterior follows by multiplying the normal likelihood and normal prior. Ignoring constants that do not involve \(\theta_j\), the log of their product is \(-\frac{1}{2}\{(y_j-\theta_j)^2/s_j^2+\theta_j^2/\tau^2\}\). Collecting the terms involving \(\theta_j^2\) and \(\theta_j\) gives precision \(1/s_j^2+1/\tau^2\) and precision-weighted mean \(y_j/s_j^2\). Completing the square yields the stated \(v_j\) and \(m_j\).
To update the component weights, integrate the likelihood over the prior component: \(\int N(y_j;\theta_j,s_j^2)N(\theta_j;0,\tau^2)\,d\theta_j=N(y_j;0,s_j^2+\tau^2)\). This follows because the sampling error and prior-distributed effect add as independent normal quantities in the prior predictive distribution. Bayes’ rule then multiplies each prior component probability by its own marginal density for the observed estimate and normalizes the two resulting quantities to sum to one.
Interpretation, Diagnostics, and Alternatives
A robust mixture is not simply a diffuse prior. It retains substantial prior mass near zero while allowing a broader component to matter if the data support it. Nor does “robust” mean that results are insensitive to all modeling choices. The component scales, initial weights, effect scale, likelihood and success threshold all affect the posterior calculation.
A useful diagnostic is to check whether the prior scales express plausible treatment effects in the units being used, and whether the observed contrast is unusual under both components. Also check the posterior component weights: a weight close to zero or one means the data have largely favored one component under the specified model. Neither extreme proves that the model is correct. If calculations across arms assume independent estimates, verify that assumption before using them. With a shared control, treatment contrasts are generally correlated; a joint likelihood and its covariance structure may be needed, especially for probabilities involving more than one arm.
The main alternatives answer related but different modeling needs. A single skeptical prior is simpler and consistently shrinks effects toward zero, but may be influential when the data are informative and conflict with its narrow scale. A single broad prior can allow large effects but does not preserve a separate skeptical component. Hierarchical shrinkage instead links arm effects through shared population-level parameters; a robust two-component prior is a specified mixture, not the same model. Choosing among them requires a defensible account of the treatment effects and trial context, not just a preference for larger or smaller posterior probabilities.
- Common mistake: Keep the prior mixture weights fixed when averaging component-specific success probabilities. Correction: First update those weights using each arm’s marginal data density, then average the component success probabilities using the posterior weights.
- Common mistake: Interpret a 13.15% posterior broad-component weight as a 13.15% probability of clinically meaningful benefit. Correction: The component weight describes which prior component is supported; the probability of benefit is calculated from the full posterior distribution.
- Common mistake: Calculate each arm independently and treat the resulting probabilities as if they describe joint or ranking events. Correction: Marginal probabilities do not determine joint probabilities when arm estimates are correlated, as they can be with a shared control.
Worked Example: When Stronger Data Shift the Mixture
For one more hypothetical arm, let \(y=2.4\) benefit units, \(s=1\), and \(\delta=0.5\), retaining the same robust prior. The posterior broad weight is \(0.2/[0.2+1.6\exp(-0.3(2.4)^2)]=0.413\), approximately.
The narrow posterior has mean \(0.2(2.4)=0.48\) and variance \(0.2\), so its success probability is \(1-\Phi((0.5-0.48)/\sqrt{0.2})=0.482\), approximately. The broad posterior has mean \(0.8(2.4)=1.92\) and variance \(0.8\), giving \(1-\Phi((0.5-1.92)/\sqrt{0.8})=0.944\), approximately. The mixture probability is \(0.587(0.482)+0.413(0.944)=0.673\), approximately.
Here the data increase the broad component’s posterior weight from 20% to about 41.3%, and the mixture’s probability of meaningful benefit is about 67.3%. The result illustrates how a robust mixture can respond to a stronger estimate while retaining some skeptical-component influence.
Check Your Understanding
Try each question, then check the solution immediately below it.
- With \(s=1\), prior \(N(0,0.25)\), and observed estimate \(y=1\), what is the posterior mean and variance under the skeptical prior?
- In a robust mixture, why can two arms with the same prior mixture weights have different posterior component weights?
- For an arm with posterior component probabilities \(0.7\) and \(0.4\) of exceeding the clinical threshold, and posterior broad-component weight \(0.25\), calculate the mixture success probability.
- Why can marginal arm success probabilities not generally be multiplied to obtain the probability that both arms succeed in a shared-control trial?
Solutions. 1. The posterior variance is \(1/(1+4)=0.2\), and the mean is \(0.2(1)=0.2\). 2. Each arm has a different observed estimate, and its component-specific marginal data densities therefore update the prior mixture weights differently. 3. Weight the narrow probability \(0.7\) by \(1-0.25=0.75\), and the broad probability \(0.4\) by \(0.25\): \(0.75(0.7)+0.25(0.4)=0.625\). 4. Shared-control contrasts are correlated, so the joint probability depends on their joint posterior distribution; the product of marginal probabilities is valid only under posterior independence.