Deriving the Robust Mixture Posterior

From a Mixture Prior to a Mixture Posterior

A robust mixture prior represents uncertainty about how well different sources of information apply to a treatment effect. One component might encode a borrowing model and another a protective model. After observing trial data, the posterior must reflect both the parameter values supported within each component and how well each component predicted the observed data. The result is a mixture of component-specific posteriors, with updated component weights.

This tutorial derives that result for a general likelihood. It does not require a normal model, a binary outcome, or a particular form of treatment effect. The key is to keep two operations distinct: updating the parameter within a component, and updating the probability assigned to that component.

The component label can be viewed as a latent indicator: before seeing the data, component \(k\) has probability \(\omega_k\), and conditional on that component, \(\theta\) has distribution \(\Pi_k\). This representation is a way to express a prior distribution; it does not assert that the component label is an observed feature of the trial.

For a continuous observation, the marginal likelihood below is the density of the observation under a component, evaluated at the observed \(y\). For discrete data, it is the probability of the observed outcome. In either case, the likelihood must be defined under the same sampling model for all components. The component priors are proper, and the component marginal likelihoods used to form component posteriors are finite and strictly positive. The overall mixture marginal likelihood must also be positive. A component with zero marginal likelihood is treated separately below.

Deriving the Posterior Distribution

Define the component marginal likelihood by integrating the likelihood over that component's prior. The mixture marginal likelihood follows by averaging those component marginal likelihoods using the prior component weights.

For a component with \(m_k(y)>0\), Bayes' rule gives its component-specific posterior distribution. Applying Bayes' rule to the mixture prior gives the full posterior distribution.

Here \(A\) is any set of treatment-effect values. Substituting the component-posterior expression into the full-posterior expression yields the mixture representation. This is the central result: the posterior distribution is a mixture of component-specific posteriors, with weights updated by the data.

The algebra also explains why the weights and the component posteriors are both necessary. The numerator for the probability of \(A\) in the full posterior is the sum of the component contributions \(\omega_k\int_A L(y\mid\theta)\Pi_k(d\theta)\). Each such contribution factors into \(\omega_km_k(y)\) times \(\Pi_k(A\mid y)\). Dividing by \(m(y)\) turns the first factor into \(\widetilde{\omega}_k(y)\), giving the stated mixture.

If \(m_k(y)=0\), then \(\omega_km_k(y)=0\), so that component contributes nothing to the full posterior, provided the overall marginal likelihood is positive. But the expression for \(\Pi_k(\theta\mid y)\) divides by \(m_k(y)\) and is not defined for that component. Omit it from the posterior-mixture formula; do not assign it a component posterior by dividing by zero. If \(\omega_k=0\), the component likewise contributes nothing, regardless of its marginal likelihood.

Zero-Contribution Components and Validity Conditions

Proper component priors do not, by themselves, guarantee that every component posterior can be calculated for every possible observation. A component may assign zero probability or density to the observed data. Its marginal likelihood is then zero, and its conditional posterior given those data is undefined. The full posterior can nevertheless be well-defined if another component assigns positive marginal likelihood to the observation.

In applications, check that the integrals defining the marginal likelihoods are finite. For each component used in a component-posterior formula, also check that its marginal likelihood is strictly positive. A positive overall marginal likelihood alone is not enough to establish positivity for every component. If all weighted components have zero marginal likelihood, the data receive zero marginal probability or density under the mixture, and Bayes' rule in this form cannot produce a posterior.

Posterior Summaries From a Mixture

For any event \(A\), the full posterior probability is the updated-weighted average of the component probabilities. The same averaging rule applies to posterior expectations when the expectations exist. In particular, if the component posterior means are \(\mu_k(y)=E(\theta\mid y,k)\), then \(E(\theta\mid y)=\sum_k\widetilde{\omega}_k(y)\mu_k(y)\). Other summaries require care: a mixture's variance includes both within-component uncertainty and variation between component means. A quantile must be obtained from the full mixture distribution, not by averaging component quantiles.

Comparisons, Diagnostics, and Limitations

The posterior mixture is not the same as selecting the component with the largest updated weight. Selection discards posterior uncertainty about which component applies; the mixture retains that uncertainty. Nor is it generally equivalent to averaging prior means and then updating one prior: the data update each component according to its own marginal likelihood and its own likelihood-weighted parameter distribution.

Useful checks are algebraic as well as substantive. Confirm that the prior component weights sum to 1, the updated weights are nonnegative and sum to 1, each included component posterior is normalized, and the resulting posterior probabilities of disjoint exhaustive events sum to 1. In numerical work, a component with a very small marginal likelihood can have a very small updated weight; rounding too early can make that component appear to have weight zero. Compute with adequate precision before rounding reported results.

The mixture construction does not guarantee robustness to every kind of model failure. It only averages over the components actually specified. If the likelihood is misspecified, if the treatment-effect parameter is not comparable across components, or if all components predict the observation poorly, the posterior mixture does not repair that problem automatically. Component choices and prior mixture weights remain assumptions that affect inference.