Bayesian Decision Rules For Enrichment

Why Use a Bayesian Rule for Enrichment?

At an interim analysis, a trial team may need to decide whether to keep enrolling a biomarker-defined population, restrict enrollment to another population, or stop a population’s enrollment. A Bayesian decision rule expresses this choice in terms of the evidence about the population-specific treatment effect after combining interim data with a prior distribution. It provides a direct probability statement for a prespecified decision, but its meaning depends on the model, prior, and decision threshold.

Let \(\theta_j\) denote the treatment effect for population \(j\), with positive values defined as favoring treatment. The effect must use the population-specific estimand and scale specified for the trial. For example, if the outcome is a change score where a larger value is better, positive effects may represent improvement; for an outcome where smaller values are better, the effect should be coded or interpreted accordingly.

The threshold \(q\) is not the significance level \(\alpha\). It is an operational decision criterion: a larger \(q\) requires stronger posterior evidence of benefit before retaining a population. Its choice should reflect the consequences of continuing enrollment in a population that is not beneficial, and of excluding one that is beneficial. The interim rule also needs to be specified before examining the data on which it will act.

A Normal Model for an Interim Effect Estimate

A useful working model assumes that the interim estimate \(\hat{\theta}_j\) has an approximately normal sampling distribution with known standard error \(SE_j\), conditional on the true effect:

Here \(m_{0j}\) is the prior mean and \(\tau_j\) is the prior standard deviation. The likelihood approximation is most appropriate when the estimator is reasonably well approximated by a normal distribution and its standard error is reliable. In practice, uncertainty in nuisance parameters, small samples, or complex endpoints may require a more suitable likelihood rather than treating \(SE_j\) as fixed.

For the calculation below, assume the prior is proper, the estimate and standard error correspond to the prespecified population-specific estimand, and the interim data are used once in the likelihood. If multiple population estimates share participants or controls, their estimates may be correlated; separate calculations that assume independence would then be inappropriate. A joint model should represent that dependence.

Posterior Calculation and the Decision Boundary

Under the normal likelihood and prior, the posterior is also normal. Its variance is determined by adding prior and likelihood precisions, and its mean is the precision-weighted average of the prior mean and the interim estimate:

The posterior probability of a beneficial effect is therefore \(P(\theta_j>0\mid D)=\Phi(m_j/\sqrt{v_j})\), where \(\Phi\) is the standard normal cumulative distribution function. This follows by standardizing the posterior: if \(\theta_j\mid D\sim N(m_j,v_j)\), then \((\theta_j-m_j)/\sqrt{v_j}\) is standard normal, so \(P(\theta_j>0\mid D)=P((\theta_j-m_j)/\sqrt{v_j}>-m_j/\sqrt{v_j})=\Phi(m_j/\sqrt{v_j})\).

The same rule can be written as a cutoff for the interim estimate. Define \(w=\tau_j^2/(\tau_j^2+SE_j^2)\). Then \(m_j=w\hat{\theta}_j+(1-w)m_{0j}\), and \(\sqrt{v_j}=\sqrt{\tau_j^2SE_j^2/(\tau_j^2+SE_j^2)}\). Since \(w>0\), the rule is equivalent to:

This gives a concrete way to see how changing the prior or \(q\) changes the data threshold for retention. It does not mean that an interim estimate alone determines the decision: the cutoff also depends on the prior and standard error.

Worked Decisions

Choosing and Checking the Rule

A probability cutoff should reflect the decision, not simply be selected because it is a familiar number. One way to make the trade-off explicit is to assign utilities to continuing or stopping. Suppose continuing has utility \(U_B\) if the effect is beneficial and \(U_N\) if it is not, while stopping has utility \(U_S\). If \(U_B>U_N\), continuing has greater expected utility than stopping when \(pU_B+(1-p)U_N>U_S\), where \(p=P(\theta>0\mid D)\). When \(U_N<U_S<U_B\), this is equivalent to \(p>(U_S-U_N)/(U_B-U_N)\). This simple calculation illustrates how costs and benefits can motivate a threshold; real trial decisions may involve more than two possible states and actions.

Prior choice deserves direct scrutiny because it can alter the posterior probability and the selected population, as the second example shows. Examine whether the prior is scientifically defensible for the specific estimand, whether its spread is appropriate, and how the decision changes under plausible alternative priors. A prior centered at zero is not automatically neutral: its concentration determines how strongly it pulls estimates toward zero. Borrowing information from other populations or earlier evidence also requires justification that the information is relevant to the population and outcome at issue.

A posterior threshold does not, by itself, guarantee a chosen frequentist false-positive rate or other long-run operating characteristic. The rule should be evaluated over plausible trial scenarios, including no-effect settings, beneficial effects, differing subgroup effects, and realistic accrual and variance patterns. Check for unstable estimates, unexpected standard errors, missing biomarker data, and discrepancies between the assumed likelihood and observed interim data. If a model or data-quality problem is material, the numerical posterior probability may not be a sound basis for an adaptation.

Common Pitfalls and Alternatives

• Equating \(q\) with \(\alpha\). A posterior cutoff is not a frequentist significance level. It is a decision rule conditional on the model and prior; long-run error behavior must be assessed separately.
• Ignoring prior sensitivity. A posterior probability can look precise while depending strongly on a prior. Report or examine the decision under plausible prior alternatives.
• Using separate subgroup calculations when estimates are dependent. Shared controls or nested populations can induce covariance. Use a joint likelihood or model that represents it.
• Updating twice with the same interim data. When incorporating later data, use a coherent likelihood that includes each observation once; do not treat interim results as an independent new data set.
• Interpreting selection as proof of a subgroup difference. Retaining one population and excluding another under separate rules does not establish that their treatment effects differ. The decision is an adaptation, not a direct comparison of effects.

A conditional-power rule, as used in frequentist design calculations, evaluates the chance of meeting a specified future statistical criterion under assumptions about the effect and remaining data. A posterior-probability rule instead evaluates the current posterior probability of a state such as \(\theta_j>0\). They answer different questions, and neither removes the need to prespecify the adaptation and assess the trial’s operating characteristics.