Predictive Success With Unequal Treatment Allocation

Why Unequal Allocation Changes Predictive Success

In a two-arm trial, the number of future participants need not be the same in the treatment and control arms. A planned 2:1 allocation, for example, gives different amounts of future information about each arm. Predictive success must therefore average over the possible outcomes using the treatment-arm and control-arm sample sizes separately. Replacing those two sizes with one pooled number can give the wrong predictive distribution and the wrong probability of meeting the final decision rule.

Predictive success is the probability, given the information currently available and the specified model, that the trial’s final data will satisfy a prespecified success criterion. It is not the probability that the treatment works, nor is it the probability that the current evidence is correct. The criterion might be a threshold on a final posterior probability, a posterior mean contrast, or another explicitly defined rule.

The allocation is part of the prediction. Write \(m_T\) and \(m_C\) for the numbers of future participants assigned to treatment and control. In general, \(m_T\ne m_C\), and the future outcomes in the two arms have different predictive distributions because their sample sizes differ. The analysis below assumes the future allocation counts are fixed. If enrollment, dropout, or the allocation mechanism makes the final counts random, their uncertainty must also be represented.

Binary Outcomes: Separate Beta-Binomial Predictions

Suppose the binary outcome is a response, with response probabilities \(p_T\) and \(p_C\). Assume independent beta priors for the two arm-specific response probabilities and independent participant outcomes within each arm conditional on those probabilities. After the observed data, write the posterior distributions as \(p_T\mid D\sim\operatorname{Beta}(a_T,b_T)\) and \(p_C\mid D\sim\operatorname{Beta}(a_C,b_C)\). Here each pair of parameters incorporates the prior and the observed arm-specific responses and nonresponses.

Let \(X_T\) and \(X_C\) be the future numbers of responses. Conditional on the respective response probabilities, \(X_T\sim\operatorname{Binomial}(m_T,p_T)\) and \(X_C\sim\operatorname{Binomial}(m_C,p_C)\). Integrating over each beta posterior gives separate beta-binomial predictive distributions:

Under independent arm posteriors and independent future outcomes, the joint predictive probability for a pair of counts is the product of the two arm-specific probabilities. To calculate predictive success, sum those joint probabilities over all pairs \((x_T,x_C)\) that make the final data satisfy the chosen rule. Unequal allocation changes both the range of possible counts and their predictive probabilities.

The formula follows directly by integrating the binomial likelihood over the beta posterior. The beta function identity \(B(u,v)=\int_0^1p^{u-1}(1-p)^{v-1}\,dp\) gives:

This integration matters: plugging a posterior mean into a binomial model ignores uncertainty about \(p_j\). For example, the predictive mean is \(E(X_j\mid D)=m_j a_j/(a_j+b_j)\), while its variance is \(m_j a_jb_j(a_j+b_j+m_j)/\{(a_j+b_j)^2(a_j+b_j+1)\}\). The extra variability relative to a binomial distribution with fixed probability reflects posterior uncertainty.

Continuous Outcomes: Predicting the Future Contrast

For a continuous endpoint, consider a normal model in which the future treatment-minus-control sample mean contrast \(Z\) estimates an effect \(\delta\), with known sampling variance determined by the two future sample sizes. Assume the current posterior is \(\delta\mid D\sim N(m,v)\), and future sampling satisfies \(Z\mid\delta\sim N(\delta,w)\), where $$ w=\sigma^2\left(\frac{1}{m_T}+\frac{1}{m_C}\right). $$ Here \(\sigma^2\) is the common known within-arm variance, and the future arm outcomes are independent conditional on their means. This contrast formulation assumes the likelihood information relevant to the decision is represented by \(Z\); it is not a substitute for a model with an important nuisance parameter or a more complex endpoint structure.

A final rule can require the posterior probability of a positive effect to reach \(q\), where \(0<q<1\). Conditional on future contrast \(Z\), normal updating gives posterior variance \(v_f=(1/v+1/w)^{-1}\) and posterior mean \(m_f=v_f(m/v+Z/w)\). The criterion \(P(\delta>0\mid D,Z)\ge q\) is equivalent to \(m_f\ge z_q\sqrt{v_f}\), where \(z_q\) is the \(q\) quantile of the standard normal distribution. The predictive distribution before observing \(Z\) is \(Z\mid D\sim N(m,v+w)\).

To see the cutoff, substitute the posterior mean into the final rule: \(v_f(m/v+Z/w)\ge z_q\sqrt{v_f}\). Rearranging for \(Z\) gives \(Z\ge \frac{v+w}{v}z_q\sqrt{v_f}-\frac{w}{v}m\), which is \(Z\ge z_{\mathrm{cut}}\). Because \(Z\mid D\) is normal with mean \(m\) and variance \(v+w\), its upper-tail probability is the expression above. The allocation enters through \(w\): a smaller future count in either arm increases the contrast variance.

Allocation Comparisons, Diagnostics, and Limitations

The allocation ratio affects precision through the sum of the two reciprocal future sample sizes. With a fixed total, a more balanced allocation generally gives a more precise unadjusted contrast when the arm variances and per-participant costs are equal. It does not follow that every decision rule’s predictive success must increase under every reallocation: the current posterior mean, variances, criterion, and allocation direction all matter. Allocation should be evaluated by recomputing the predictive probability, not by relying on a general slogan.

Useful checks help reveal mistakes. For binary outcomes, verify that each arm’s predictive probabilities sum to 1 and that the treatment count ranges from 0 to \(m_T\), while the control count ranges from 0 to \(m_C\). For continuous outcomes, check that increasing either future sample size lowers \(w\), and check that the predictive variance is \(v+w\), not \(w\) alone. If a calculation produces a probability outside 0 to 1, or a success event inconsistent with the stated final rule, revisit the threshold and tail direction.

An alternative to exact summation or a normal-tail calculation is simulation: draw arm-specific parameters from their current posteriors, generate future data using each arm’s own future sample size, update the posterior, and record whether the final rule passes. Simulation is useful when the model or rule makes exact calculation impractical, but it introduces Monte Carlo error. A plug-in calculation that fixes each response probability at its posterior mean is simpler but generally understates predictive variability; it does not integrate over posterior uncertainty.

These calculations depend on the model. Independence between arm parameters may be inappropriate under a shared hierarchical model; overdispersed binary outcomes or correlated continuous outcomes require a suitable likelihood; and uncertainty about the control response model is not resolved simply by using unequal counts. Also, a planned allocation is not the same as the realized final allocation if participants are lost or enrollment stops early. Model those mechanisms when material rather than silently substituting planned counts.

• Pooling the future counts. The arms have distinct outcomes and distinct predictive distributions. Keep \(m_T\) and \(m_C\) separate.
• Using a binomial plug-in prediction. A posterior mean response probability does not represent posterior uncertainty. Integrate over the posterior or simulate from it.
• Confusing a final posterior threshold with predictive success. The former is evaluated after the final data; the latter is the probability, before those data arrive, that the rule will be met.
• Assuming the most balanced allocation always wins. That conclusion requires conditions such as equal variances and costs; assess the actual model and decision rule.