Pending Outcomes in Adaptive Allocation

Why Pending Outcomes Need an Explicit Rule

In an adaptive trial, allocation probabilities may be updated as outcome data accumulate. But participants can be enrolled faster than their outcomes are observed. If the next update uses only outcomes already available, patients with short follow-up contribute sooner than patients whose outcomes take longer to ascertain. A protocol therefore needs to specify how pending responses enter the allocation calculation, rather than treating their status as an incidental data-management detail.

A pending outcome is not necessarily a missing outcome. It may be unobserved only because the participant has not yet reached the assessment time or because adjudication is still underway. The distinction matters: a prediction for an outcome that is expected to arrive can be used to anticipate a later update, whereas a permanently unavailable outcome requires assumptions about why it is missing. The earlier material on Allocation Updates With Delayed Outcomes concerns when an update occurs; here the focus is what information to use at that update.

The calculations below use a binary response and a simple allocation rule based on posterior expected response. These examples isolate the pending-outcome issue; actual trials may use a different prespecified allocation rule, outcome model, or utility. Let \(D_{\mathrm{obs}}\) denote the available observed data and \(Y_{\mathrm{pend}}\) the vector of pending outcomes. Write \(\pi_a\) for the probability of assigning the next participant to arm \(a\).

Three Ways to Handle Pending Responses

Exclusion calculates the current posterior using observed outcomes only. Pending participants contribute no response value to this update, although their known treatment assignments and other observed information may remain in the trial records. This is simple and transparent: the allocation rule uses only verified responses. Its cost is that an update may lag behind the enrolled sample, and the available responses may not represent all enrolled participants if outcome availability is related to prognosis, treatment, or follow-up time.

Imputation replaces each pending response with a selected value, such as a predicted mean, a success, or a failure, and then calculates allocation probabilities from the resulting completed data. A fixed imputation produces one completed data set and one allocation vector. The result can be sensitive to the selected value. In a Bayesian analysis, inserting a predictive mean as if it were an observed fractional response is generally not the same operation as integrating over the possible outcomes.

Predictive averaging represents pending outcomes by their posterior predictive distribution given the data currently available. For each possible pending-outcome configuration, calculate the allocation probabilities that would follow if that configuration were observed. Then average those allocation probabilities using the predictive probabilities of the configurations.

For continuous outcomes, the sum is replaced by an integral. The predictive distribution must be based on a specified outcome model and the information available at the decision time. For example, a binary-response model might use a Beta prior and a Bernoulli sampling model. If pending outcomes are conditionally dependent, the joint predictive distribution—not a product of marginal distributions—must be used.

A Simple Allocation Rule for Illustration

Suppose arm \(a\) has binary response probability \(\theta_a\), and use independent \(\operatorname{Beta}(1,1)\) priors for the arms. If \(r_a\) responses are observed among \(n_a\) participants, the posterior is \(\operatorname{Beta}(1+r_a,1+n_a-r_a)\), with posterior mean \((1+r_a)/(2+n_a)\). For this tutorial’s numerical examples, define the allocation probability to arm A as its posterior mean response divided by the sum of the posterior mean responses for the two arms:

This rule is a deliberately simple example of allocation by posterior expected utility when utility is binary response. It is not a universal rule, and it should not be confused with posterior probability matching. The pending-outcome comparison applies more broadly: replace this allocation function with the trial’s prespecified function, then calculate or approximate its predictive average.

Why Predictive Averaging Can Change the Update

For a binary pending response \(Y\), the posterior predictive distribution describes the probability of each response value conditional on the observed data. For example, if \(\theta\mid D_{\mathrm{obs}}\sim\operatorname{Beta}(a,b)\) and \(Y\mid\theta\sim\operatorname{Bernoulli}(\theta)\), then \(P(Y=1\mid D_{\mathrm{obs}})=a/(a+b)\). If the response is later observed to be 1, the posterior becomes \(\operatorname{Beta}(a+1,b)\); if it is 0, it becomes \(\operatorname{Beta}(a,b+1)\).

A short justification for predictive averaging follows directly from conditioning on the unknown pending response. Given a possible value \(y\), the allocation rule returns \(\pi_a(D_{\mathrm{obs}},y)\). Before \(y\) is known, the expected value of that allocation probability is the sum of these possible values weighted by their predictive probabilities. Thus the average is based on both the outcome model and the allocation rule. If the rule is nonlinear, putting the predictive mean outcome into the rule need not give the same result.

Imputation Choices and Their Consequences

Imputation is not one uniquely defined procedure. A trial might assign every pending response a fixed category, use a model-based predicted value, or draw a plausible value from the predictive distribution. These choices answer different questions. A fixed success imputation can temporarily favor an arm; a fixed failure imputation can disfavor it. A predictive-mean imputation suppresses uncertainty about the pending response, while a predictive draw retains uncertainty for one completed data set but does not, by itself, average over that uncertainty.

The distinction between one imputation and repeated predictive averaging is important when the allocation function is nonlinear. A multiple-imputation-style calculation can approximate predictive averaging by drawing many pending-outcome configurations, applying the allocation rule to each, and averaging the resulting allocation probabilities. It should use the joint predictive model and enough draws for Monte Carlo error to be small relative to any operationally important allocation difference.

Assumptions, Diagnostics, and Failure Modes

Each approach is conditional on a clear definition of which outcomes are pending and what data are available at the allocation cutoff. Predictive averaging additionally requires a predictive model that is appropriate for the endpoint and follow-up process. The model must reflect the response definition, assessment window, and any relevant information observed before the update. If the likelihood assumes independent outcomes, that independence assumption must be reasonable conditional on the model parameters and available data; shared site effects or other dependence may require a joint model.

A useful operational check is to compare the number and pattern of pending outcomes across arms and over calendar time. Large arm differences in outcome availability, or a concentration of pending outcomes among participants with particular baseline characteristics or early clinical events, can make an exclusion-only update behave differently from an update based on the eventual outcomes. For predictive methods, compare observed outcomes with predictive distributions as they arrive, and check calibration by arm and by relevant follow-up time. A model that systematically overpredicts responses will tend to allocate too favorably to the affected arm.

The main failure modes are treating a delayed value as if it were permanently missing, using a predictive model that ignores informative outcome timing, and allowing an undocumented imputation choice to influence allocation. A further risk is operational: an elaborate predictive calculation may be difficult to reproduce or may introduce substantial Monte Carlo error if implemented carelessly. The protocol should identify the data cutoff, model, calculation, software checks, and what happens when data are unavailable or the model cannot be fit.

These approaches are alternatives for calculating allocation probabilities, not substitutes for defining the trial’s endpoint or follow-up procedures. Exclusion may be reasonable when outcome delays are short and an update based on verified responses is the prespecified rule. Predictive averaging can be attractive when pending data are numerous and a suitable model is available, but it does not repair a misspecified outcome model. Fixed imputation may be useful as a prespecified sensitivity or operational rule, but its implications should be evaluated before the trial begins.