Multiple Imputation for Pending Trial Outcomes

Why Impute More Than Once?

When outcomes are delayed, an allocation update may be due while some enrolled patients still have unknown outcomes. A predictive model can supply plausible values for those pending outcomes, but any one completed data set is only one plausible account of what happened. If the allocation rule responds to the completed data, using just one account can make the next allocation depend too heavily on an arbitrary draw or point prediction.

Multiple imputation addresses this by drawing several plausible sets of pending outcomes, applying the allocation rule to each completed data set, and aggregating the resulting targets. The uncertainty about pending responses is therefore carried into the allocation calculation. The procedure is not a license to treat imputed values as observed facts: the draws are model-based predictions, and the quality of the result depends on the model and its assumptions.

The target aggregation rule must be specified. In this tutorial, the illustrative allocation rule assigns an arm a target proportion equal to its estimated response probability divided by the sum of the estimated response probabilities across arms. Other rules are possible; the key multiple-imputation step is to calculate the chosen rule within each imputation before aggregating.

Predictive Draws for a Pending Binary Outcome

Suppose the outcome is binary, with response coded as 1 and no response as 0. For an arm with unknown response probability \(\theta\), use a Bernoulli sampling model and a Beta prior. Conditional on \(\theta\), outcomes are independent Bernoulli variables. A Beta prior is conjugate: after observing \(s\) responses and \(f\) nonresponses, the posterior is Beta with parameters equal to the prior parameters plus those counts. The posterior predictive probability of response for one pending patient is the posterior mean of \(\theta\).

For a transparent hypothetical example, use independent \(\mathrm{Beta}(1,1)\) priors for arms A and B. Assume the pending patients’ outcomes are exchangeable with the matured outcomes within their assigned arms under this model. The example does not include covariates, and it assumes that outcome availability does not convey extra information about the response itself after conditioning on the model. These assumptions may be unsuitable in a real trial and must be assessed rather than presumed.

Calculate a Target in Every Imputation

Let \(g(D^{(m)})\) be the allocation target calculated from completed data set \(m\), where \(m=1,\ldots,M\). A common Monte Carlo aggregation is

Each \(D^{(m)}\) includes a predictive draw for every pending outcome. The transformation \(g\) is applied after making those draws. In general, averaging the transformed targets is not the same as applying the allocation rule to averaged imputed outcomes, because allocation rules are usually nonlinear.

Why the Order of Calculation Matters

A single imputation might replace the pending outcome by its most likely value, here a response because \(0.6364>0.5\). That produces the target 0.5946. If the single imputation instead happened to be a nonresponse, the target would be 0.5620. Neither result averages over the predictive uncertainty. A point prediction that substitutes the response probability itself is another shortcut: it is a fractional value, not a possible binary outcome, and it must not be confused with a predictive draw.

For this example, substituting the expected outcome \(7/11\) into the posterior update gives an A mean of

The resulting target is \((7/11)/(7/11+5/11)=7/12\approx 0.5833\), close to but not equal to the multiple-imputation average 0.5828. The difference arises because the target is a nonlinear function of the completed data. In general, \(E\{g(D_{\mathrm{complete}})\}\) need not equal \(g\{E(D_{\mathrm{complete}})\}\).

This distinction is the central calculation principle: draw completed data, calculate a target for each draw, then aggregate targets. Do not average pending outcomes first and assume the allocation rule will give the same answer.

Several Pending Outcomes and Shared Parameter Uncertainty

When several outcomes from the same arm are pending, a coherent Bayesian predictive draw reflects uncertainty both about the arm’s response probability and about each patient’s response. One approach is to draw \(\theta_A\) from its posterior once within an imputation, then draw each pending outcome conditionally independently as Bernoulli with probability \(\theta_A\). This induces dependence between pending outcomes after \(\theta_A\) is integrated out. Drawing each outcome independently from a Bernoulli distribution with the posterior mean instead ignores that shared-parameter uncertainty.

Implementation Checks, Alternatives, and Limitations

A multiple-imputation allocation procedure needs a sufficiently specified predictive model and a sufficiently precise aggregation rule. Before using it, check that the model uses the information available at the update, that pending patients are predicted using their assigned arms and any relevant modeled characteristics, and that the predictive draws reflect parameter uncertainty as intended. For the simple Beta-Bernoulli model here, the posterior is proper because both arms have observed responses and nonresponses. Other models require their own checks.

The principal alternative is to use matured outcomes only, which avoids assigning predicted values but can leave the allocation rule behind the current enrollment. A single imputation is computationally simpler, but its target can depend strongly on one arbitrary completed data set. Multiple imputation averages that dependence across predictive draws; it does not remove bias caused by a misspecified predictive model or make an incorrect delay assumption harmless. It also requires more computation, and a small \(M\) can make the target noisy. Increasing \(M\) reduces Monte Carlo error in the average, but cannot repair a poor model.

Keep three quantities distinct: the pending patient’s unknown response, the predictive distribution describing uncertainty about it, and the allocation target calculated from a completed data set. The target is a design input, not an observed response rate, a guarantee of patient benefit, or a substitute for the trial’s prespecified randomization and safety rules.

• Check the draw mechanism. For multiple pending outcomes sharing an uncertain arm probability, determine whether the implementation draws the probability and then the outcomes, or incorrectly treats outcomes as independent at a fixed posterior mean.
• Check the aggregation. Confirm that the allocation rule is applied within each imputation before targets are combined.
• Check Monte Carlo stability. Repeat the computation with more imputations or independent random seeds. Large changes in the average indicate that the simulation count may be inadequate.
• Check sensitivity. Compare targets under plausible predictive-model specifications. A stable Monte Carlo estimate can still be sensitive to modeling assumptions.

A common mistake is to say, “The most likely pending outcome is a response, so impute a response once and use that target.” That reasoning turns a probability of 0.6364 into a certainty. The correction is to draw multiple plausible outcomes from the predictive model, calculate the target separately for each, and aggregate those targets according to the prespecified rule.