Why Impute Pending Responses?
When outcome follow-up is delayed, a new allocation may be due before every previously assigned participant has a recorded response. Waiting avoids filling in unknown outcomes but leaves the allocation rule based on less information. Plug-in imputation offers a different operational choice: predict each pending response from a model fitted to outcomes that have matured, then use the predictions in an allocation summary. This can make baseline information about pending participants relevant to the update.
The prediction is not an observed response. It is a model-based stand-in used to calculate an allocation target. In particular, substituting predicted values does not make the pending outcomes known, and the resulting allocation summary should not automatically be treated as an ordinary complete-data estimate.
Let \(i\) denote an arm, \(n_i\) the number assigned to that arm so far, \(m_i\) the number whose outcomes have matured, and \(r_i=n_i-m_i\) the number with pending outcomes. For each pending participant \(j\), write \(\tilde y_{ij}\) for the plug-in prediction. The imputed arm summary combines observed responses with these predictions, divided by all \(n_i\) assigned participants. The summary is an input to an allocation rule, not a claim that the \(r_i\) unknown responses equal their predictions.
Models and Assumptions
For a binary endpoint coded \(Y=1\) for success and \(Y=0\) otherwise, a fitted model supplies a predicted success probability \(\hat p_i(x)\) for arm \(i\) and available predictors \(x\). A logistic model is one common choice, with \(\hat p_i(x)=1/(1+\exp[-(\hat\beta_0+\hat\beta_i+\hat\beta_x^\mathsf{T}x)])\). The specific model need not be logistic, but predictions used as probabilities must lie between zero and one.
For a continuous outcome, a fitted conditional-mean model supplies \(\hat\mu_i(x)\), such as a regression prediction. In either case, the model is fitted using matured outcomes and information available for those participants. Predictors for a pending participant must also be available when the allocation is updated; baseline covariates are a straightforward example.
- Outcome-model adequacy: the fitted model should reasonably describe the conditional mean for participants whose outcomes are pending, not only those who have matured.
- Representativeness of matured responses: conditional on modeled information, matured participants should provide relevant information about pending participants. If maturity depends on unmodeled outcome features, the fitted relationship may not transfer.
- Correct timing: the model and predictions use only information available at the scheduled update. No pending outcome is allowed to influence its own prediction.
- Allocation-rule definition: the mapping from imputed summaries to target probabilities must be specified, including any lower bounds or caps required by the design.
These are substantive conditions, not consequences of the arithmetic. In particular, randomization does not by itself guarantee that the observed matured subset is representative when outcome delay is related to response. A plug-in update can still be calculated when assumptions are doubtful, but the allocation target may then be misleading.
Forming Imputed Arm Summaries
For a binary outcome, let \(S_i\) be the number of observed successes among the \(m_i\) matured participants. For the pending participants, use the fitted probabilities \(\hat p_i(x_{ij})\). The plug-in arm success summary is
where \(\mathcal P_i\) is the set of pending participants assigned to arm \(i\). The numerator can be fractional: a predicted success probability contributes its expected number of successes under the fitted model, not a fabricated observed success or failure.
For a continuous outcome, let \(T_i\) be the sum of matured outcomes in arm \(i\). The corresponding summary is
These formulas simply average observed values and conditional-mean predictions over everyone assigned. They do not describe the uncertainty in the unobserved outcomes or in the fitted model.
Worked Example: Binary Outcomes With Different Pending Predictions
Suppose a hypothetical trial has two arms, and higher binary success rates are preferred. In arm A, 10 outcomes have matured, with 6 successes; two more participants are pending. A fitted model using matured outcomes and baseline predictors gives pending success probabilities 0.70 and 0.40. In arm B, 10 outcomes have matured, with 4 successes; two are pending, with fitted probabilities 0.50 and 0.30. Assume for this illustration that the model is adequate for the pending participants and all listed predictors were available at the update.
For arm A, the predicted number of pending successes is \(0.70+0.40=1.10\). Thus \(\tilde p_A=(6+1.10)/12=7.10/12=0.5917\). For arm B, it is \(0.50+0.30=0.80\), giving \(\tilde p_B=(4+0.80)/12=4.80/12=0.4000\).
For a simple target, suppose allocation probabilities are proportional to these positive success summaries, with no additional cap or minimum in this example. Using the unrounded summaries, \(q_A=0.5916667/(0.5916667+0.4000)=0.5966\), and \(q_B=1-q_A=0.4034\). These are target allocation probabilities for the next randomization under the stated rule, not probabilities that a particular participant will succeed. The calculation also depends on the fitted predictions: if the pending participants’ covariates were ignored, the summaries and target could differ.
From Imputed Summaries to an Allocation Target
An allocation target is the probability distribution that the design aims to use for assigning the next participant or cohort. It is obtained by applying a prespecified rule to the current response summaries. For example, if a binary success rate is the utility and both arm summaries are positive, a proportional target is \(q_i=\tilde p_i/\sum_k\tilde p_k\). Other response-adaptive rules can transform summaries into scores first; the scores are then normalized to sum to one. Any required exploration minimum or allocation cap is applied as specified by the design.
For continuous outcomes, the direction of benefit matters. If smaller values are better, a rule that treats the mean itself as a positive utility would favor the wrong arm. A decreasing transformation can instead convert a mean outcome into a positive score. The transformation, its scale, and the direction of benefit must all be declared before interpreting the resulting target.
Worked Example: Continuous Outcomes and a Decreasing Utility
Consider a hypothetical trial of a continuous endpoint measured in units for which lower values are better. Arm A has four matured outcomes summing to 20 and two pending participants whose fitted conditional means are 5 and 7. Arm B has three matured outcomes summing to 20 and one pending participant whose fitted conditional mean is 7. Assume the continuous outcome model is adequate for these participants, and that all predictions use predictors available at the allocation update.
Arm A’s imputed mean is \(\tilde\mu_A=(20+5+7)/6=32/6=5.333\) outcome units. Arm B’s is \(\tilde\mu_B=(20+7)/4=27/4=6.750\) outcome units. To reward lower means, suppose the specified allocation score is \(u_i=\exp(-\tilde\mu_i/10)\), where 10 outcome units is the prespecified scale. The scores are \(u_A=\exp(-(32/6)/10)=\exp(-8/15)=0.5866\) and \(u_B=\exp(-6.750/10)=\exp(-0.675)=0.5092\), to four decimal places.
The target is formed from the unrounded scores: \(q_A=u_A/(u_A+u_B)=0.535\) and \(q_B=1-q_A=0.465\), to three decimal places. Thus this rule gives arm A the larger target probability because its imputed mean is lower. The scale of 10 is part of the rule: a different scale would change the relative scores and therefore the target. It does not change the underlying outcome means.
Why Constant-Mean Imputation May Do Nothing
A particularly important case is imputing each pending outcome in an arm with that arm’s matured sample mean. This may sound like including the pending participants, but it leaves the arm mean unchanged. The short derivation makes the point explicit.
The same identity holds for a binary success proportion if each pending outcome is imputed with the matured success proportion. Such imputations do not add new information about differences between pending participants; they merely reproduce the current arm estimate. By contrast, covariate-specific predictions can change the summary when the pending participants have predictors associated with outcome. This change is useful only if the predictor relationship is credible.
Worked Example: Constant Binary Imputation Leaves the Summary Unchanged
Suppose a hypothetical arm has three matured binary outcomes, \(0,1,1\), and two pending participants. The matured success proportion is \(2/3=0.6667\). If each pending response is assigned that same plug-in probability, the imputed total success count is \(2+2(2/3)=10/3\). The summary across all five assigned participants is \(\tilde p=(10/3)/5=2/3=0.6667\).
The arithmetic is internally consistent, but the two predictions do not distinguish the pending participants or alter the estimate. If the allocation target changes after such an update, the change must come from some other component of the rule, not from new information supplied by these constant imputations.
Diagnostics, Alternatives, and Limitations
Before using a plug-in target, check whether the predictions are plausible in the context of the matured data. For binary outcomes, inspect the predicted probabilities and compare predicted success totals with observed totals across arms and relevant baseline groups. For continuous outcomes, check whether predicted values lie in a plausible range and whether residual patterns among matured participants suggest a poor conditional-mean fit. Also inspect how many responses have matured in each arm: predictions based on very few matured outcomes can be unstable even if the model’s formula looks reasonable.
A further diagnostic is sensitivity to plausible modeling choices. Recalculate the target under reasonable alternative predictor sets, model specifications, or assumptions about pending participants. If modest changes produce substantially different targets, the allocation update is model-sensitive. A target that moves toward one arm can also reduce future data from another arm, making poor predictions increasingly consequential. Design safeguards such as minimum allocation probabilities can limit this feedback, but they do not repair a misspecified outcome model.
| Approach | What enters the update | Main trade-off |
|---|---|---|
| Use matured outcomes only | Observed responses whose outcomes are available | Avoids imputing pending values, but ignores their predictors and may use a small or stale response set. |
| Wait for more outcomes | Observed responses after a scheduled wait | Avoids plug-in predictions, but delays the allocation update. |
| Plug-in imputation | Observed responses and fitted conditional-mean predictions for pending participants | Can use available predictor information promptly, but may understate uncertainty and inherit model or maturity-selection bias. |
Plug-in imputation has several limitations. Predicted means do not represent the random variation of individual pending outcomes, and an allocation rule based on them treats the fitted predictions as fixed inputs for that update. The resulting target is therefore conditional on the chosen model and current data. It is not, by itself, a full accounting of uncertainty in outcomes or model parameters. Moreover, a prediction model fitted to matured responses may be biased for pending participants if maturity is associated with outcomes after accounting for the variables in the model. The appropriate response to serious lack of overlap or poor fit may be to wait, use a more defensible model, or rely on the design’s prespecified safeguards—not to present an unreliable prediction as an observed response.
The practical distinction is between computing a target and validating the assumptions that make the target informative. A calculation can be exact while its input predictions are poor. Document the model, predictors, fitting data, update time, and transformation from summaries to allocation probabilities so the operational rule is reproducible.
Check Your Understanding
Answer each question, then compare with the brief solutions below.
- An arm has 8 matured binary outcomes with 5 successes and 2 pending participants. Their fitted success probabilities are 0.25 and 0.75. What is the plug-in success summary?
- Why does imputing every pending continuous outcome with the matured arm mean leave that arm’s imputed mean unchanged?
- In a trial where smaller continuous outcomes are better, what must be checked before normalizing transformed arm scores into an allocation target?
- Name two reasons a plug-in target can be misleading even if its arithmetic is correct.
Solutions. (1) The predicted pending total is \(0.25+0.75=1.00\), so the summary is \((5+1.00)/(8+2)=0.6000\). (2) Each prediction equals \(T_i/m_i\), so adding \(r_iT_i/m_i\) to the observed total and dividing by \(m_i+r_i\) simplifies to \(T_i/m_i\). (3) The score transformation must favor lower means, be defined on the chosen scale, and produce valid nonnegative scores that can be normalized; any caps or minimum probabilities must also be applied as specified. (4) The outcome model may be misspecified or poorly estimated, and matured participants may not represent pending participants conditional on modeled information. Predictions also omit individual outcome and model uncertainty.