Using Partial Outcome Information

Why Use Partial Outcome Information?

In a response-adaptive trial, final outcomes may arrive too slowly for every allocation update to use only matured responses. An interim measurement can provide information sooner: for example, an early symptom score may help predict a later score. Using that information may make updates more responsive without pretending that the final response has already been observed.

The central statistical task is to connect the interim measurement to the eventual response under an explicit predictive model. The connection is not automatic. A partial measurement that is strongly associated with the final outcome in one setting may be poorly predictive in another, and a prediction is not a realized endpoint. The allocation rule should use predictions only under stated assumptions and with a clear account of their uncertainty.

The Prediction Assumptions

Let \(D\) denote all information available at an allocation update, including the allocation history and currently observed trial data. For a pending participant \(j\), a useful predictive quantity is the conditional mean \(m_j=E(Y_j\mid D,A_j,Z_j,X_j)\). In a Bayesian analysis, this expectation can also average over uncertainty in model parameters given the data. In a frequentist implementation, a fitted conditional mean may be used, but its estimation uncertainty should not be ignored when it is material.

The conditioning set matters. It is not generally valid to replace \(E(Y_j\mid D)\) with a conditional expectation that omits parts of \(D\): those omitted parts might contain information about \(Y_j\). A model using \(A_j,Z_j,X_j\) alone requires a condition that these variables capture all relevant predictive information in \(D\), or an appropriate conditional-sufficiency assumption. Otherwise, include relevant components of \(D\) in the prediction model.

Several practical assumptions support this use of \(m_j\). The partial measurement must be recorded before the final response is available and be measured consistently. The model linking \(Z_j\) to \(Y_j\) must be appropriate for the treatment arm and trial population at the time of the update. The measurement process and the reasons a partial measurement is unavailable must not create unmodelled selection that distorts predictions. Finally, the randomization mechanism must be followed: treatment assignment is randomized according to the specified probabilities conditional on the information available before assignment. These conditions do not make every prediction correct, but they clarify what the prediction relies on.

The outcome scale also needs attention. If a larger \(Y_j\) means worse health, the allocation score should represent the trial’s intended utility or benefit direction rather than blindly favoring a larger numerical response. And although treatment is randomized, the association between \(Z_j\) and \(Y_j\) is a predictive relationship, not evidence that changing \(Z_j\) would cause a change in \(Y_j\).

Combining Matured Outcomes and Predictions

One simple arm-level score is the average of observed final outcomes and predicted final outcomes among participants in that arm. It can be useful for illustrating the update, but it is not automatically an unbiased estimate of an arm’s population mean: participant composition, prediction error, and adaptive assignment all matter. More elaborate allocation rules may use posterior predictive distributions or other summaries. Whatever the rule, the predicted values should remain distinguishable from observed outcomes.

The following calculation uses a deliberately simple, hypothetical allocation rule. Suppose larger values of \(Y\) are better, and define the probability of assigning the next participant to arm B by \(p_B=1/\{1+\exp[-\kappa(\hat{\mu}_B-\hat{\mu}_A)]\}\), with \(p_A=1-p_B\). Here \(\hat{\mu}_a\) is an arm-level response score and \(\kappa\) controls how strongly the allocation responds to a score difference. The logistic rule is an illustration, not a general prescription; the chosen rule should also preserve acceptable allocation probabilities to each arm.

Justifying Predictions and Representing Uncertainty

The relevant expectation and variance identities follow from conditioning on nested information sets. Let \(D\) be information available before an additional partial measurement \(Z_j\) is observed. Since the information in \(D\) is contained in the information in \((D,Z_j)\), the law of iterated expectations gives

The corresponding conditional variance decomposition is

The first term is the expected residual uncertainty about the final outcome after the partial measurement is known. The second is uncertainty in the predicted mean before that measurement is known. Both terms matter for a pending outcome. If the current information already includes \(Z_j\), condition on that information in the prediction; do not use the pre-measurement decomposition as if \(Z_j\) were still unknown. More generally, conditioning on \((D,Z_j)\) is important precisely because \(D\) may contain relevant information beyond \(A_j,Z_j,X_j\).

Checking the Link Between Interim and Final Outcomes

The prediction model should be checked against participants whose final outcomes have matured. A useful check compares predicted and observed final outcomes at the relevant prediction times, separately by treatment arm and, where appropriate, by important baseline groups or calendar period. Systematic prediction errors, such as consistently overpredicting final response for one arm, indicate that the partial-to-final link may not transport to the current trial data.

Calibration summaries and plots can reveal whether predictions are approximately correct on average and across the range of predicted values. Residual patterns can reveal a misspecified relationship, such as a slope that differs by treatment or a nonlinear association. Check whether the interim measure is collected at comparable times and whether missing interim measurements cluster among participants with different likely outcomes. When the relationship changes over time, a model estimated from earlier participants may become unreliable for later updates.

Allocation probabilities themselves are also a diagnostic target. If small changes in a few predictions cause large shifts in allocation, the update may be too sensitive to model noise. Sensitivity analyses can compare plausible prediction models, shrink extreme predictions toward a reference value, or limit how strongly the score difference changes allocation. Such safeguards do not repair a biased prediction model, but they can make the design less vulnerable to unstable predictions.

Alternatives, Limitations, and Common Mistakes

Using only matured outcomes avoids relying on an interim-to-final prediction model, but it can leave fewer observations informing an update and can make the information used by the design lag behind enrollment. Partial-outcome prediction can use more current information, at the cost of additional modelling assumptions. Imputation approaches for pending outcomes provide another way to represent uncertainty; a single plug-in prediction is not equivalent to carrying uncertainty across imputations or through a predictive distribution. These approaches should be compared based on their assumptions and their consequences for allocation, not just on how quickly they produce an update.

Partial-outcome updates have limits. A predictive relationship can change with treatment, population, measurement timing, or clinical practice. A strong association does not guarantee accurate predictions for every subgroup or accurate arm comparisons. Predictions also depend on the amount of mature data available to check and estimate the model. When the link is weak or poorly supported, using the partial measurement may add noise rather than useful information.

A common mistake is to say, “The interim score is 6, so the final response is 6,” and enter 6 as though it were observed. The correction is to use a model for \(E(Y_j\mid D,A_j,Z_j,X_j)\), state why that model is credible, and retain the distinction between predicted and observed responses. A second mistake is to condition only on a convenient set of predictors while silently assuming it contains all relevant information. The correction is either to justify that sufficiency condition or to condition the prediction on the other relevant available information as well.

In practice, document the interim measurement schedule, the prediction model and its treatment-specific assumptions, the information used at each update, and the allocation rule that converts scores to probabilities. Prespecifying how model failure or extreme probabilities will be handled makes the update more transparent. Randomization should remain genuine: each eligible arm should retain a positive assignment probability under the design.