Handling Missing Outcomes at the Stage Transition

Why the Phase 2 Cutoff Needs a Missing-Data Rule

At a seamless phase 2/3 transition, the phase 2 data cutoff is a boundary for the adaptation decision, not necessarily the date on which every participant’s endpoint can be observed. Someone randomized before the transition may still be awaiting an endpoint assessment when the phase 2 data are extracted. That participant belongs to the stage 1 cohort, even though their outcome is absent from the cutoff data.

This distinction matters because the adaptation uses stage 1 evidence, while confirmatory inference depends on a clearly defined population and analysis. Excluding everyone without an observed outcome changes the analyzed set. Whether that changes the estimand or biases its estimate depends on the missingness process and the analysis assumptions; randomization alone does not make missing outcomes harmless.

For clarity, let \(A_i\) denote participant \(i\)’s randomized treatment, \(Y_i\) the endpoint defined by the protocol, and \(R_i=1\) if that endpoint is observed in the phase 2 cutoff extract (with \(R_i=0\) otherwise). The cutoff does not itself determine whether \(Y_i\) is permanently missing: an endpoint may be pending, missed, or unavailable for another reason. Record the reason and timing where possible. A cutoff snapshot should not silently turn “not yet observed” into “participant excluded.”

The protocol should define the stage 1 target population, endpoint, estimand, cutoff, and missing-data procedure before the adaptation is made. For example, if the target is the treatment-policy difference in mean endpoint values among all stage 1 randomized participants, the target includes those participants regardless of treatment discontinuation. A complete-case analysis instead calculates a contrast among participants with observed outcomes; it estimates the all-randomized contrast only under additional assumptions.

Missingness Assumptions and What They Permit

Let \(X_i\) denote measured baseline covariates and, where relevant, observed information collected before the cutoff. Missingness assumptions describe how the probability of \(R_i=1\) relates to the outcome and these observed data. They are not interchangeable labels for the same situation.

For an outcome mean within treatment arm \(a\), MCAR implies \(R_i\) is independent of \(Y_i\) within that arm. Under MAR, a more general condition is \(R_i\) independent of \(Y_i\) conditional on \(A_i\) and \(X_i\). These conditions concern the data-generating process, not merely the analyst’s ability to fit a model. In particular, similar observed baseline summaries across participants with and without outcomes do not prove MCAR.

A common first analysis is complete-case analysis: calculate the treatment contrast using only participants with \(R_i=1\). If MCAR holds within treatment arms, the observed outcomes represent the outcome distributions in those arms. More precisely, conditional on a nonzero observed sample size, the observed values have the same distribution as the arm’s outcomes, so their sample mean targets the arm’s all-randomized mean. Under MAR rather than MCAR, an unadjusted complete-case comparison can still be biased if observation rates vary with covariates related to the outcome.

Why Complete-Case Analysis Can Work Under MCAR

For arm \(a\), suppose outcomes are independently sampled from the arm’s target population and observation is independent of the outcome within that arm. Conditional on there being \(m_a>0\) observed outcomes, each observed value has the same mean as a randomly selected outcome in that arm. Therefore:

Here \(\bar{Y}_{a,\mathrm{obs}}\) is the mean among observed outcomes in arm \(a\). The complete-case difference between arms consequently targets the all-randomized treatment difference under these assumptions. This justification does not apply merely because treatment was randomized: randomization balances treatment assignment at baseline, but does not ensure that \(R_i\) is independent of \(Y_i\).

Using Observed Predictors Under MAR

When observation depends on measured variables, analyses can use those variables to account for differences between observed and unobserved participants. Two common approaches are multiple imputation and inverse probability weighting. Multiple imputation draws plausible missing outcomes from a model conditional on observed information, analyzes each completed data set, and combines the results to reflect imputation uncertainty. It requires a sufficiently specified imputation model and a MAR assumption given the variables included. Inverse probability weighting gives more influence to observed participants who resemble participants with a lower probability of observation.

For inverse probability weighting, let \(\pi_i=P(R_i=1\mid A_i,X_i)\) be the probability that participant \(i\)’s outcome is observed, estimated or known under the assumed observation model. A weighted mean for arm \(a\) is:

This estimator requires an appropriate observation model, MAR conditional on the variables used, and positivity: each relevant type of participant in each arm must have a nonzero chance of an observed outcome. If some \(\pi_i\) values are very small, weights can become large and estimates unstable. Uncertainty calculations should account for estimation of the observation probabilities when they are estimated from the data.

MNAR Sensitivity Analysis and Diagnostics

When missingness may depend on the unobserved outcome after conditioning on available information, the data alone generally cannot identify the missing outcomes’ distribution. An MNAR sensitivity analysis makes the unverifiable assumption explicit and examines how results change across plausible values. One simple approach for a binary outcome is a pattern-mixture calculation: specify a success probability among participants whose outcomes are missing, separately by treatment arm, and combine it with the observed successes.

Diagnostics begin with accounting, not a model: report the numbers randomized, with observed outcomes, and without observed outcomes in each treatment arm, along with reasons and timing. Compare observation rates across arms and across baseline predictors and observed histories. Check whether the model predicts probabilities near zero, whether weighted estimates are dominated by a few participants, and whether imputation models respect the endpoint’s range and design. Such checks can reveal model problems or limited overlap, but no observed-data diagnostic can establish that missingness is MCAR or distinguish MAR from MNAR.

Implications for the Stage Transition

The phase 2 analysis used for treatment selection or another transition decision should apply a prespecified missing-data rule to a defined stage 1 population. Otherwise, the effective population may depend on who happened to have an endpoint available at the cutoff. If a stage 1 statistic or p-value is used in confirmatory inference, its validity depends on the analysis and missingness assumptions that produced it; a later combination of evidence does not repair an invalid stage 1 input.

The central choice is not “include or exclude missing participants” in isolation. It is to define the target question, state which participants belong to it, specify how unobserved outcomes are handled, and examine how credible departures from the primary assumption affect the conclusion. Pending outcomes should be distinguished in reporting from outcomes that will never be obtained, even when the same prespecified cutoff rule governs the adaptation.