Interim Decisions With Delayed Treatment Responses

Why Pending Responses Matter

In a trial with a fixed response-assessment window, patients enrolled near an interim analysis may not yet have completed that window. Their responses are unknown, not necessarily absent. Treating every pending response as a failure can make the treatment look less effective than it is; ignoring the pending patients can make the interim evidence appear more decisive than the eventual evidence. A Bayesian analysis can represent the uncertainty about these outcomes directly and calculate how likely each possible completed-data decision is.

The key distinction is between an observed-data posterior, based only on responses that have been assessed, and a posterior predictive distribution for responses that are still pending. Predictive calculations do not reveal the pending outcomes. They quantify how those outcomes could change the analysis and, under the model, how likely each change is.

Model and Decision Rule

Suppose each patient has a binary response outcome assessed at a prespecified time, and let \(p\) be the probability of response by that time. A basic model assumes that patient outcomes are conditionally independent Bernoulli variables with the same response probability \(p\). Use a beta prior, \(p\sim\operatorname{Beta}(a_0,b_0)\). After observing \(s\) responses and \(f\) nonresponses, the posterior is \(\operatorname{Beta}(a_0+s,b_0+f)\). In the examples, a stated beta distribution may summarize the prior combined with all outcomes already assessed.

For a target response rate \(p_0\), a common Bayesian efficacy rule is to declare that the evidence meets a prespecified criterion if \(P(p>p_0\mid\text{data})\) is at least a threshold \(\gamma\). The target and threshold must be chosen in advance for the trial's decision context. Meeting this rule is a posterior-probability statement under the specified model; it is not a claim that the null hypothesis has a particular probability of being true.

This model requires a clear and consistent outcome definition and assessment time. It also assumes that the pending patients belong to the same response-probability model as the assessed patients, and that their outcomes are not selectively missing in a way the model ignores. If response probabilities differ substantially across patient groups or assessment timing is informative, a single common \(p\) may be inadequate. A pending assessment should not be coded as a failure merely to make the data complete.

Updating for Pending Outcomes

If the current posterior is \(\operatorname{Beta}(a,b)\), and the \(m\) pending assessments yield \(K=k\) responses and \(m-k\) nonresponses, conjugate updating gives a completed-data posterior of \(\operatorname{Beta}(a+k,b+m-k)\). Before those outcomes are known, the predictive probability of each possible \(k\) is beta-binomial:

where \(B(\cdot,\cdot)\) is the beta function. This is not the same as plugging a single estimate of \(p\) into a binomial distribution: the beta-binomial calculation incorporates posterior uncertainty about \(p\). For example, the predictive probability of no responses among \(m\) pending patients is \(B(a,b+m)/B(a,b)\).

For integer beta parameters, a useful way to calculate the efficacy probability is \(P(p>p_0\mid\text{data})=P\{Y\leq a-1\}\), where \(Y\sim\operatorname{Binomial}(a+b-1,p_0)\) and the posterior is \(\operatorname{Beta}(a,b)\). This identity makes the following numerical calculations straightforward to check.

Justification: The Predictive Distribution

The beta-binomial formula follows by conditioning on \(p\). Given \(p\), the probability of \(k\) responses among \(m\) conditionally independent patients is \({m\choose k}p^k(1-p)^{m-k}\). Integrate this probability against the current beta posterior density. The integral is the beta function ratio in the predictive formula:

This derivation also explains why the predictive probabilities sum to one: they are the probabilities of all possible values \(k=0,\ldots,m\) under the posterior predictive distribution. For each possible \(k\), the completed-data posterior can be calculated first, and the decision rule can then be applied to that posterior. Summing the predictive probabilities for the values of \(k\) that cross the rule gives the predictive probability of crossing the threshold.

Quantifying the Risk of a Reversed Decision

A decision based on observed outcomes alone can be reversed when pending assessments arrive. The probability of reversal can be quantified by identifying which possible values of \(K\) change the decision and summing their predictive probabilities. This makes the meaning of “risk of premature stopping” precise for a specified model and decision rule: it is a posterior predictive probability of a future decision change, not a universal error rate.

Predictive Decisions Are Not the Same as Final Decisions

A separate question is whether to act now based on the probability that the eventual analysis will cross a threshold. A trial may specify a predictive-probability boundary for continuing, stopping, or waiting. Such a rule differs from applying the final posterior criterion to the observed data, and its operating characteristics should be evaluated for the intended design.

Diagnostics, Alternatives, and Limitations

Before interpreting a predictive calculation, check that the model represents the trial and that the pending group is being handled as intended. Useful checks include:

• Confirm outcome maturity. Verify that “pending” means the assessment window has not elapsed, rather than an unrecorded outcome after a completed window.
• Check exchangeability. Compare the pending patients’ enrollment periods and relevant characteristics with those of assessed patients. A common response probability may be questionable if the groups differ materially.
• Check the response-time process. If patients who respond tend to be assessed earlier than nonresponders, pending status may carry information about the outcome. The simple model above does not account for that dependence.
• Vary the prior and thresholds. Small interim samples can be sensitive to the prior and to the chosen posterior-probability cutoff. Report how the decision or predictive probability changes under defensible alternatives.

The main alternatives answer different questions. A complete-case analysis uses only assessed outcomes and is transparent, but does not quantify how pending patients could change the decision. Coding pending outcomes as failures is conservative in a particular direction, but it changes the data definition and may distort response probability when assessment delay is administrative. Waiting until all assessments mature avoids this interim uncertainty, at the cost of delay. A posterior predictive analysis makes uncertainty explicit, but its conclusions depend on its likelihood, prior, and assumptions about assessment and patient comparability.

A model for binary outcomes at a fixed assessment time is not automatically suitable for time-to-response data, competing events, or informative dropout. Nor does a posterior predictive reversal probability equal a frequentist false-positive rate under a null hypothesis. If enrollment, assessment timing, or patient mix changes over time, a richer model or sensitivity analysis may be needed. The calculation is most useful when it is prespecified, communicated as conditional on the model, and considered alongside clinical and operational consequences.

Common Mistakes

• Counting pending outcomes as failures. This substitutes an observed nonresponse for an unknown outcome. Keep pending patients separate until the assessment is complete, or use a prespecified model for their uncertainty.
• Updating only for pending responses. If \(k\) of \(m\) pending patients respond, the posterior is \(\operatorname{Beta}(a+k,b+m-k)\); the \(m-k\) nonresponses also update the posterior.
• Using the wrong predictive distribution. The pending count is beta-binomial after integrating over uncertainty in \(p\), not simply binomial at a plug-in estimate.
• Calling predictive reversal risk a type I error. It is the probability of a specified future decision change under the current posterior predictive model, not a repeated-sampling error rate under a null.