Why Pending Outcomes Matter for Dropping
An interim analysis may occur while some participants have not yet reached the endpoint assessment. A rule applied only to resolved outcomes can therefore give a different decision from one that accounts for the pending participants. This matters when dropping an arm is permanent: a decision made before outcomes arrive can prevent the trial from learning whether the arm was promising.
The key distinction is between evidence from observed outcomes and prediction about unobserved outcomes. A complete-case calculation uses only participants whose outcomes are resolved. Predictive handling retains the uncertainty about pending outcomes and averages over the outcomes they could eventually have. Neither approach is automatically appropriate in every trial; the outcome process and the rule must be specified in advance.
For illustration, consider a binary efficacy response. Within one treatment arm, let \(p_a\) be the response probability, and let \(y\) responses and \(m-y\) nonresponses be resolved. Assume conditionally independent Bernoulli outcomes with common probability \(p_a\), and a beta prior \(p_a\sim\operatorname{Beta}(\alpha,\beta)\). The resolved data give the posterior \(p_a\mid\text{data}\sim\operatorname{Beta}(\alpha+y,\beta+m-y)\). If \(r\) participants have pending responses, let \(K\) be the number of those participants who will respond.
Using this model requires that the resolved outcomes are relevant to the pending participants: conditional on \(p_a\), their response outcomes follow the same Bernoulli model. The delay itself must not give unmodelled information about the response. For example, if participants who respond tend to respond sooner, the resolved cases may not be representative of those still pending. In that setting, the simple model below is not justified without an appropriate model for response timing.
Predicting the Pending Outcomes
Given the resolved data, the posterior distribution of \(p_a\) is \(\operatorname{Beta}(A,B)\), where \(A=\alpha+y\) and \(B=\beta+m-y\). Conditional on \(p_a\), \(K\) is binomial with \(r\) trials. Integrating over uncertainty in \(p_a\) gives a beta-binomial predictive distribution:
Here \(B(\cdot,\cdot)\) is the beta function. This distribution is not generally the binomial distribution with \(p_a\) fixed at its posterior mean. Integrating over the uncertain response probability accounts for dependence between future responses induced by shared uncertainty about \(p_a\).
For each possible value of \(K\), the posterior after those outcomes are observed would be \(\operatorname{Beta}(A+K,B+r-K)\). A predictive rule can therefore calculate the probability that the usual posterior criterion will be met after pending outcomes resolve. For instance, if the criterion is posterior futility probability at least \(c\), calculate the criterion for every possible \(K\), then sum \(P(K=k\mid\text{data})\) over the values that satisfy it. A predictive threshold for acting on this probability is a separate design choice and should also be prespecified.
Worked Examples
Worked Example: Complete-Case Futility Versus Predictive Futility
Suppose a hypothetical arm has 0 responses among 6 participants with resolved outcomes, while 4 more participants have pending binary responses. Use a \(\operatorname{Beta}(1,1)\) prior. The prespecified posterior futility criterion is \(P(p_a<0.40\mid\text{data})\geq0.95\). Compare the complete-case decision with the predictive probability that this criterion will hold once the four pending outcomes resolve.
Complete cases. The resolved outcomes give \(p_a\mid\text{data}\sim\operatorname{Beta}(1,7)\). For a \(\operatorname{Beta}(1,b)\) distribution, \(P(p_a<x)=1-(1-x)^b\). Thus:
The complete-case rule meets the 0.95 futility threshold and would drop the arm if applied at this interim.
Predictive calculation. Here \(K\mid\text{data}\) has a beta-binomial distribution with \(r=4\), \(A=1\), and \(B=7\). For example,
If \(K=0\), the completed-data posterior is \(\operatorname{Beta}(1,11)\), giving \(P(p_a<0.40)=1-(0.60)^{11}=0.996372\). If \(K=1\), it is \(\operatorname{Beta}(2,10)\), giving \(1-(0.60)^{11}-11(0.40)(0.60)^{10}=0.969767\). Both meet the futility criterion. If \(K=2\), the posterior is \(\operatorname{Beta}(3,9)\), and the futility probability is \(0.881083\), below the criterion. Larger \(K\) also fails it.
Therefore, the predictive probability that the completed-data futility criterion will be met is:
The complete-case rule says to drop now, while a prespecified predictive rule requiring at least 0.95 probability of eventual futility would not yet drop. The difference is not a contradiction: the rules answer different questions.
Worked Example: Predicting Whether Pending Responses Can Rescue an Arm
Suppose a hypothetical arm has 4 responses and 4 nonresponses resolved, with 2 additional responses pending. Use a \(\operatorname{Beta}(1,1)\) prior. Define a completed-data success criterion as \(P(p_a>0.50\mid\text{data})\geq0.70\). The prespecified predictive rule drops the arm if the probability that this criterion will be met after the two pending outcomes resolve is below 0.30.
The current posterior is \(\operatorname{Beta}(5,5)\), so \(P(p_a>0.50\mid\text{data})=0.50\). The current posterior alone does not meet the completed-data success criterion. The pending count has a beta-binomial distribution with \(r=2\) and \(A=B=5\):
For \(K=0,1,2\), the completed-data posteriors are respectively \(\operatorname{Beta}(5,7)\), \(\operatorname{Beta}(6,6)\), and \(\operatorname{Beta}(7,5)\). Their probabilities of \(p_a>0.50\) are \(0.274414\), \(0.500000\), and \(0.725586\). These values follow from the integer-beta/binomial identity: for \(X\sim\operatorname{Beta}(a,b)\), \(P(X>x)=P\{\operatorname{Binomial}(a+b-1,x)\leq a-1\}\). Only \(K=2\) meets the 0.70 criterion.
Consequently, the predictive probability of meeting the success criterion is \(P(K=2)=3/11=0.272727\). Since this is below the prespecified 0.30 threshold, the predictive rule drops the arm. This conclusion is about the chance of crossing the specified posterior threshold after these two outcomes resolve; it is not a claim that the arm's response probability is known to be low.
Worked Example: How Much Information Is in a Large Pending Group?
In a hypothetical arm, 4 responses and 1 nonresponse are resolved, and 10 participants have pending responses. With a \(\operatorname{Beta}(1,1)\) prior, the posterior for \(p_a\) is \(\operatorname{Beta}(5,2)\). The pending response count has a beta-binomial predictive distribution. Its mean and variance are:
The predictive standard deviation is \(\sqrt{4.337}=2.082\) responses. Thus, the expected number of responses among the pending group is about 7.1, but there is substantial uncertainty around that expectation. Replacing the 10 pending responses with exactly 7.1, or treating them as 7 successes and 3 failures, would discard that uncertainty and can change a threshold-based decision. The predictive distribution, rather than its mean alone, is what a predictive dropping rule needs.
Derivation and Prespecification
The beta-binomial formula follows directly from conditional probability. Conditional on \(p_a\), the probability of \(k\) responses among \(r\) pending participants is \(\binom{r}{k}p_a^k(1-p_a)^{r-k}\). The posterior for \(p_a\) after resolved outcomes is \(\operatorname{Beta}(A,B)\). Integrating the conditional probability over that posterior gives:
The last equality is the defining beta integral. This derivation also shows why predictive handling is model-dependent: the distribution used for pending outcomes is the posterior predictive distribution implied by the chosen response model and prior.
Before enrollment, a protocol or statistical analysis plan should specify the interim timing, the outcome model, how pending participants enter the prediction, the clinical margin or posterior criterion, and the threshold for acting on the predictive probability. The rule should also state what happens if outcomes remain pending beyond the expected resolution period. Choosing these elements after seeing the interim data risks making the decision depend on the particular pending pattern.
Diagnostics, Alternatives, and Limitations
Compare complete-case and predictive results as a diagnostic, not as a search for the more favorable answer. A large difference indicates that pending outcomes are decision-relevant. Check the number pending by arm, their follow-up time, endpoint ascertainment, and whether resolved and pending participants differ in prognostic characteristics. Examine whether the delay mechanism could be associated with response. If pending status is informative, a simple beta-binomial prediction may be systematically wrong; model response time and response status jointly, or use a justified model conditional on relevant predictors.
| Approach | What it uses | Main strength | Main concern |
|---|---|---|---|
| Complete-case rule | Resolved outcomes only | Simple and transparent | Can be unstable or unrepresentative when few outcomes are resolved or delay is informative |
| Predictive handling | Resolved outcomes plus a model for pending outcomes | Quantifies the chance pending outcomes will change the decision | Depends on the outcome and delay assumptions and prior specification |
| Wait for resolution | Waits before applying the rule | Avoids predicting these pending outcomes | Can delay a decision and may be impractical when resolution is slow |
A complete-case rule is not automatically biased merely because some outcomes are pending. Under a correctly specified exchangeable response model, the resolved data can support valid posterior inference about \(p_a\). But the rule does not directly represent the information expected from the pending group, and the current evidence can be imprecise. Conversely, predictive handling is not automatically more reliable: if response delays depend on outcome, or the model poorly describes variation across participants, its calculated predictive probability may be misleading.
These methods address uncertainty from pending outcomes, not every issue in an adaptive trial. They do not by themselves guarantee appropriate operating characteristics across repeated interim looks, account for unplanned changes to the rule, or solve problems caused by informative dropout. Those properties require evaluation of the complete prespecified design under plausible scenarios.
Common Pitfall
Wrong reasoning: “The resolved cases have a high posterior probability of futility, so the pending participants can be treated as failures too.” This converts uncertainty about future outcomes into observed evidence and overstates what the data show.
Correction: Keep the observed-data posterior distinct from the predictive distribution of pending outcomes. If the decision is meant to anticipate resolution, calculate the probability of the prespecified completed-data criterion over all possible pending outcomes. If the outcome-delay assumptions are implausible, revise the model or wait for more information rather than treating predictions as facts.
Check Your Understanding
Answer each question, then compare with the solutions below.
- Under a \(\operatorname{Beta}(1,1)\) prior, an arm has 0 responses among 4 resolved participants and 2 pending participants. What is the posterior after the resolved data, and what is the predictive probability that both pending participants respond?
- Why is replacing pending outcomes by their expected number of responses not equivalent to predictive handling in a threshold rule?
- A complete-case futility probability exceeds its threshold, but the predictive probability of meeting that criterion after pending outcomes resolve is below the prespecified action threshold. What is the correct interpretation?
- Name one circumstance in which a beta-binomial prediction for pending responses may be unreliable, and state a suitable response.
Solutions. (1) The resolved data give \(\operatorname{Beta}(1,5)\). The probability of two responses is \(E(p_a^2\mid\text{data})=B(3,5)/B(1,5)=(1\cdot2)/(6\cdot7)=1/21=0.04762\). (2) A threshold rule depends on the distribution of possible completed data, not only its average. Substituting an expected count removes predictive uncertainty and can put the result on the wrong side of the threshold. (3) The observed evidence meets the complete-case criterion, but the predictive rule says the probability of meeting it after pending outcomes resolve is too low for the prespecified action. These are different criteria; follow the rule chosen in advance and report which criterion drove the decision. (4) If response time depends on response status, the pending group may not be exchangeable with resolved participants. Use a justified model that accounts for response timing, or wait for additional outcomes rather than relying on the simple beta-binomial prediction.