Why Delayed Responses Need an Explicit Policy
In a trial with a binary response measured at a fixed follow-up time, participants enrolled most recently may not yet have reached that time. At an interim review, their outcomes are pending: they are not known to be responses or nonresponses. A review rule must say in advance how pending outcomes affect the analysis and the next allocation decision. Otherwise, identical data could lead to different decisions depending on an improvised choice about what to count.
This tutorial works through one policy: use the outcomes observed by the scheduled review, exclude pending outcomes from the response likelihood, and apply a fixed rule to the resulting posterior means. The policy is suitable only under stated assumptions. In particular, the timing of whether an outcome is pending must not itself provide information about that outcome after accounting for treatment arm and enrollment timing.
Model and Prespecified Allocation Rule
Suppose a trial compares a control arm \(C\) with an experimental arm \(E\). For each arm \(j\), let \(\theta_j\) be the probability of response by the specified assessment time. Conditional on \(\theta_j\), each participant’s binary outcome is modeled as Bernoulli with response probability \(\theta_j\). The model assumes outcomes are exchangeable within each arm, and the arm-specific probabilities have independent uniform priors, \(\theta_j\sim\mathrm{Beta}(1,1)\). The analysis below assumes a common response definition and assessment window, and no outcome-dependent delay in ascertainment.
At a review, let \(r_j\) be the number of observed responses and \(f_j\) the number of observed nonresponses in arm \(j\). Pending participants are recorded separately; they are not included in either count. With the stated prior, conjugacy gives the following update and posterior mean:
For a concrete decision rule, suppose the protocol specifies that the next cohort is randomized \(2:1\) to \(E:C\) if \(m_E-m_C\geq 0.20\), \(1:2\) if \(m_E-m_C\leq-0.20\), and \(1:1\) otherwise. This is an operational rule based on posterior means, not a claim that a threshold has been crossed with a particular posterior probability. The numerical margin and allocation ratios are hypothetical; a real protocol would justify and prespecify them.
Worked Example: First Review With Three Pending Outcomes
Worked Example: First Review With Three Pending Outcomes
Hypothetical trial data. At the first scheduled review, the control arm has 5 observed responses and 5 observed nonresponses. The experimental arm has 5 observed responses and 1 observed nonresponse, while 3 additional experimental-arm participants have not yet reached the response assessment time. Apply the stated policy and allocation rule.
The pending count is recorded as 3, but it does not enter the posterior at this review. In the control arm, the posterior is \(\mathrm{Beta}(1+5,1+5)=\mathrm{Beta}(6,6)\), so \(m_C=6/12=0.5000\). In the experimental arm, the posterior is \(\mathrm{Beta}(1+5,1+1)=\mathrm{Beta}(6,2)\), so \(m_E=6/8=0.7500\).
The posterior-mean difference is \(m_E-m_C=0.7500-0.5000=0.2500\). Since \(0.2500\geq0.20\), the protocol assigns the next cohort in a \(2:1\) ratio favoring the experimental arm. The decision is based on the observed outcomes and the prespecified threshold. It does not assert that the three pending participants responded, nor does it estimate their responses as known facts.
The pending participants can still matter at a later review. If their assessments become available, their recorded outcomes are added to the appropriate response and nonresponse counts at that later review. The earlier decision remains the decision made from the data available at its scheduled update.
Why the Update Works
For one arm, if the observed data contain \(r\) responses and \(f\) nonresponses, the likelihood for \(\theta\), up to a factor not involving \(\theta\), is \(\theta^r(1-\theta)^f\). The \(\mathrm{Beta}(a,b)\) prior has density proportional to \(\theta^{a-1}(1-\theta)^{b-1}\). Multiplying prior and likelihood gives a density proportional to \(\theta^{a+r-1}(1-\theta)^{b+f-1}\), which is the kernel of a \(\mathrm{Beta}(a+r,b+f)\) distribution. With \(a=b=1\), this yields the posterior used above.
Under the policy’s assumption that pending status is independent of the latent response given arm and enrollment timing, the pending-status contribution to the likelihood does not depend on \(\theta\). It therefore does not change the posterior based on the observed outcomes. This is not a general theorem that missing outcomes never matter; it is a consequence of the particular ascertainment assumption and policy.
A useful bookkeeping check is that the posterior parameters sum to \(2+r+f\), while \(r+f\) is exactly the number of observed outcomes. Pending participants are not included in that observed total. If the posterior’s success and failure parameters do not match the corresponding observed counts plus the prior parameters, the update should be checked before applying the allocation rule.
Worked Example: Updating When Pending Outcomes Mature
Worked Example: Updating When Pending Outcomes Mature
Hypothetical continuation. At the next review, the control arm still has 5 responses and 5 nonresponses. In the experimental arm, the original 5 responses and 1 nonresponse remain observed; three more recent participants have now provided outcomes, with 1 response and 2 nonresponses; and the 3 participants previously pending have now each responded. Calculate the updated posterior means and apply the same allocation rule.
The experimental-arm response count is \(5+1+3=9\). The nonresponse count is \(1+2=3\). Thus there are \(9+3=12\) observed experimental outcomes. The posterior is \(\mathrm{Beta}(1+9,1+3)=\mathrm{Beta}(10,4)\), and its mean is \(10/(10+4)=10/14\approx0.7143\).
The control-arm posterior remains \(\mathrm{Beta}(6,6)\), with mean \(6/12=0.5000\). Therefore the posterior-mean difference is \(10/14-6/12=10/14-7/14=3/14\approx0.2143\). Because \(0.2143\geq0.20\), the next cohort is again assigned in a \(2:1\) ratio favoring the experimental arm.
This update accounts for each experimental outcome once: the original six observed outcomes, the three outcomes from the more recent participants, and the three formerly pending responses. The posterior’s parameters sum to \(10+4=14\), which is two more than the 12 observed outcomes because the uniform prior contributes one to each parameter.
Worked Example: What Changes If Pending Outcomes Are Called Failures?
Worked Example: What Changes If Pending Outcomes Are Called Failures?
Return to the first review, with 5 experimental responses, 1 experimental nonresponse, and 3 pending outcomes, alongside 5 control responses and 5 control nonresponses. Consider an alternative rule that codes all three pending experimental outcomes as nonresponses for this review. This is not the observed-outcomes-only policy; it is included to show how a different prespecified rule can change the decision.
Under that alternative, the experimental counts would be \(r_E=5\) and \(f_E=1+3=4\). The posterior would be \(\mathrm{Beta}(6,5)\), with mean \(6/11\approx0.5455\). The control mean remains \(0.5000\), so the difference is \(0.5455-0.5000=0.0455\). The threshold rule would therefore assign the next cohort \(1:1\), rather than \(2:1\).
The arithmetic is internally consistent, but the interpretation is different: coding pending as failures treats outcomes that have not yet been assessed as if nonresponse were already observed. If the delay is merely because the response window has not elapsed, that is not the same as observing a failure. The example shows why the handling rule must be fixed in advance and why its assumptions affect operational decisions.
Operational Checks, Alternatives, and Limits
A review should keep separate counts of observed responses, observed nonresponses, and pending participants for each arm. Before computing a posterior, confirm that the observed total equals responses plus nonresponses and that pending participants are absent from both. Then verify the prior, posterior parameters, posterior means, signed difference, and threshold comparison. These checks catch common count errors, including adding a newly matured outcome twice or inadvertently treating a pending participant as a nonresponse.
Apply the prespecified data cutoff and list observed responses, observed nonresponses, and pending outcomes separately by arm.
Use only the observed response and nonresponse counts for the stated policy; record the resulting posterior parameters and means.
Compare the posterior-mean difference with the prespecified thresholds and document the resulting allocation ratio.
When outcomes mature, classify them once and update the cumulative observed counts according to the same policy.
The main alternative approaches answer different operational questions. Waiting until all participants have completed follow-up avoids decisions based on incomplete current outcomes, but delays adaptation. Using observed outcomes at scheduled reviews permits earlier decisions, but the observed sample may be small and sensitive to the timing of enrollment. Coding pending outcomes as failures is more conservative in some circumstances, but can systematically lower response estimates when pending mainly means “not yet assessed.” A model that explicitly represents time to response or the observation process can use more information, but requires assumptions beyond the simple binary model used here.
Several failure modes deserve attention. The response window must be defined consistently; otherwise, “response” can refer to different follow-up durations across participants. Delays caused by clinical deterioration, treatment discontinuation, or other outcome-related events violate the simple administrative-delay assumption. A low or unequal number of observed outcomes can also make the posterior means unstable, even though the rule is mathematically well-defined. Finally, a posterior-mean threshold can be crossed by a small amount; the rule does not automatically account for uncertainty around the difference or guarantee a particular error rate.
A common mistake is to say, “There are three pending outcomes, so count them as three failures to be cautious.” That changes the stated data model and decision policy: it replaces unknown outcomes with observed nonresponses. Under the policy taught here, the correct update uses only observed responses and nonresponses, keeps pending counts visible, and revisits them when assessments arrive. A protocol may choose another policy, but it must define that policy in advance and justify its assumptions.
Check Your Understanding
Answer each question using the observed-outcomes-only policy and the hypothetical \(0.20\) allocation threshold defined above. Full solutions follow the questions.
- An arm has 4 observed responses, 2 observed nonresponses, and 5 pending outcomes under a \(\mathrm{Beta}(1,1)\) prior. What is its posterior distribution and posterior mean?
- At a review, the control posterior mean is \(0.52\) and the experimental posterior mean is \(0.69\). What allocation ratio does the rule specify, and what comparison determines it?
- At the next review, 2 of the 5 pending outcomes in Question 1 are responses and 1 is a nonresponse; the other 2 remain pending. What are the updated observed counts and posterior?
- Why is treating all pending outcomes as nonresponses not equivalent to excluding them from the likelihood?
Solutions. (1) The pending count is excluded, so the posterior is \(\mathrm{Beta}(1+4,1+2)=\mathrm{Beta}(5,3)\), with mean \(5/8=0.6250\). (2) The difference is \(0.69-0.52=0.17\), which is below \(0.20\) and above \(-0.20\); the allocation is \(1:1\). (3) The updated observed counts are 6 responses and 3 nonresponses; 2 participants remain pending. The posterior is \(\mathrm{Beta}(7,4)\). (4) Exclusion contributes no response or nonresponse count, whereas coding pending as failures adds them to the failure count and changes the posterior as though failures had been observed.