Why Cohort Boundaries Matter
In a delayed-outcome trial, enrollment can continue before every participant’s response is known. A response-adaptive rule must therefore specify not only how responses influence allocation, but also when the available responses are used. With cohort-based allocation, the trial makes one update at a cohort boundary and holds the resulting probabilities fixed for assignments in the next cohort. Outcomes can resolve at different times within and across cohorts; a pending outcome is not yet an observed response for this update.
This tutorial traces a particular, transparent rule for two treatments, A and B, and a binary outcome coded as success or failure. The rule is an example for learning how to audit updates, not a claim that this allocation rule is optimal. Its central bookkeeping principle applies more broadly: identify the information available at the decision time, calculate the new probabilities from that information, and record which probabilities governed each cohort.
A Resolved-Outcome Allocation Rule
Let \(\theta_A\) and \(\theta_B\) denote the success probabilities for the two treatments. For this worked rule, assume binary responses are generated with a stable success probability within each treatment, and use independent \(\operatorname{Beta}(1,1)\) starting distributions for \(\theta_A\) and \(\theta_B\). At a cohort boundary, use only resolved outcomes observed by that boundary. For treatment \(j\), let \(s_j\) and \(f_j\) be the numbers of resolved successes and failures, respectively.
The added 1 in the numerator and 2 in the denominator are the contributions of the \(\operatorname{Beta}(1,1)\) starting distribution. In particular, when there are no resolved responses in either arm, both estimates are \(1/2\), so both treatments receive assignment probability \(1/2\). The allocation is randomized: \(q_B\) is the probability of assignment to B for each participant assigned under this rule, not a guarantee that exactly \(q_B\) of a cohort will receive B.
The interpretation of \(m_j\) as an estimate of the treatment’s success probability depends on assumptions. Responses must be correctly classified as successes or failures when observed, and the stable treatment-specific success probabilities must be a reasonable working model across enrollment time. For resolved outcomes to represent the enrolled participants in an arm, their resolution must not selectively depend on response in a way that distorts the observed mix. If resolution is informative, the simple resolved-only estimate may be misleading. This rule does not predict pending responses: it excludes them from the counts until they are observed.
Why the Update Formula Has This Form
The calculation follows directly from the binary-response model and the stated starting distribution. For one treatment, write \(\theta\) for its success probability. With \(s\) resolved successes and \(f\) resolved failures, the likelihood is proportional to \(\theta^s(1-\theta)^f\). Multiplying by the \(\operatorname{Beta}(1,1)\) density, which is constant on \(0<\theta<1\), gives a density proportional to
This is the kernel of a \(\operatorname{Beta}(s+1,f+1)\) distribution. The mean of a \(\operatorname{Beta}(a,b)\) distribution is \(a/(a+b)\), so the mean here is \((s+1)/(s+f+2)\), the stated \(m_j\). The rule then converts the two positive estimates into probabilities summing to 1 by dividing each estimate by their sum. This gives \(q_B=m_B/(m_A+m_B)\) and \(q_A=m_A/(m_A+m_B)\).
The derivation explains the smoothing, but it does not make the allocation rule universally appropriate. The rule uses the current estimates to tilt assignment toward the arm with the larger estimate; another trial objective or response model could call for a different rule. Whatever rule is chosen, its timing and the information used at each update need to be explicit.
Worked Example: Tracing Three Enrollment Cohorts
Worked Example: Tracing Three Enrollment Cohorts
Suppose the first cohort contains four participants. With no resolved outcomes at the initial decision, \(m_A=m_B=1/2\), so \(q_B=(1/2)/(1/2+1/2)=1/2\), and \(q_A=1/2\). A possible randomized assignment is A, B, A, B. Their outcomes resolve at different times: the first A participant succeeds on day 5, the first B participant fails on day 7, the second A participant fails on day 12, and the second B participant succeeds on day 15.
The second cohort starts on day 8, after only the day-5 and day-7 outcomes have resolved. For A, \(s_A=1\) and \(f_A=0\), so \(m_A=(1+1)/(1+0+2)=2/3\). For B, \(s_B=0\) and \(f_B=1\), so \(m_B=(0+1)/(0+1+2)=1/3\). Therefore
The second cohort uses these probabilities for all its assignments. Suppose it contains three participants, assigned A, B, A. Their outcomes resolve on days 10, 14, and 20, respectively: the day-10 A response is a success, the day-14 B response is a success, and the final A response is still pending on day 16.
The third cohort starts on day 16. By then, both remaining responses from cohort 1 have resolved, as have the day-10 and day-14 responses from cohort 2. For A, the resolved outcomes are two successes and one failure, so \(m_A=(2+1)/(2+1+2)=3/5=0.6000\). For B, they are one success and one failure, so \(m_B=(1+1)/(1+1+2)=1/2=0.5000\). Thus
The cohort-2 A outcome still pending on day 16 is excluded. It is neither a success nor a failure in the day-16 counts. A possible assignment sequence in cohort 3 is B, A, B, A, but the probabilities do not require that the realized fraction assigned to B equal \(5/11\).
Worked Example: Checking an Update at a Cohort Boundary
Worked Example: Checking an Update at a Cohort Boundary
At a later boundary, suppose the resolved records show four successes and two failures for A, and two successes and four failures for B. Any unresolved responses in either arm are not included. The estimates are
The next cohort’s assignment probabilities are therefore
If the cohort has eight participants, the expected number assigned to B is \(8(0.3750)=3\). This is an expectation over the randomization, not a required count: the realized number may be smaller or larger. The calculation also shows why the denominator for each response estimate must use resolved successes plus resolved failures, not the total number enrolled in that arm.
Worked Example: A Response Resolves During a Cohort
Worked Example: A Response Resolves During a Cohort
At the start of a cohort, suppose the resolved counts are those from the day-16 boundary in the first example: A has two successes and one failure, while B has one success and one failure. The assignment probabilities are \(q_B=5/11\approx0.4545\) and \(q_A=6/11\approx0.5455\). During this cohort, an additional B response resolves and is a success.
That new response does not change the probabilities for participants assigned later in the same cohort: the probabilities were set at the cohort boundary. At the next boundary, it enters the resolved counts. Now B has two successes and one failure, so \(m_B=(2+1)/(2+1+2)=3/5=0.6000\). A remains at \(m_A=3/5=0.6000\). The following cohort therefore has
This example separates two decisions that are easy to conflate: recording a newly resolved response and applying an allocation update. The response is recorded when it resolves, but the probability update takes effect at the next scheduled boundary.
Auditing the Timeline and Recognizing Failure Modes
A reliable audit follows the information in chronological order. For each cohort, record its start time, the resolved outcomes available then, the resulting probabilities, and the assignments made under those probabilities. Separately record the resolution time and response for each participant. This makes it possible to check that no outcome was used before it was observed and that an update did not silently alter assignments partway through a cohort.
Identify the cohort boundary and list the outcomes resolved by that time. Do not count outcomes that resolve later.
For each treatment, confirm that the resolved successes plus resolved failures equal the number of resolved responses included in the update.
Apply the specified response rule, then verify that \(q_A+q_B=1\) and both probabilities are between 0 and 1.
Record the probabilities used for each assignment. Apply newly resolved outcomes at the next scheduled update, not retrospectively.
The most common bookkeeping error is to use all outcomes from an earlier cohort even though some resolved after the next cohort had already started. That produces probabilities based on information that was not available at the decision time. The reverse error is to leave out an outcome that had resolved by the boundary. Both can be detected by comparing each included response’s resolution time with the boundary time.
A second error is to treat the allocation probability as the observed allocation fraction. Randomization can produce a cohort with an unequal number in each arm even when the probabilities are fixed. A third is to use the total enrolled in an arm as the estimate’s denominator while leaving pending responses out of the numerator; the estimate’s denominator must correspond to the resolved successes and failures. Finally, check whether resolution patterns or enrollment-time changes undermine the working assumptions. If, for example, response-dependent resolution makes observed responses unrepresentative, a resolved-only estimate may not describe the arm’s enrolled participants. The update arithmetic can be correct even when the data feeding it are not suitable for the intended interpretation.
How This Approach Differs From Alternatives
A fixed-allocation design uses the same assignment probabilities throughout, regardless of observed responses. It avoids response-based probability updates, but does not use resolved response information to alter later allocation. Cohort-based adaptation instead uses a prespecified update rule at scheduled boundaries. This makes the update history auditable and prevents mid-cohort changes, at the cost of using information only at those boundaries.
An alternative is to update after each individual response or assignment. That approach has more frequent decision points and can make assignment probabilities change rapidly as information arrives. It is not the method illustrated here: the cohort rule holds probabilities constant within each cohort. Neither frequency of updating nor a larger allocation to an arm should be interpreted on its own as proof that the arm is superior or that an individual participant will benefit. The allocation rule is a design procedure, and its suitability depends on the trial’s aims and assumptions.
Common Mistakes
- Counting pending outcomes as failures or successes. Exclude them from both resolved-outcome counts until they are observed.
- Updating partway through a cohort. Under the rule here, a response resolving mid-cohort affects the next cohort’s probabilities, not the current ones.
- Confusing a probability with a realized count. Assignment probabilities govern randomization; they do not dictate exact cohort proportions.
- Ignoring the timing assumption. If resolved outcomes are a selected subset related to response, the resolved-only estimates may be unrepresentative.
- Comparing treatment rates without checking their denominators. Each estimate uses resolved responses in that arm, not everyone enrolled in it.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- At a boundary, A has one resolved success and one resolved failure, while B has no resolved responses. Under the stated rule, calculate \(m_A\), \(m_B\), \(q_B\), and \(q_A\).
- A response resolves after a cohort begins but before its final participant is assigned. When does it enter the allocation calculation under the cohort rule?
- At a boundary, A has two successes and two failures, and B has one success and three failures. Calculate the next cohort’s probability of assignment to B.
- Why can a cohort with a B assignment probability of 0.30 have an observed B fraction different from 0.30?
Solutions. (1) \(m_A=(1+1)/(1+1+2)=1/2\), and \(m_B=(0+1)/(0+0+2)=1/2\); thus \(q_B=q_A=1/2\). (2) It enters at the next cohort boundary, because the current cohort’s probabilities were fixed when that cohort began. (3) \(m_A=(2+1)/(2+2+2)=1/2\) and \(m_B=(1+1)/(1+3+2)=1/3\), so \(q_B=(1/3)/(1/2+1/3)=2/5=0.4000\). (4) Assignments are randomized with probability 0.30 for B, so the realized number assigned to B varies by chance; the probability is not a quota.