Why Define Looks by Cohort?
In a single-arm trial with a binary response, a predictive-probability rule asks whether the observed data make terminal success sufficiently likely to stop early for efficacy, or sufficiently unlikely to stop for futility. But the answer depends on which outcomes are available at the analysis. A rule that says “look after each cohort” is incomplete unless it also specifies what counts as a cohort, when its outcomes are ready, and what happens if the planned analysis date arrives first.
A cohort-based design schedules assessments after prespecified increments of participants or evaluable outcomes. The schedule can make the monitoring plan operationally clear and prevent ad hoc decisions about when to look. It does not, by itself, determine a predictive probability or guarantee that outcomes are available on schedule. Those details belong in the design specification.
Define the Look on the Right Scale
Let \(N\) be the planned total number of participants with an endpoint available for the terminal analysis. At a look, let \(n\) be the number with an observed binary endpoint and \(y\) the number of responses among them. The remaining number of unobserved endpoints is \(m=N-n\). In the basic beta-binomial monitoring model, responses are conditionally independent Bernoulli outcomes with common response probability \(p\), and the prior is \(p\sim\operatorname{Beta}(a,b)\). The predictive probability of terminal success is the probability, conditional on the data available at the look, that the terminal success criterion will be met.
For example, suppose terminal success means at least \(R\) responses among \(N\) participants. Given \(y\) responses among \(n\) observed endpoints, the posterior for \(p\) is \(\operatorname{Beta}(a+y,b+n-y)\). If \(X\) is the number of responses among the \(m\) unobserved endpoints, then the predictive probability is \(P(y+X\ge R\mid y,n)\). In the beta-binomial model, each possible value of \(X\) has probability
where \(B(\cdot,\cdot)\) is the beta function. Thus the terminal success probability is the sum of these probabilities for values of \(k\) that satisfy \(y+k\ge R\). This calculation is made at each planned look using the data then available; cohort scheduling determines when that calculation is run and which \(y,n\) enter it.
The usual assumptions include a prespecified response definition and terminal criterion, comparable response probabilities across participants, and a suitable prior for \(p\). The basic calculation also treats the observed endpoints as the data available for updating the model and the unobserved endpoints as outcomes yet to be learned. If outcome delays or missingness make that representation unsuitable, the protocol needs an explicit plan; a delayed endpoint must not simply be recoded as a nonresponse.
Specify Enrollment and Analysis Separately
A cohort may mean a batch enrolled together, or it may mean an increment of participants whose endpoints have become evaluable. These are not interchangeable. Enrollment happens at one time; a binary endpoint may only be known after a follow-up period and assessment. A design should say which event triggers the look.
If the planned total \(N\) is divided into fixed increments of size \(c\), a simple schedule is
Here \(n_j\) is the planned cumulative number of evaluable endpoints at look \(j\). If \(N\) is not divisible by \(c\), the last increment is smaller. The formula defines target counts; it does not imply that the outcomes will be available on a particular calendar date. A protocol can instead define an analysis date or a readiness condition, such as “analyze when all endpoints in the next cohort have reached the prespecified assessment window.”
State \(N\), the endpoint definition, and the terminal success criterion, such as at least \(R\) responses.
Specify whether the increment counts enrolled participants or evaluable endpoints, and state the cohort size and any smaller final increment.
Say when outcomes qualify for a look and whether a scheduled assessment is postponed if required endpoints are not ready.
At each authorized look, calculate using the observed \(y\) and \(n\), then apply the prespecified efficacy and futility thresholds.
Worked Example: Planning the Cohort Counts
A hypothetical single-arm study plans \(N=23\) participants with an evaluable response endpoint. The protocol calls for looks after each increment of \(c=5\) evaluable endpoints. List the planned cumulative counts and identify the last increment.
For the regular increments, \(n_j=\min(5j,23)\). Substituting \(j=1,2,3,4,5\) gives \(n_1=\min(5,23)=5\), \(n_2=\min(10,23)=10\), \(n_3=\min(15,23)=15\), \(n_4=\min(20,23)=20\), and \(n_5=\min(25,23)=23\).
The planned looks are therefore after 5, 10, 15, 20, and 23 evaluable endpoints. The first four increments contain 5 endpoints each, and the final increment contains \(23-20=3\) endpoints. The schedule is defined on evaluable endpoints, not on the number enrolled. If a calendar date arrives when only 19 endpoints are ready, the count-based trigger for the look at 20 has not yet been met.
Calculate at a Scheduled Look
The following calculation illustrates how cohort timing supplies the data for a predictive assessment. It uses a small hypothetical design so the predictive probability can be evaluated directly.
Worked Example: Predictive Probability After One Cohort
Suppose \(N=4\), with a planned cohort increment of \(c=2\). The prior is \(p\sim\operatorname{Beta}(1,1)\), and terminal success means at least \(R=2\) responses among the four participants. At the first planned look, both endpoints in the first cohort are available; one participant responded, so \(n=2\) and \(y=1\). Calculate the predictive probability of terminal success.
There are \(m=N-n=4-2=2\) unobserved endpoints. The posterior is \(p\mid y,n\sim\operatorname{Beta}(1+1,1+2-1)=\operatorname{Beta}(2,2)\). Terminal success requires at least one response among the two unobserved endpoints, because one response has already been observed. Thus \(P(\text{terminal success}\mid y,n)=P(X\ge1\mid y=1,n=2)=1-P(X=0\mid y=1,n=2)\).
Using the beta-binomial formula, \(P(X=0\mid y,n)=B(2,4)/B(2,2)\). Since \(B(2,4)=1/20\) and \(B(2,2)=1/6\), this gives \(P(X=0\mid y,n)=(1/20)/(1/6)=6/20=0.30\). Therefore \(P(\text{terminal success}\mid y,n)=1-0.30=0.70\).
If the prespecified efficacy threshold were 0.80 and the futility threshold were 0.10, this look would trigger neither stopping rule: \(0.10<0.70<0.80\). The conclusion is “continue under this rule,” not “terminal success is certain” or “the null hypothesis has a 70% probability.” The value 0.70 is a model-based probability of meeting the defined terminal criterion, conditional on the data and assumptions.
Enrollment Batches, Endpoint Availability, and Analysis Time
When cohorts are enrolled in batches, the protocol should distinguish three clocks: enrollment of a batch, completion of the response-assessment window, and conduct of the analysis. An analysis may be scheduled after each cohort has completed follow-up, or at a fixed calendar date subject to a minimum number of evaluable endpoints. Either can be specified, but the data rule must be clear.
Worked Example: A Calendar Date Does Not Guarantee a Look
A hypothetical trial plans \(N=20\) participants in cohorts of 5. Each participant’s response is assessed 6 weeks after enrollment. The first four cohorts begin enrollment at weeks 0, 4, 8, and 12. The protocol allows a formal analysis only after a cohort’s full response-assessment window is complete, with planned count triggers at 5, 10, 15, and 20 evaluable endpoints.
At week 12, all 15 participants in the first three cohorts have enrolled. But the third cohort began at week 8, so its six-week assessment window does not finish until week 14. If the first two cohorts have completed assessment, only \(2\times5=10\) endpoints are ready. The planned count is \(n=10\), not \(n=15\), and the 15-participant enrollment count does not authorize the look at 15 evaluable endpoints.
Under the stated readiness rule, the next look occurs no earlier than week 14, after the third cohort’s assessments are complete and the data are ready for analysis. The actual analysis date may be later if data cleaning or review is required. This schedule prevents the analyst from silently treating the five unavailable outcomes as failures or as known responses. If the protocol instead intends to use information from participants with pending endpoints, it must specify an appropriate method and decision rule for that situation.
A practical design may pause enrollment while a cohort’s outcomes are collected, or may allow cohorts to overlap. Pausing can make cohort-by-cohort decisions easier to interpret, but may lengthen the study. Overlapping accrual can be faster, but at an analysis time some enrolled participants may not yet have a known endpoint. The cohort definition, analysis population, and handling of those participants must then be aligned; an enrollment milestone alone does not resolve the modeling question.
Justification for Using the Observed Count
The reason the look is indexed by available outcomes is that the predictive probability conditions on the data observed by the time of analysis. Under the model, after observing \(y\) responses and \(n-y\) nonresponses, the likelihood for \(p\) is proportional to \(p^y(1-p)^{n-y}\). Multiplying by a \(\operatorname{Beta}(a,b)\) prior gives a posterior proportional to
This is the kernel of a \(\operatorname{Beta}(a+y,b+n-y)\) distribution. Therefore both the timing of the look and the definition of “observed endpoint” matter: they determine \(n\) and \(y\), which determine the posterior and predictive distribution. Counting a participant as evaluable before the endpoint is known changes the data supplied to this calculation and is not justified by the cohort schedule alone.
Alternatives, Diagnostics, and Limitations
A cohort-count schedule is one way to organize monitoring, not the only one. Calendar-time looks run at prespecified dates; they can be operationally convenient but may have different numbers of evaluable endpoints at each date. Rolling individual looks assess after each endpoint becomes available; they avoid waiting for a full batch but may create more frequent decision opportunities and greater operational burden. Cohort-based looks sit between these approaches: they limit assessment frequency while anchoring decisions to prespecified information increments.
| Schedule | Trigger | Main operational consideration |
|---|---|---|
| Cohort-based | A specified increment of evaluable endpoints, or a defined completed cohort | Requires a clear cohort definition and readiness rule |
| Calendar-time | A prespecified date or time since trial start | The evaluable sample size can vary with accrual and follow-up |
| Rolling individual | Each new endpoint becomes available | Can lead to frequent assessments and decisions |
Before opening a trial, check that each scheduled count is reachable, that the final increment is defined, and that the terminal analysis is not accidentally omitted or duplicated. During monitoring, verify that the analysis dataset contains exactly the endpoints allowed by the trigger, that the observed count agrees with the schedule, and that response definitions and follow-up windows have not changed. If outcomes are delayed, do not substitute enrollment totals for evaluable outcomes or treat missing information as a nonresponse without a prespecified basis.
A cohort plan does not remove the assumptions of the response model or the dependence of conclusions on the prior, terminal criterion, and stopping thresholds. A common mistake is to say, “We enrolled the next five participants, so we can look,” even though their responses have not been assessed. The correction is to apply the trigger as written: if it is based on evaluable endpoints, wait until the required endpoints are available; if it is based on a calendar date, use only the data and method the protocol specified for that date.
Check Your Understanding
Answer each question using the definitions and rules above. Brief solutions follow the questions.
- A trial plans \(N=18\) and looks after each 4 evaluable endpoints. What are the cumulative planned look counts, including the final increment?
- At a scheduled look, \(N=30\) and 20 endpoints are evaluable. How many endpoints remain unobserved in the predictive calculation?
- A cohort of 6 has enrolled, but only 4 participants have completed the response assessment. If the look is triggered by cohorts of 6 evaluable endpoints, is the look ready?
- In the worked predictive example, why was the terminal success probability calculated as \(P(X\ge1)\), rather than \(P(X\ge2)\)?
- Name one advantage and one operational limitation of cohort-based looks compared with rolling individual looks.
Solutions. (1) The counts are 4, 8, 12, 16, and 18; the last increment contains \(18-16=2\) endpoints. (2) \(m=N-n=30-20=10\) endpoints remain unobserved. (3) No. The trigger requires 6 evaluable endpoints, and only 4 are available; enrollment of the cohort is not completion of its outcomes. (4) One response was already observed and terminal success required at least 2 total, so at least one of the two unobserved endpoints must be a response. (5) Cohort-based looks can reduce the frequency of analyses, but they may require waiting for the remaining endpoints in a batch before assessing the rule.