Why Cohort Size Changes the Rule
In rule-based escalation, a cohort is not just a convenient batch of participants. Its size determines how many observed dose-limiting toxicities (DLTs) trigger expansion or prevent escalation. A change from cohorts of three to cohorts of four therefore changes both the number of participants treated before a decision and the evidence required by the rule.
The conventional 3+3 algorithm treats three participants at a dose first. With zero DLTs it escalates; with one it treats three more at that dose; with two or more it does not escalate. After expansion, escalation is permitted if there is still no more than one DLT among all six. This tutorial compares that procedure with prespecified rules that use other fixed cohort sizes. The comparison concerns how the rule allocates participants and makes decisions; it does not establish that one cohort size is universally preferable.
Under this definition, the initial decision is based on a count in the first cohort, and the expanded decision is based on the total count across both cohorts. The rule is path-dependent: the same total number of DLTs can lead to different actions depending on whether the first cohort had zero, one, or at least two. For example, zero DLTs among the first cohort leads directly to escalation, while one DLT in the first cohort leads to expansion before a decision.
What Changes When \(m\) Changes?
For an \(m+m\) rule, the initial escalation condition is zero DLTs among \(m\). Exactly one DLT triggers expansion. After expansion, escalation is allowed with zero or one DLT among \(2m\). Thus, although the final acceptable count remains one in this family of rules, its proportion of the full expanded sample changes with \(m\): it is \(1/6\) for 3+3 and \(1/8\) for 4+4.
| Rule | Initial cohort | Initial action with exactly one DLT | Expanded total permitting escalation | Maximum enrollment at the dose |
|---|---|---|---|---|
| 3+3 | 3 | Expand by 3 | No more than 1 of 6 | 6 |
| 4+4 | 4 | Expand by 4 | No more than 1 of 8 | 8 |
| \(m+m\) | \(m\) | Expand by \(m\) | No more than 1 of \(2m\) | \(2m\) |
A lower allowable final DLT proportion does not by itself make the larger-cohort rule “stricter” in every possible sense. The rule also changes the probability that the first cohort has zero or exactly one DLT, and therefore changes how often it escalates immediately, expands, or stops. Moreover, decisions depend on the sequence of cohort outcomes, not only on a pooled proportion.
Worked Example: Trace 3+3 and 4+4 Decisions
Suppose a hypothetical dose is being considered for escalation. For the 3+3 rule, one DLT occurs among the first three participants. The rule therefore enrolls three more. Suppose the second cohort has zero DLTs. The expanded total is one DLT among six, so the rule permits escalation.
For a separate hypothetical application of the 4+4 rule, suppose one DLT occurs among the first four participants. The rule enrolls four more at the same dose. If the second cohort has one DLT, the total is two among eight, so the rule does not permit escalation.
These are two specified outcome paths, not a claim that a total of one DLT among four would determine what the 3+3 rule would do. In particular, for a 3+3 trial, one DLT among four participants is not a valid description of the initial cohort: if the fourth participant has the DLT, the first three had zero and the rule would already have escalated. Always identify which cohort produced each outcome before applying the rule.
Probability of Escalation Under a Simple Model
To quantify how cohort size affects decisions, suppose that at a particular dose each participant independently experiences a DLT with the same probability \(p\). This is a binomial model for outcomes at that dose. It assumes independent outcomes and a constant DLT probability for all participants treated at that dose, and it assumes that every enrolled participant has a fully assessed DLT outcome before the next decision. This model is a calculation aid; it does not mean that the rule estimates \(p\) or selects a dose by fitting a statistical model.
Let \(X\) be the number of DLTs in the first cohort. Escalation occurs immediately if \(X=0\). If \(X=1\), escalation occurs only if the second cohort has zero DLTs. The probability of the second cohort having zero DLTs is \((1-p)^m\), independently of the first cohort under the stated model. Therefore:
The first term is the probability of a DLT-free initial cohort. The second term is the probability of exactly one initial DLT followed by a DLT-free expansion cohort. No other path permits escalation under the defined rule.
Worked Example: Compare Escalation Probabilities at \(p=0.20\)
Suppose the hypothetical true DLT probability at a dose is \(p=0.20\), and outcomes follow the independent binomial model. For 3+3, \(m=3\), so:
Thus, under these assumptions, the 3+3 rule escalates from this dose with probability about 0.709. For 4+4, \(m=4\):
The 4+4 rule escalates with probability about 0.577 under the same hypothetical \(p\). This difference arises from the rules’ different initial and expansion sample sizes; it is not a general claim that a 4+4 rule always lowers escalation probability for every value of \(p\).
Expected Enrollment at a Dose
A larger cohort size also affects participant allocation, even when the rule reaches the same decision. Under the same binomial model, every dose opened enrolls \(m\) participants initially. A second cohort is enrolled only when the initial cohort has exactly one DLT. The probability of that event is \(mp(1-p)^{m-1}\). Hence the expected number treated at the dose, conditional on reaching it, is:
This expectation lies between \(m\) and \(2m\). It is not the number that will be enrolled in any particular trial: an individual dose receives either \(m\) participants or \(2m\), depending on the initial cohort’s DLT count.
Worked Example: Expected Participants at a Dose
Continue with the hypothetical \(p=0.20\). For 3+3, the probability of exactly one DLT in the first three is \(3(0.20)(0.80)^2=0.384\). The expected enrollment at that dose is:
For 4+4, the probability of exactly one DLT in the first four is \(4(0.20)(0.80)^3=0.4096\), giving:
These are expected enrollments conditional on opening the dose. They show that, at this value of \(p\), 4+4 uses more participants per opened dose on average. The expected total enrollment across a full trial also depends on how often each dose is reached and where escalation stops.
Following Allocation Across Dose Levels
The consequences of a cohort-size choice accumulate across dose levels. Consider a hypothetical three-dose trial using 3+3. At dose 1, suppose the first three participants have zero DLTs, so the rule escalates after treating three. At dose 2, suppose one DLT occurs among the first three and none among the next three; the rule escalates after treating six at dose 2. At dose 3, suppose two DLTs occur in the first three, so escalation stops after three participants at that dose. The resulting allocation is \(3+6+3=12\) participants.
A separate hypothetical 4+4 path might have zero of four DLTs at dose 1, one of four followed by zero of four at dose 2, and two of four at dose 3. The allocation is \(4+8+4=16\) participants, with no escalation beyond dose 3. The total is larger because the prescribed cohorts are larger; the two paths should not be interpreted as identical participant-level outcomes or as a controlled comparison of efficacy or safety.
More generally, the expected total enrollment can be expressed as a sum across doses: for each dose, multiply the expected enrollment conditional on reaching that dose by the probability of reaching it. The reach probabilities depend on the dose-specific DLT probabilities and on all earlier decisions. Consequently, comparing only the maximum cohort size or only the final DLT proportion does not describe total trial allocation.
Derivation and Interpretation of the Allocation Formula
The expected-enrollment expression follows directly from the two possible enrollment counts at a dose. Let \(N_m=m\) when the first cohort does not trigger expansion, and \(N_m=2m\) when it has exactly one DLT. The chance of expansion is the binomial probability of exactly one DLT:
Using the expectation of a variable with these two possible values gives:
The formula counts participants, not independent observations of dose safety after the fact. At \(p=0\), expansion never occurs and \(E(N_m)=m\). For intermediate \(p\), expansion can occur; when \(p=1\), the initial cohort has multiple DLTs for \(m\geq2\), so the dose receives only \(m\) participants. These checks agree with the rule.
Diagnostics, Comparisons, and Limitations
When reviewing a cohort-size variant, make the decision mechanics explicit before comparing it with another rule. A useful audit is to list, for each dose, the initial cohort size, the trigger for expansion, the total-count stopping boundary, and the maximum enrollment. Then trace each possible path. Under the \(m+m\) definition here, zero initial DLTs means immediate escalation; exactly one means expansion; two or more means no escalation. After expansion, two or more total DLTs means no escalation.
A common mistake is to treat the expanded-sample boundary as a simple cutoff on the pooled DLT proportion, ignoring the route by which the sample was collected. For example, “one DLT among four” cannot be used to infer a 3+3 action unless it is clear that the DLT occurred among the first three: if the fourth participant is the only one with a DLT, the initial three had zero and escalation would already have occurred. The correction is to record outcomes by cohort and apply the rule in sequence.
It is also important not to claim that two rules reach different decisions at the same observed DLT proportion without checking the possible sample sizes and counts. The expanded 3+3 sample has size six and the expanded 4+4 sample has size eight. Their possible shared proportions include zero, one-half, and one, and those proportions do not yield different expanded decisions under the rule defined here. The rules can produce different decisions across different proportions and paths, but an example must show the actual counts and denominators rather than imply a same-proportion contrast.
The 3+3+3 design is a different prespecified rule, not simply a change in the first cohort size: adding a further cohort after a particular outcome changes the decision sequence as well as enrollment. A model-based dose-finding approach is another broad alternative; it uses a statistical model to relate dose to toxicity rather than relying only on fixed count boundaries. These approaches answer a different design question and are not interchangeable by changing \(m\) alone.
Fixed cohort-size rules have important limitations. Their count thresholds are coarse, their decisions depend strongly on the observed path, and they do not directly target a specified DLT probability. Increasing \(m\) may yield more observations at a dose, but it also exposes more participants before a decision and changes the rule’s operating characteristics. Calculations based on a constant \(p\) are informative only to the extent that independent, fully assessed outcomes and a stable dose-specific risk are reasonable approximations. If outcomes are delayed, missing, or not comparable across participants, the simple calculation does not describe the actual decision process.
Check Your Understanding
Answer each question using the fixed \(m+m\) rule defined above. Solutions follow the questions.
- Under a 4+4 rule, what happens if the first cohort has zero DLTs? What if it has exactly one?
- Under 3+3, the first cohort has one DLT and the second has one. Does the rule permit escalation? Give the count and denominator.
- At a dose with hypothetical \(p=0.10\), what is the probability that a 3+3 rule escalates immediately after its first cohort?
- For a 4+4 rule with hypothetical \(p=0.20\), what is the probability of expanding after the initial cohort?
- Why is “one DLT among four participants” insufficient by itself to determine the 3+3 action?
Solutions. 1. With zero of four, escalate immediately; with exactly one of four, enroll four more at that dose. 2. The total is two of six, so the rule does not permit escalation. 3. Immediate escalation requires zero DLTs, with probability \((1-0.10)^3=0.729\). 4. Expansion requires exactly one DLT in four, with probability \(4(0.20)(0.80)^3=0.4096\). 5. The fourth participant may be outside the initial cohort. If the only DLT is the fourth participant, the first three had zero and the 3+3 rule would already have escalated; the initial three’s outcomes must be known.