Why Calculate the Escalation Probability?
The 3+3 design makes decisions from small cohorts: after three participants are evaluated at a dose, the observed number of dose-limiting toxicities (DLTs) determines whether to escalate, treat three more participants, or stop escalation. A dose with a particular underlying DLT probability can therefore lead to different decisions in different trials. The escalation probability describes how often the 3+3 rule would permit escalation at that dose under a specified probability model.
This is a probability of a rule-based decision at one dose, conditional on reaching and evaluating that dose. It is not the probability that the trial will reach the dose, nor does it by itself describe which dose will ultimately be selected. Those questions require following decisions across dose levels.
Model and 3+3 Decision Pathways
The model has important conditions. The Bernoulli outcome is defined for the same DLT window for every participant. The common \(p\) assumption means the participants at this dose have the same DLT probability; independence means one participant’s outcome does not change another’s probability. Complete follow-up ensures that each outcome can be classified before the rule is applied. These assumptions allow probabilities for separate participants to be multiplied.
There are exactly two disjoint ways to pass the escalation rule. The first is to observe no DLT among the initial three. The second is to observe exactly one DLT among the initial three, followed by no DLTs among the additional three. The second pathway requires the expansion cohort; it is not enough to have exactly one DLT in the first three.
Deriving the Escalation Probability
Let \(E\) denote the event that the 3+3 rule permits escalation from this dose. Let \(A\) denote the event of no DLTs among the first three participants. Let \(B\) denote the event of exactly one DLT among the first three, and \(C\) the event of no DLTs among the additional three participants, when expansion occurs. The escalation event is \(E=A\cup(B\cap C)\). The two parts are mutually exclusive because \(A\) and \(B\) cannot both occur.
By independence, the probability of no DLTs in the initial cohort is \(q^3\). Exactly one DLT in that cohort can occur in any of three participant positions, with probability \(p q^2\) for each position, so \(P(B)=3p q^2\). The additional cohort’s three outcomes are independent of the first cohort, and all three must be DLT-free, giving \(P(C\mid B)=q^3\). Thus:
This calculation follows the protocol’s branching rule. It does not assume that six participants are always treated. If the first three have no DLTs, the rule permits escalation immediately; if they have at least two DLTs, the rule does not send the dose into the expansion pathway.
Worked Example: A Dose With a 10% DLT Probability
Suppose, hypothetically, that the DLT probability at a dose is \(p=0.10\), so \(q=0.90\). Find the probability that this dose passes the 3+3 rules for escalation.
The probability of zero DLTs among the first three is:
The probability of exactly one DLT among the first three is \(3p q^2\). Given that outcome, the probability of zero DLTs in the additional three is \(q^3\). The expansion pathway therefore has probability:
Adding the disjoint pathways gives:
Under this hypothetical model, a dose with a 10% DLT probability has about a 90.6% probability of passing the escalation rule, conditional on reaching and evaluating the dose. The calculation is about the rule’s decision, not a guarantee that escalation is clinically appropriate.
Worked Example: A Dose With a 25% DLT Probability
Suppose, hypothetically, that \(p=0.25\), so \(q=0.75\). Calculate each decision pathway separately to check how the expansion rule contributes.
The immediate-escalation pathway requires zero DLTs in the first three:
The chance of exactly one DLT in those three is:
Conditional on expansion, all three additional participants must have no DLT:
Consequently, the expansion pathway contributes \(0.421875\times0.421875=0.1779785\), and the total is:
Thus the hypothetical probability of passing the escalation rule is about 60.0%. The expansion pathway is multiplied by the probability of getting exactly one DLT initially because it occurs only when that initial result triggers expansion.
Worked Example: A Dose With a 40% DLT Probability
Suppose, hypothetically, that \(p=0.40\), so \(q=0.60\). The probability of zero DLTs in the initial cohort is:
For the expansion pathway, the probability of exactly one DLT among the first three is \(3(0.40)(0.60)^2=0.432\). The probability of zero DLTs among the additional three is \((0.60)^3=0.216\). Their product is \(0.432\times0.216=0.093312\). Therefore:
Under this hypothetical model, the dose passes the escalation rule with probability about 30.9%, conditional on reaching and evaluating it. The result is lower than in the first example, as expected: when the DLT probability is higher, DLT-free outcomes required for escalation are less likely.
Checks, Comparisons, and Limitations
Simple checks help detect errors in the formula or its implementation. At \(p=0\), every participant is DLT-free, so \(P(E)=1\). At \(p=1\), the first three all experience DLTs, so \(P(E)=0\). The expression \(q^3+3p q^5\) gives both results. Also, the expansion term must contain \(q^5\): after exactly one DLT among the first three, there are two DLT-free outcomes in that initial cohort and three more required in the expansion cohort.
The rule is more specific than a fixed six-participant cutoff. For example, a tempting alternative is to calculate the probability of at most one DLT in six participants, \(q^6+6p q^5\). That expression describes a different procedure in which six outcomes are considered together. The 3+3 rule does not always treat six participants: it escalates immediately after 0/3, and does not expand after at least 2/3. Its escalation probability is therefore \(q^3+3p q^5\), not the fixed-six expression.
A second useful comparison is a rule that escalates only after 0 DLTs in three participants. Its probability is \(q^3\). The 3+3 rule has the additional nonnegative expansion term \(3p q^5\), because it can still permit escalation after one initial DLT if the next three participants have no DLTs. These are different decision rules, even though both begin with three participants.
The formula is a model-based probability, not a calculation from a particular trial’s observed DLT count. Its interpretation relies on the specified \(p\), independence, a common probability at the dose, complete follow-up, and the stated 3+3 decision pathway. If outcomes are correlated, participants have different DLT probabilities, observation windows differ, or outcomes are unresolved when a decision is made, the formula may not describe the trial’s actual decision process. In particular, incomplete follow-up cannot simply be treated as absence of a DLT.
A common mistake is to say, “The dose passes if there is at most one DLT among six, so I will use a binomial probability with \(n=6\).” That reasoning ignores the adaptive cohort rule: the number treated depends on outcomes in the first three. The correction is to partition the possible paths first—0/3, or 1/3 followed by 0/3—and calculate the probability of each permitted path.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Under the stated model, what two disjoint outcome pathways permit escalation from a dose?
- If \(p=0.20\), calculate the probability of escalation using the derived formula.
- Why is the probability of exactly one DLT in the initial cohort \(3p q^2\), rather than \(p q^2\)?
- For \(p=0.50\), calculate the immediate-escalation probability and the expansion-pathway probability separately.
- Explain why the probability of at most one DLT in six participants is not the escalation probability for the 3+3 rule.
Solutions. (1) Escalation occurs either after 0 DLTs among the first three, or after exactly 1 DLT among the first three followed by 0 DLTs among the additional three. (2) Here \(q=0.80\), so \(P(E)=(0.80)^3+3(0.20)(0.80)^5=0.512+0.196608=0.708608\), or about 70.9%. (3) The single DLT can be in any of three positions; the three mutually exclusive arrangements each have probability \(p q^2\), giving \(3p q^2\). (4) With \(p=q=0.50\), the immediate pathway is \(q^3=0.125\), and the expansion pathway is \(3p q^5=3(0.50)(0.50)^5=0.046875\). (5) The fixed-six calculation assumes six outcomes are assessed as one group, but the 3+3 rule can escalate after 0/3 without treating six and does not expand after at least 2/3. It must be calculated from the rule’s actual branching pathways.