Why Evaluability Matters in a 3+3 Trial
The 3+3 rule makes dose decisions from the number of dose-limiting toxicities (DLTs) among a specified number of participants whose DLT outcomes can be assessed. A participant who leaves the study before adequate DLT observation, without having a protocol-defined DLT, may not provide an outcome for that decision. Counting that participant as having no DLT would make the dose appear safer than the available observation supports.
A prespecified replacement rule addresses this problem by allowing another participant to fill the unevaluable participant’s slot at the same dose. It does not change the DLT outcome of anyone already assessed, and it does not make an incomplete observation equivalent to a non-DLT. The key distinction is between incomplete observation without a DLT outcome and a toxicity outcome that the protocol counts as a DLT, even if toxicity leads to withdrawal before the assessment period ends.
These definitions depend on the trial protocol. For example, a participant may be evaluable if a DLT occurs before the end of the assessment period, because the DLT outcome is already known. Conversely, stopping treatment early for a non-toxicity reason may leave the outcome unknown if the protocol requires observation through the full DLT window. The reason for withdrawal alone does not settle every case: the protocol’s DLT definition and replacement criteria govern the classification.
Replacement and the 3+3 Decision Rule
For a standard 3+3 design, decisions are made using evaluable outcomes at a dose. With zero DLTs among three evaluable participants, the design escalates. With one DLT among three, it adds three evaluable participants at that dose. If no more than one DLT is observed among six evaluable participants, the design may escalate; with at least two among three or at least two among six, the dose is too toxic under the rule. These rules apply to evaluable participants, not merely to everyone who started treatment.
Let \(m\) be the planned number of evaluable participants for a decision stage, \(E\) the number with an assigned DLT outcome, and \(D\) the number of those outcomes that are DLTs. When \(E=m\), the observed DLT proportion is \(D/E\). If fewer than \(m\) participants have outcomes, the decision stage is not complete. The evaluable proportion of the planned group is \(E/m\); it is not the DLT proportion.
The second expression is undefined when \(E=0\). Neither proportion is a substitute for the protocol’s decision rule: the 3+3 rule uses the actual DLT count and requires the prescribed number of evaluable outcomes. A replacement adds an outcome to the evaluable group; it does not remove or overwrite an outcome already assigned.
Determine whether an outcome can be assigned, including whether a toxicity event already meets the protocol’s DLT definition.
If there is no assigned DLT outcome and the participant has not met the required observation, do not count that participant as a non-DLT merely to complete the group.
Enroll a replacement at the same dose, and assess that participant under the same DLT rules.
Use the DLT count among the required number of evaluable participants to apply the 3+3 rule.
Worked Applications
Worked Example: An Incomplete Assessment With No DLT Outcome
Suppose a hypothetical protocol requires completion of a 28-day DLT assessment unless a DLT occurs earlier. Three participants start at one dose. Participant A completes the assessment with no DLT; participant B completes it and has one DLT; participant C withdraws on day 9 for a documented relocation and has no DLT outcome. The protocol says to replace participants who stop for non-toxicity reasons before completing the assessment.
Among the original three, two have assigned outcomes: one DLT and one no DLT. Thus \(E=2\), \(D=1\), and the planned evaluable group size is \(m=3\).
The evaluable proportion is \(2/3\), while the DLT proportion among the two participants with assigned outcomes is \(1/2\). The 3+3 decision is not based on a completed three-person evaluable group. Under the stated rule, participant C is replaced at the same dose. If the replacement completes assessment without a DLT, the evaluable group has three outcomes and \(D=1\). The result is one DLT among three evaluable participants, so the dose is expanded to six under the standard 3+3 rule; it is not treated as zero DLTs among three.
Worked Example: Withdrawal After a Protocol-Defined DLT
In a hypothetical trial, three participants begin a dose. One develops an event that meets the protocol’s DLT definition on day 11 and stops treatment because of that event. The other two complete the assessment period without DLTs. The protocol counts a qualifying DLT as an assigned outcome even when it causes early treatment discontinuation.
All three participants have assigned outcomes: the participant with the qualifying event has a DLT outcome, and the other two have non-DLT outcomes. Therefore \(E=3\) and \(D=1\). There is no unevaluable slot to fill simply because the participant with a DLT did not complete the full period.
The standard 3+3 rule calls for three additional evaluable participants at the same dose after one DLT among the first three. Suppose, hypothetically, that all three additional participants complete assessment without a DLT. Then the combined result is \(D=1\) among \(E=6\), or \(1/6=0.167\). Since no more than one of six has a DLT, the rule permits escalation. Replacing the participant who had the DLT would incorrectly erase a known toxicity outcome and is not justified by incomplete follow-up.
Worked Example: Replacement Completes a Three-Person Group
At a hypothetical dose, three participants start assessment. One has a DLT, one completes the window without a DLT, and one withdraws early for a non-toxicity reason with no assigned DLT outcome. The protocol requires replacement in this situation. A replacement is enrolled at that dose and experiences a DLT that meets the protocol definition.
Before replacement, the original group has \(E=2\), \(D=1\), and \(m=3\). Its evaluable proportion is \(2/3\); the observed DLT proportion among assigned outcomes is \(1/2\). After replacement, the completed evaluable group consists of the two original participants with outcomes and the replacement. The unevaluable participant does not enter the DLT denominator.
There are two DLTs among three evaluable participants. Under the standard rule, the dose is too toxic; the result is not treated as one DLT among three, and no additional three-person expansion at this dose is called for by the usual 3+3 rule. The replacement completes the denominator while preserving both observed DLT outcomes.
Why the Replacement Rule Must Be Prespecified
A replacement rule is an operational part of the design, not an ad hoc adjustment made after seeing the dose outcomes. It should specify the assessment window, the conditions that make an outcome evaluable, which forms of early discontinuation trigger replacement, whether a participant with a qualifying DLT is evaluable, and whether enrollment pauses while outcomes are pending. It should also say whether replacement is at the same dose and how replacement participants fit into the defined evaluable group.
The reason for a withdrawal matters, but it does not automatically determine the outcome. A participant who withdraws because of an event that meets the protocol’s DLT definition has a DLT outcome, even if follow-up ends early. A participant who withdraws for an unrelated reason before enough observation has accrued may be unevaluable under the protocol and replaced. If a withdrawal is associated with toxicity but the event’s status under the DLT definition is not yet clear, investigators should apply the protocol’s stated classification process rather than automatically treating the participant as either a DLT or a replacement.
Replacement can also affect trial conduct. It may extend recruitment and delay a dose decision, especially when early withdrawals are frequent. Replacements do not make the enrolled sample size equal to the number used in a 3+3 decision: some enrolled participants may not contribute an evaluable outcome, while replacement participants do. Accurate records should therefore show, separately, participants enrolled, evaluable outcomes, DLT counts, and replacements at each dose.
| Situation under the stated protocol | How it contributes to the decision group | Usual consequence |
|---|---|---|
| Completes DLT assessment with no DLT | One evaluable non-DLT outcome | Count in the evaluable denominator |
| Meets the protocol’s DLT definition, then stops treatment | One evaluable DLT outcome | Count the DLT; do not replace to remove it |
| Stops early for a non-toxicity reason, with no assigned DLT outcome | Not an evaluable outcome if the protocol requires the full window | Replace if the prespecified rule requires it |
| Early withdrawal with unresolved DLT status | Determined by the protocol’s classification rules | Do not make an improvised classification |
Common Mistakes and Diagnostics
- Calling an unevaluable participant a non-DLT. The absence of a recorded DLT is not evidence of a non-DLT when the required assessment was incomplete and no outcome was assigned. Follow the replacement rule.
- Using the wrong denominator. The fraction of the planned group with evaluable outcomes is \(E/m\); the DLT fraction among assigned outcomes is \(D/E\). They answer different questions.
- Replacing a known DLT. A participant with a protocol-defined DLT contributes that DLT outcome even if treatment stops early. Replacing that participant as though no outcome existed would undercount toxicity.
- Making replacement decisions inconsistently. If similar non-toxicity withdrawals are replaced in one cohort but not another, the dose decisions may no longer follow one prespecified rule. Check the protocol and the participant-level records.
- Making a decision before the evaluable group is complete. If a three-person decision requires three evaluable outcomes and only two are available, the decision is pending under a replacement rule; do not apply the completed-cohort rule prematurely.
A useful diagnostic is to reconcile the participant flow at each dose: planned evaluable slots, enrolled participants, assigned outcomes, unevaluable participants, replacements, and final DLT count. If the final denominator cannot be reconstructed from those records, the escalation decision cannot be reliably audited.
Check Your Understanding
For each situation, identify what enters the evaluable group and state the decision consequence supported by the information given.
- A planned group has three participants. Two have assigned outcomes, one DLT and one non-DLT; the third withdraws early for a non-toxicity reason and is unevaluable under the protocol. What are the evaluable proportion and the DLT proportion among assigned outcomes? Is the three-person decision complete?
- One of three participants experiences a protocol-defined DLT and withdraws, while the other two complete the assessment without DLT. Should the participant with the DLT be replaced to complete the three-person group?
- A non-toxicity withdrawal is replaced. The two original assigned outcomes are both non-DLT, and the replacement has a DLT. State \(E\), \(D\), and the 3+3 action.
- Why is it incorrect to count an unevaluable participant as a non-DLT just because no DLT was recorded before withdrawal?
Solutions. (1) \(E=2\), \(m=3\), and \(D=1\). The evaluable proportion is \(2/3=0.667\); the DLT proportion among assigned outcomes is \(1/2=0.500\). The three-person decision is incomplete, so apply the protocol’s replacement rule rather than deciding from three participants who started. (2) No. The qualifying event is an assigned DLT outcome, so \(E=3\) and \(D=1\); the standard rule expands to six evaluable participants. Replacement must not remove that DLT. (3) After replacement, \(E=3\) and \(D=1\). One DLT among three calls for three additional evaluable participants at that dose. (4) Incomplete observation does not establish that the participant would have remained free of DLT through the required assessment period. Treating the participant as a non-DLT would add an outcome that was not observed or assigned under the protocol.