Interpreting 3+3 Operating Characteristics

From Decision Probabilities to Trial Performance

A 3+3 design gives decisions from small cohorts: treat three participants, expand to six in certain cases, and escalate or stop according to the observed dose-limiting toxicities (DLTs). Those rules determine what happens in one trial. Operating characteristics describe what the design would do over many hypothetical repetitions under a specified set of true toxicity probabilities. They turn a sequence of cohort decisions into quantities such as the chance of selecting a particular dose or stopping at the starting dose.

These probabilities are not guaranteed outcomes for a single trial, nor are they probabilities that a dose is safe after observing the trial. They are calculated under an assumed toxicity scenario. Change the true DLT probabilities, and the operating characteristics change—even though the 3+3 rules stay the same.

For interpretation, a dose-selection probability is a long-run frequency: among many hypothetical trials conducted with the same true toxicity probabilities and the same design rules, it is the proportion expected to select that dose. It does not mean that the probability the dose is the true maximum tolerated dose (MTD), given the data from one completed trial, equals that number.

Escalation and Stopping at One Dose

Use the conventional 3+3 rule: escalate after zero DLTs among the first three participants; if exactly one DLT occurs, treat three more at that dose and escalate only if there are no DLTs among those additional three; stop escalation at that dose if at least two DLTs occur among the first three, or if at least one additional DLT occurs in the expansion cohort. Assume for now that each participant’s DLT outcome is fully observed, that outcomes are independent Bernoulli trials at a fixed dose, and that the true DLT probability \(p\) does not change during the trial.

Write \(A(p)\) for the probability of escalating past a dose with DLT probability \(p\). Escalation happens either when the first three have zero DLTs, or when they have exactly one DLT and the next three have zero DLTs. Independence gives the following expression.

The probability of stopping at that dose, rather than escalating past it, is \(1-A(p)\). “Stopping at a dose” and “selecting that dose” are not interchangeable. Under the usual rule, if the trial stops at dose \(i>1\) because that dose is too toxic, the selected dose is the preceding dose, \(i-1\). If the lowest dose is too toxic, there is no lower dose to select, so the outcome is no dose.

Selection Probabilities Along a Dose Ladder

Consider \(J\) ordered dose levels with escalation probabilities \(A_i=A(p_i)\). Assume the design starts at dose 1, proceeds only to the next dose, and selects the dose immediately below the first dose at which escalation fails. If escalation passes the highest dose, that highest dose is selected. The probability of reaching dose \(i\) is the product of escalation probabilities at all preceding doses, because the trial must pass each one in turn.

Consequently, selecting dose \(k<J\) requires passing doses 1 through \(k\), then failing to escalate past dose \(k+1\). Selecting dose \(J\) requires passing every dose. If escalation fails at dose 1, no dose is selected. These mutually exclusive outcomes have probabilities:

The probabilities sum to one: the trial either stops at the lowest dose without selecting a dose, or passes the lowest dose and ultimately selects one dose. More explicitly, for three levels the sum of the selection probabilities is \(A_1\), since the trial must pass dose 1 to select any dose; adding \(1-A_1\) for no dose gives 1. This is a useful arithmetic check on an enumeration.

Interpreting the Categories and the Stop Event

For a prespecified target threshold \(\pi_T\), and an assumed monotone dose-toxicity pattern, define the true MTD among the tested levels as the highest dose \(k^*\) satisfying \(p_{k^*}\leq\pi_T\), if such a dose exists. Then “correct selection” means selecting \(k^*\); “below-target selection” means selecting a dose lower than \(k^*\); and “above-target selection” means selecting a dose higher than \(k^*\). If no tested dose meets the threshold, there is no such \(k^*\), so report selection and stopping probabilities without labeling one dose the correct MTD. The threshold is a definition for evaluating performance, not a quantity the 3+3 algorithm estimates directly.

It is also useful to distinguish a stopping event from a final selection category. In a three-level design, the probability of stopping at the starting dose because it fails the 3+3 rule is \(1-A_1\), and this results in no dose being selected. The probability of stopping at dose 2 is \(A_1(1-A_2)\); under the conventional selection rule this results in selecting dose 1. More generally, stopping at dose \(i>1\) yields selection of dose \(i-1\), not dose \(i\). Therefore the chance of “stopping without escalation” can refer specifically to failure to escalate beyond the starting dose, while “no dose selected” is the corresponding final outcome in this setup.

Why the Scenario Matters

Operating characteristics are conditional on the assumed dose-toxicity scenario. A design can have a high probability of selecting the target dose in one scenario and a low probability in another. The same is true of overdose and underdose probabilities. For that reason, a single set of operating characteristics should not be treated as a universal performance guarantee. Evaluation typically considers several clinically plausible patterns of \(p_i\), including patterns where the target is near the lowest dose, near the middle, near the highest, or absent from the tested ladder.

A calculation should also state the assumptions that make its probabilities meaningful. The formula for \(A(p)\) uses independent Bernoulli outcomes with a common probability at a dose; the ladder calculation assumes the trial visits doses in order and follows the stated escalation and selection rules. In practice, incomplete DLT observation, changing eligibility or dose definitions, or deviations from the algorithm can make these assumptions poor descriptions of the trial. A probability calculated under the idealized rules then may not represent the actual conduct of the study.

A useful comparison is with model-based dose-finding designs, which use an explicit dose-toxicity model to guide assignments and selection. The 3+3 design instead uses a fixed set of cohort rules; its operating characteristics can still be calculated under specified scenarios, but the rule does not adapt by estimating a dose-toxicity curve. This distinction matters when interpreting its performance: a table of selection probabilities evaluates the fixed algorithm under scenarios, rather than showing a model-based estimate of the toxicity probability at each dose.

Diagnostics and Common Pitfalls

• Check the escalation probability. \(A(p)\) must lie between 0 and 1. At \(p=0\), \(A(0)=1\); at \(p=1\), \(A(1)=0\). Values outside this range indicate an arithmetic or formula error.
• Check the outcome accounting. Across all selected doses and the no-dose outcome, probabilities should sum to 1, allowing only for small rounding differences. Missing probability often means a stopping outcome was omitted.
• Do not call every stop an overdose selection. A stop at dose \(i>1\) ordinarily selects dose \(i-1\). A stop at the starting dose produces no selection; neither event should be silently assigned to another category.
• Do not confuse a long-run probability with a posterior probability. For example, \(P(S=2)=0.4764\) means about 47.6% of hypothetical trials select dose 2 under the stated scenario. It is not a 47.6% probability that dose 2 is truly the MTD after seeing one trial.
• Keep the target definition explicit. Correct, above-target, and below-target categories require a stated threshold and a rule for defining the target dose. Without these, dose-selection probabilities remain interpretable, but the labels “correct” and “overdose” may not be.

A common mistake is to interpret “the design stops at dose 2” as “the design selects dose 2.” The correction is to apply the selection convention after identifying the stop: if escalation fails at dose 2, the selected dose is dose 1. Another mistake is to present one probability without its toxicity scenario. The correction is to state the assumed \(p_i\) values and the design assumptions alongside the result.