Why Audit the Decision Trail?
In a rule-based dose-finding trial, a dose assignment is not justified merely because the observed toxicity count seems acceptable. It must also be permitted by the protocol at that point in the study. An audit checks that connection: given the protocol version in force, the outcomes available at each decision time, and the required safety gates, was the action taken allowed?
The audit is a reconstruction of decisions in chronological order, not a retrospective opportunity to choose a more attractive rule. It can identify a wrong assignment, a missed pause, an incorrect cohort count, or an outcome that was not yet available when a decision was made. It can also show which later decisions relied on an earlier departure. An audit does not itself decide how to manage an individual participant; immediate clinical and safety responsibilities remain with the trial's designated oversight processes.
Specify the Rule Before Checking the Record
An audit needs the protocol version and the exact rule set that applied to each decision. The protocol should establish the dose-level order, the cohort and evaluability rules, how DLT outcomes are classified, when outcomes are considered available for a decision, any pause or restart gates, and the actions required for each possible state. An approved amendment may change the applicable rules from its effective date; it does not usually make an earlier action compliant retroactively.
For illustration, the examples below use one conventional 3+3 rule set. At a dose level, treat three evaluable participants first. With 0 DLTs among 3, escalation to the next permitted dose is allowed. With 1 DLT among 3, treat three additional evaluable participants at the same dose. If there is 1 DLT among all 6, escalation is allowed; if there are 2 or more DLTs among the first 3 or all 6, stop escalation and select the next lower dose as the MTD, if a lower dose exists. If the lowest dose meets the stopping condition, this illustrative rule selects no lower dose as the MTD. This is one rule set for the examples, not a universal definition of 3+3.
For this rule, a decision cannot be made from an incomplete cohort. If the protocol requires an unevaluable participant to be replaced, the replacement outcome must be available before that participant slot counts toward a decision. A required safety pause or restart condition is another gate: the dose-count rule does not override it. The protocol's own definitions govern if they differ from these illustrative assumptions.
Build a Chronological, Verifiable Record
A useful audit links participant-level source records to each cohort and decision. For every participant, reconcile assigned dose, cohort membership, treatment and follow-up timing, DLT classification, evaluability, and whether the outcome was available by the decision time. Then check the protocol gates in order. The denominator must represent the participants who count under the protocol, not simply everyone who received a dose or everyone whose record is easiest to find.
A concise audit ledger can record the protocol version, dose level, evaluable DLT count, evidence reviewed, required action, actual action, status, and downstream consequence. Status should distinguish at least three cases: compliant, deviation, and unresolved. “Unresolved” is appropriate when, for example, the DLT assessment is pending or the cohort record conflicts with the source record. It avoids converting uncertainty into an unsupported conclusion.
Identify the protocol and amendment effective at the decision time, including dose order, outcome definitions, replacement requirements, and safety gates.
Use dated records to identify who was assigned to each dose and cohort, whose outcomes were evaluable and available, and what safety conditions were active.
Apply the stated rule to that reconstructed state. If a required input is missing or ambiguous, record the decision as unresolved pending clarification.
Compare the required action with the action actually taken. If they differ, identify the affected assignment and review each subsequent decision that depended on it.
Record the evidence, finding, and consequence; refer unresolved or safety-relevant findings through the trial's established oversight procedures.
Worked Audits
Worked Example: A Compliant Escalation and Stop
Hypothetical record. Under the illustrative 3+3 rule above, three evaluable participants at dose 1 have 0 DLTs. Three evaluable participants at dose 2 have 1 DLT; three additional evaluable participants are then treated at dose 2, with 0 additional DLTs. At dose 3, the first three evaluable participants have 2 DLTs. No pause or restart gate is active.
Check dose 1. The observed count is 0/3. The rule permits escalation to dose 2, and the record shows that dose 2 was assigned next. This decision is compliant.
Check dose 2. The initial count is 1/3, so escalation is not yet allowed; the rule requires three additional evaluable participants at dose 2. The additional cohort has 0 DLTs, giving \(1+0=1\) DLT among \(3+3=6\) evaluable participants. Since \(1/6\) meets the rule's escalation condition, proceeding to dose 3 is compliant.
Check dose 3. The first cohort has 2 DLTs among 3 evaluable participants. Since \(2\geq2\), the rule stops escalation. Under the illustrative selection convention, dose 2 is the next lower dose and is the selected MTD. The audit records the stop as compliant and confirms that no higher dose assignment follows it.
This conclusion depends on the stated evaluability and timing assumptions. If one of the six dose-2 outcomes was not yet available when escalation occurred, the decision would need to be reassessed using the information available at that time.
Worked Example: Escalation Before Required Expansion
Hypothetical record. Dose 1 has 0 DLTs among 3 evaluable participants. Dose 2 has 1 DLT among its first 3 evaluable participants. Before enrolling the required additional three participants at dose 2, the team assigns three participants to dose 3. The additional dose-2 cohort is later completed with 0 DLTs.
At the decision time, dose 2 has \(1/3\) DLTs. The applicable rule requires expansion at dose 2; it does not yet permit escalation. Therefore, the dose-3 assignment is a protocol deviation. The later dose-2 total is \(1+0=1\) DLT among \(3+3=6\), which would permit escalation once known, but it does not change what information was available when dose 3 was assigned.
The audit should identify the three dose-3 assignments as occurring before the protocol gate was satisfied, record when the missing dose-2 outcomes became available, and trace any further assignments that relied on the premature escalation. It should not label the earlier assignment compliant on the strength of later results. The finding should be routed to the trial's designated safety and oversight processes; the audit alone does not determine individual participant management.
Worked Example: An Unevaluable Participant and a Premature Decision
Hypothetical record. The protocol requires three evaluable participants before a dose decision and requires replacement of an unevaluable participant. At dose 1, three participants are treated: one has a DLT, one has no DLT, and one is classified as unevaluable. The team records 1 DLT among the two evaluable participants and escalates without treating a replacement. A replacement is treated later and has no DLT.
At the time of escalation, the cohort has only two evaluable participants. The protocol-required set of three evaluable outcomes is incomplete, so the dose decision is not ready. Escalation before the replacement outcome is available is a deviation, even though the observed count among the two evaluable participants is only \(1/2\). That fraction is not the protocol's \(1/3\) decision state.
After the replacement is evaluated, the completed cohort has \(1+0=1\) DLT among 3 evaluable participants. The correct next action under the illustrative rule is to add three more evaluable participants at dose 1, not to escalate immediately. The replacement's later no-DLT outcome clarifies the correct rule state going forward; it does not erase the earlier premature assignment. The audit records the unresolved period, the deviation, and the subsequent action needed to bring decisions back to a documented protocol state.
Why a Local Error Can Affect Later Decisions
A rule-based design can be represented as a sequence of states. A state contains the information the protocol uses at a decision point: for example, dose level, evaluable DLT count, cohort completion, and active safety gates. The protocol maps that state to an allowed action. An audit reconstructs the state at each time and checks whether the observed action matches that mapping.
The key justification for chronological checking is that later decisions depend on earlier ones. If all participant records and decision times are correct, and every action follows the protocol mapping, then the sequence of states follows the protocol from its initial state. This follows by induction: the initial state is the protocol's starting state; if the current state is correct and the action is the prescribed one, the next state is the one generated by that action and the newly available outcomes. If an action differs, the next state may no longer be the state the protocol would have reached. The audit must therefore mark the point of divergence and examine downstream decisions rather than treating each later dose as an isolated event.
This comparison also explains why the order of records matters. An outcome recorded later cannot be treated as though it were known earlier. Likewise, a participant assigned to a higher dose in error is still part of the actual exposure record, but that assignment should not be counted as a protocol-authorized transition when reconstructing the rule path.
Comparison, Diagnostics, and Limits
An audit complements, rather than replaces, other trial activities. Real-time safety review asks what should happen now given current safety information; an audit checks whether past actions followed the applicable protocol. Source-data verification checks whether records accurately reflect source documentation; the decision audit additionally checks whether the reconciled information leads to a permitted action. A tally of total DLTs by dose can summarize outcomes, but it can miss whether a decision was made too early, whether the cohort was complete, or whether an active pause prohibited escalation.
| Approach | Main question | What it does not establish by itself |
|---|---|---|
| Decision audit | Was each action permitted by the applicable rule and information available then? | Whether the design has desirable statistical operating characteristics |
| Real-time safety oversight | What safety response is needed as the trial proceeds? | A complete retrospective account of protocol compliance |
| Source-data review | Are recorded assignments and outcomes supported by source records? | Whether the protocol's escalation rule was followed |
| Overall dose-level tally | How many recorded DLTs occurred at each dose? | The timing, cohort completeness, and authorization of each decision |
Useful warning signs include a dose assignment dated before the preceding cohort's outcomes were available, a denominator that includes an unevaluable participant contrary to the protocol, a missing replacement, use of an outdated dose map, or a decision made during a required safety pause. Conflicting dates, unclassified events, or an unclear rule are reasons to pause the audit conclusion and reconcile the evidence—not reasons to assume the most convenient interpretation.
A common mistake is to argue, “The later cohort had an acceptable DLT count, so the earlier escalation was effectively correct.” This confuses the eventual data with the information available at the original decision time. The correction is to judge the action against the rule and information available then, record the later outcomes separately, and trace their consequences without retroactively authorizing the earlier action.
An audit also has limits. It can establish whether documented actions match a specified rule, but it cannot repair an ambiguous protocol, determine the clinical significance of an adverse event without appropriate assessment, or show that a design performs well across possible trials. If the governing rule is unclear, the audit should report that limitation and seek an authorized interpretation rather than inventing a rule after seeing the outcomes.
Check Your Understanding
Answer each question using the illustrative 3+3 rule described above. The solutions are included for checking your reasoning.
- At a dose level, the first three evaluable participants have 1 DLT. What action is required before escalation can be considered?
- Three participants at dose 2 have 0 DLTs, and three at dose 3 have 1 DLT. The team escalates to dose 4 before completing dose 3's additional cohort. Is the escalation permitted? Why?
- A participant's DLT classification is unresolved when the team makes a dose decision. What should the audit record?
- Why does a later no-DLT replacement not make a dose escalation that occurred before replacement compliant?
Solutions. (1) Treat three additional evaluable participants at the same dose; 1/3 does not permit immediate escalation. (2) No. The dose-3 count is 1/3, so the rule requires expansion there before escalation. (3) Record the decision state as unresolved until the classification and its availability time are clarified; do not silently count the participant as having or not having a DLT. (4) The decision must be judged using information available at its original time. The replacement outcome may establish the later rule state, but it cannot satisfy a gate retrospectively.