Why an Adaptive Decision Needs Its Own Record
In an adaptive Bayesian trial, an interim analysis may lead to a change in enrollment, treatment allocation, or another action specified by the trial’s decision process. A later reviewer needs more than a final analysis or a meeting summary to understand that action. The reviewer needs to be able to trace the path from the data available at the time, through the analysis and decision rule, to the authorized decision and the action actually taken.
This is the purpose of an audit trail for adaptive decisions. It is not a claim that every decision can be recreated from a single document, nor is it simply a record that a meeting occurred. It is a linked set of records that identifies the inputs, analysis, outputs, decision authority, and implementation for each decision point. The records should preserve what was known and used at that time, as distinct from later corrected or updated information.
The earlier material on regulatory governance of interim decisions, independent monitoring and access to unblinded data, protocol amendments, and data quality provides relevant context. Here the focus is narrower: what records make an individual adaptive decision traceable and reconstructable. The audit trail documents the decision process; it does not replace the prespecified adaptive rule or the governance arrangements that determine who may see data and act.
The Reconstruction Chain
A practical way to define the record is to treat a decision as the output of a documented chain. Let \(D_j\) denote the decision at decision point \(j\). The record should identify the data snapshot \(S_j\), the analysis specification and implementation \(A_j\), the output \(O_j\), the authorized decision \(D_j\), and the action \(K_j\) taken in response. It should also identify the time and responsible roles associated with each link.
This chain gives a simple completeness test. If the decision record names an output but not the dataset that generated it, the evidence input is unclear. If it names the data and output but not the analysis version, the calculation may be ambiguous. If it records a recommendation but not the person or body authorized to decide, the decision’s status is unclear. If the decision is documented but its implementation is not, the record does not establish whether the trial followed the decision.
The chain also explains why preserving the exact contemporaneous record matters. Re-running a later, corrected dataset or a newer analysis version may produce a different result. Such a rerun can be useful for investigating an issue, but it does not by itself establish what information supported the original decision.
What to Capture at Each Link
A useful decision-level record contains the following linked elements. The level of detail can differ by trial and decision, but the information should be sufficiently specific to resolve which data, analysis, authority, and action are meant.
- Decision-point identity and timing. Give the decision point a unique identifier, state the relevant data cutoff and analysis date, and record when the analysis was run, when results were reviewed, when the decision was authorized, and when the resulting action took effect. These may be different times.
- Data snapshot. Identify the source and version of the analysis dataset, the cutoff rules, relevant inclusion and exclusion criteria, and the data transfer or extraction date. Record the status of data review and query resolution, the handling of corrections, and any external data version used. Preserve or identify the snapshot itself, with enough metadata to distinguish it from later revisions. When controlled access is required, preserve the access and transfer history as well.
- Analysis specification and execution. Identify the protocol or plan version and any applicable amendment, the model and decision rule versions, and the analysis program and configuration used. Record the execution date, operator or responsible role, and relevant computational settings. Where computation involves random draws or stochastic algorithms, retain the settings needed to identify the run, such as the random seed when applicable. This is provenance for the decision record, not a substitute for a separate assessment of software validation.
- Outputs actually reviewed. Identify the output files, tables, or reports presented to decision-makers, including their versions and creation times. Record the decision-relevant quantities, warnings, deviations, or limitations that were communicated. If a result was regenerated, corrected, or replaced, identify which version was considered and explain the disposition of the earlier output.
- Decision and authority. Record whether the output was a recommendation or a formal decision, the decision made, the person or body with authority, and the date and means of authorization. Include the rationale at a level that connects the decision to the prespecified rule and any permitted judgment. A meeting minute should identify the decision, not merely state that the topic was discussed.
- Action and confirmation. Specify the operational action, the responsible role, the effective date, and how completion was confirmed. Examples include a change in enrollment or a documented instruction to continue under the current plan. If no action followed, record that too and explain why.
For each element, a controlled identifier and version can be more useful than a file name alone. A record should make it possible to retrieve the cited object, establish its status at the relevant time, and distinguish it from similarly named files. Appropriate access controls and a time-ordered history help protect the integrity and confidentiality of the record, particularly when interim results are unblinded.
Worked Examples
Worked Example: Recording a Continue Decision
Hypothetical trial. A trial’s prespecified interim rule says to continue enrollment if the posterior mean response probability is greater than \(0.60\). For illustration, the model is binomial with a \(\operatorname{Beta}(1,1)\) prior. At the first interim, the locked snapshot contains 20 participants with evaluable outcomes, of whom 13 responded. The team records the posterior mean as the rule’s decision statistic.
Under this model, if \(x\) responses are observed among \(n\) participants, the posterior distribution is \(\operatorname{Beta}(x+1,n-x+1)\), and its mean is \((x+1)/(n+2)\). Substitution gives:
Since \(0.6364>0.60\), the illustrative rule indicates continuing enrollment. The audit record would identify the interim decision-point ID; the snapshot identifier, cutoff, and 20-person evaluable set; the model, prior, and rule version; the analysis run and output showing the calculation; the authorized decision-maker and authorization time; and the instruction and effective date for continuing enrollment. The calculation alone is not the audit trail: it must be linked to those records.
Worked Example: Preserving an Original Decision After a Data Correction
Hypothetical trial. At a later review, a data correction changes the count for a previously analyzed snapshot. The original snapshot recorded 11 responses among 20 participants. The decision rule is still to continue if the posterior mean under a \(\operatorname{Beta}(1,1)\) prior exceeds \(0.60\). A corrected snapshot later records 10 responses among the same 20 participants.
For the original snapshot, the statistic was:
Because \(0.5455\) is not greater than \(0.60\), that snapshot did not meet the illustrative continue criterion. For the corrected snapshot, the calculation is:
The corrected value also does not meet the criterion. The audit record should retain both snapshot identities and their status, document what value changed and when it was corrected, and link each calculation to the corresponding analysis run. It should separately document whether the correction prompted a new review or decision. Overwriting the original file or replacing its result without a dated correction history would obscure what the original decision-makers saw.
Worked Example: Connecting a Recommendation to an Authorized Action
Hypothetical trial. At an interim review, a designated analysis team reports 8 responses among 12 evaluable participants. The illustrative rule is to continue if the posterior mean under a \(\operatorname{Beta}(1,1)\) prior exceeds \(0.60\). The calculation is:
The statistic exceeds \(0.60\), so the analysis output supports a continue recommendation under this illustrative rule. Suppose the authorized committee reviews the output on a recorded date and authorizes continuation, while an operational team implements that instruction two days later. The audit trail should distinguish the analysis team’s recommendation, the committee’s authorization, and the operational action. It should identify the output version reviewed, record the committee’s decision and rationale, and confirm the implementation date and responsible role. Calling the analysis team’s recommendation “the decision” would incorrectly conflate analysis with authority.
Justification, Comparisons, and Diagnostics
The reconstruction requirement follows directly from the dependency chain. If \(O_j\) is the output produced from \(S_j\) using \(A_j\), then interpreting \(O_j\) requires knowing both inputs. If \(D_j\) is justified by reference to \(O_j\), the record must identify which output the authorized decision-maker reviewed. And if \(K_j\) is the operational consequence of \(D_j\), the record must show that the consequence matches the decision. Leaving out any link prevents a complete check of the chain.
These expressions are a bookkeeping representation, not a claim that all human decisions can be reduced to a numerical function. They show which inputs and transitions need documentation. A committee may consider contextual information allowed by the governance process; the record should identify that information and explain its role rather than imply that the model output alone determined the decision.
An audit trail is complementary to, but different from, several other records. Narrative minutes can explain deliberation, but may not uniquely identify the data or analysis output. A final report can summarize the trial, but may not preserve the version used at each interim. A data-management history can describe corrections, but may not show which analysis used which data. The decision-level audit trail links these records rather than replacing them.
| Record type | What it is useful for | What it may not establish alone |
|---|---|---|
| Decision-level audit trail | Links snapshot, analysis, output, authority, and action | Every scientific rationale or technical validation detail |
| Meeting minutes | Documents discussion, attendance, and deliberation | The exact analysis inputs and output version |
| Data history | Documents data entry, review, and corrections | Which data version informed a particular decision |
| Final report | Summarizes methods and results for the completed trial | The contemporaneous decision process at each interim |
Several practical diagnostics can reveal gaps. Ask whether an independent reviewer can retrieve the cited snapshot and output; whether the cutoff and data status are explicit; whether the model, rule, and analysis versions are unambiguous; whether the authorization is distinguishable from a recommendation; and whether the action has a dated confirmation. A version label such as “final” is not sufficient if multiple files carry that label. Nor is a statement that “the committee reviewed the analysis” adequate if it does not identify which analysis.
Contradictory dates, unexplained changes in participant counts, an output timestamp after the purported decision, missing authorization, or implementation inconsistent with the decision are signals to investigate. The audit trail should preserve the discrepancy, its resolution, and any resulting follow-up; it should not silently normalize conflicting records. This documentation cannot make a poorly specified rule sound, correct an inappropriate analysis, or establish that an action was scientifically justified. It makes the actual process inspectable so those questions can be assessed.
Common Pitfalls and Limitations
- Keeping only the final result. A reported posterior summary does not identify the snapshot, analysis version, or output reviewed. Link it to those objects.
- Replacing rather than versioning. A corrected dataset or regenerated output should not erase the contemporaneous version. Preserve the original and document its relationship to the replacement.
- Treating a recommendation as authorization. Record who recommended, who had authority, and who acted. These roles may differ.
- Recording an action without checking implementation. An instruction is not evidence that the instruction took effect. Record completion or explain why it did not.
- Assuming a complete trail proves correctness. Traceability supports review; it does not itself validate software, establish data quality, or justify the statistical model or decision threshold.
The appropriate record should be proportionate to the decision’s consequences and the trial’s governance, while still allowing the chain to be reconstructed. Protect sensitive data and unblinded results through suitable access controls. Preserve enough information for review without treating unrestricted access as a requirement. The central standard is not maximal paperwork; it is an unambiguous, time-ordered link from evidence to analysis, decision, authority, and action.
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions.
- An interim report gives the posterior mean and says “continue,” but does not identify the dataset version. What essential reconstruction link is missing?
- Why should a corrected snapshot not overwrite the snapshot used for an earlier decision?
- An analysis team recommends continuing, and a committee has authority to decide. Which roles and events should the record distinguish?
- A decision is documented, but there is no evidence that the operational instruction took effect. What part of the chain is incomplete?
- Does a complete audit trail establish that the Bayesian model or software is valid? Explain.
Solutions. (1) The report must link its output to the specific data snapshot and cutoff used. (2) Retaining the original preserves the evidence available at the time; the corrected version and its effect can then be reviewed separately. (3) Document the analysis team’s recommendation, the committee’s authorization and rationale, and the implementation by the responsible operational role. (4) The action-and-confirmation link is incomplete; record whether, when, and by whom the instruction was implemented, or why it was not. (5) No. The trail supports reconstruction and review, but validity of the model or software requires separate assessment.