Documenting Protocol Deviations and Data Decisions

Why Data Decisions Need Explicit Documentation

A population pharmacokinetic (PK) analysis depends not only on the model but also on which records enter the analysis and how those records are represented. A dose entered at the wrong time can alter the inferred concentration-time relationship. A missed sample, an implausible concentration, or an unusual participant may prompt review, but each situation calls for a different decision. If the report only says that the data were “cleaned,” readers cannot tell what was changed, what was retained, or whether the decisions affected the findings.

Good documentation connects each decision to a stated rule, a count, a reason, and its analytical consequence. Rules defined before results are examined reduce the risk of selectively handling observations to obtain a preferred result. They do not eliminate judgment: unexpected cases may require a decision not covered by the plan. In that situation, the report should identify the decision as post hoc, explain why it was made, and describe the analysis used to assess its impact.

Describe the Decision, Not Just the Label

A protocol deviation describes what happened relative to the protocol; it does not automatically determine how the data should be analyzed. For example, a blood sample collected outside its nominal time may still be usable if the actual collection time is known and the analysis uses that time. Conversely, a deviation may make a particular record unusable if a required dose or sampling time cannot be established. The report should state both the event and the analysis consequence rather than implying that every deviation led to exclusion.

Data corrections need a traceable basis. A correction is appropriate when a source record or documented review supports a different value, such as a transcription error confirmed against the original source. It is not a correction to replace an unusual but verified concentration with a more typical value. Retain the original value in the audit trail, state the corrected value and its units, identify the evidence and reason, and specify which analysis dataset used the corrected value. If the value cannot be resolved, describe that uncertainty and apply the relevant prespecified rule; do not silently choose a convenient value.

Exclusions should identify their unit and scope. A participant may be excluded from the entire population PK analysis, while a single concentration record may be excluded but the participant’s other records retained. State whether the rule applies to participants, doses, samples, or observations. Report both the number affected and the denominator, because “six excluded” has different meanings in a study of 20 participants than in one of 2,000 records. Where a participant has several affected records, do not add participant and record counts as if they were mutually exclusive.

A useful decision record gives the rule, whether it was prespecified, the number affected, and the reason. It also makes clear whether the decision was applied before model fitting or arose after model results were reviewed. The precise timing matters: a rule established before examining concentrations is not equivalent to an exclusion introduced after a high residual or an unexpected parameter estimate is observed.

Prespecified Rules and Post Hoc Decisions

A prespecified rule should be operational: a reviewer should be able to determine whether a case meets it without guessing. “Exclude unreliable samples” is too vague unless reliability is defined by observable criteria, such as a missing collection time that cannot be recovered from source documentation. State the action attached to each criterion, including whether it affects an entire participant, a particular dose, or a single record. When a rule has exceptions, specify how those exceptions are handled.

Not every decision can be anticipated. When a case falls outside the plan, document when it was identified, what information prompted review, who or what process assessed it, and why the chosen action was reasonable. If the decision was made after model results were available, say so. Transparency about timing allows readers to judge the possibility of result-driven decisions; it is more informative than presenting every decision as though it had been planned from the start.

For each decision category, counts should be internally consistent. A participant can have multiple protocol deviations, and a single deviation can affect multiple records. A flow summary can therefore report participant counts and record counts separately, while noting overlap. State whether counts refer to the dataset before or after a correction or exclusion. If a correction changes one record but does not remove it, do not count it as an excluded record.

Influential Observations: Report Influence, Then Investigate

An observation can be influential because it lies in a region of the concentration-time data with little support from other records, because it has an unusual but valid value, or because it is affected by a data or model problem. Influence is a property of the analysis: it describes how much results change when a record or participant is omitted or otherwise assessed. It is not, on its own, evidence that a record is wrong. In particular, excluding a valid extreme concentration solely because it changes an estimate can bias the analysis toward a less representative result.

Report how influence was assessed, which results were examined, the number of observations or participants flagged, and what follow-up established. If the original analysis retains the observation, say so. If a separate sensitivity analysis omits it, label that analysis as a sensitivity analysis and show the relevant comparison. When several records belong to one participant, specify whether the sensitivity analysis removes one record or all records from that participant; these answer different questions.

A useful comparison reports the primary estimate and the estimate under the alternative decision, with units and enough context to assess the size of the change. It should not imply that a sensitivity result is a replacement primary result unless the analysis plan or evidence supports that change. A conclusion that is similar across analyses may be described as robust to that particular decision; it does not establish robustness to every possible data choice.

Diagnostics, Comparisons, and Limitations

Before finalizing a report, check that every stated count has a defined unit and denominator; that corrected records are not also described as excluded unless they truly were; and that participant-level and record-level counts can be reconciled. Confirm that exclusions match the stated rule and that post hoc decisions are visibly distinguished from prespecified ones. If a decision affects many records, a table can make the categories and counts easier to audit than a long narrative.

The main alternative to a strict exclusion rule is retaining records while representing their known features appropriately, such as using actual rather than nominal sampling times when documented. Neither approach is universally preferable: a record with a known deviation may remain informative, whereas a record lacking essential information may not satisfy the analysis requirements. A second alternative is to present a primary analysis under the planned rule and a sensitivity analysis under a reasonable competing rule. This reveals dependence on the decision without disguising the primary analysis.

Documentation cannot resolve every uncertainty. Source records may be incomplete, the reason for a deviation may be unknown, and sensitivity analyses can only assess the alternatives actually considered. Also, a small change in one parameter does not guarantee that all predictions or clinical conclusions are unchanged. Describe what was assessed and avoid broad claims that the data are “clean” or that results are “unaffected” unless the supporting comparisons justify those statements.

Common Pitfalls

• Wrong reasoning: “The observation is influential, so it must be an error and should be removed.” Correction: Influence indicates sensitivity of results, not invalidity. Verify the record, state the evidence, and show an appropriate sensitivity comparison.
• Wrong reasoning: “Fourteen samples were out of window, so 14 participants were excluded.” Correction: Samples and participants are different units. State how many records and how many unique participants were affected, and whether anything was excluded.
• Wrong reasoning: “We corrected the data” is enough detail. Correction: Give the original and corrected values, units, source of confirmation, and whether the record remained in the analysis.
• Wrong reasoning: A post hoc decision can be described as prespecified if it matches a sensible rule. Correction: Report when the rule was applied. A defensible decision still needs transparent timing and justification.