Defining Analysis Sets And Data Cutoffs

Why the Evidence Set Must Be Defined

A population pharmacokinetic result is an inference from particular participants and particular records—not from every item that happens to be present in a study database. A reviewer therefore needs to understand which participants were eligible for the analysis, which observations contributed, what information was available by the cutoff, and why anything was excluded. These decisions define the evidence set behind the analysis.

The analysis plan should make these rules clear before results are reviewed. A useful specification connects each rule to the analysis question. For example, an analysis of exposure after a particular formulation may require participants who received that formulation, while an analysis of dose-related concentrations may need a defined relationship between dose and sample time. The required criteria depend on the question; a single generic “PK population” label does not explain them.

These are related but distinct decisions. A participant can belong to the analysis population while contributing no eligible concentration observations—for example, if no usable sample is available. Conversely, a record that appears plausible cannot be included merely because its participant is eligible: the record must also meet the observation-level rules. Eligibility criteria specify the evidence set; model development decisions specify how the eligible evidence is analyzed.

Write Criteria as Operational Rules

A criterion is operational when two qualified reviewers can apply it to the same information and reach the same decision. Avoid wording such as “adequate PK data” unless the plan defines what adequate means. State the relevant subject conditions and record conditions separately, and specify how exceptions are handled.

• Subject-level criteria may identify the study, treatment or formulation, dosing status, and any other population-defining condition required by the analysis question.
• Observation-level criteria may state which concentration records are eligible, how sample and dose times are required to be represented, and how missing, invalid, or otherwise unusable values are handled.
• Exclusion rules should name the condition that makes a participant or observation ineligible, and distinguish a prespecified rule from a decision made after examining the data.
• Cutoff rules should specify both the time boundary and what “available” means—for example, received and included in the defined data extract by that boundary.

Do not assume that every unusual value is an exclusion. A concentration that is unexpectedly high, or a sample time that seems unusual, may be valid. If the record fails a defined quality or consistency rule, document the rule and evidence supporting that decision. If it is valid but influential, that alone is not a reason to remove it from the evidence set. Handling of special measurement categories should be specified using the analysis rules appropriate to the intended model; the category should not be silently discarded by a general “invalid value” rule.

Observation Eligibility and Exclusion Rationale

For each observation type, state which records can contribute and how the decision is made. In concentration-time analyses, records need enough information to connect the measurement with the participant, the relevant dose history, and the sampling time, as required by the analysis. If one of those elements is missing or inconsistent, the record may not be interpretable for that analysis. The plan should say which fields and conditions are required rather than relying on an analyst’s undocumented judgment.

Use a consistent rationale that answers three questions: what was excluded, which rule applied, and what evidence supported the decision? For example, “two concentration records excluded because the participant identifiers were absent from the finalized subject roster” is more informative than “two records removed as unusable.” A reason should be specific enough to distinguish true data problems from analytic choices. Retain an accounting of the number of participants and observations at each relevant step, so the reported analysis set can be reconciled with the source set.

Specify a Reproducible Data Cutoff

A date alone may not define a reproducible cutoff. State the calendar date, clock time, and time zone, and define what must have occurred by that instant for a record to count as available. A practical rule might refer to information received and included in a specified analysis extract by the cutoff. The exact rule should fit the data process and be applied consistently. A sample collection date is not necessarily the date on which the result became available for analysis.

A cutoff is not a substitute for the inclusion criteria. It limits which available information can be considered; it does not make an ineligible participant or record eligible. Likewise, a record collected before the cutoff but not available under the specified rule by that time is not automatically part of the analysis evidence set.

Justification, Diagnostics, and Limitations

The evidence-set logic can be expressed as a sequence of membership rules. Let \(S\) be the set of participants in the defined study source, \(P\) the participants satisfying the population criteria, \(R\) the candidate records, \(E\) the records satisfying observation-eligibility criteria, and \(C\) the records available by the cutoff. The records available for analysis are the intersection of the eligible records, records belonging to population members, and records within the cutoff:

This is a set-membership definition, not a statistical model. Its value is that it makes the analysis evidence auditable: each included record must satisfy all stated conditions. If any one condition fails, the record does not enter \(R_{\text{analysis}}\). Participant eligibility and record eligibility therefore cannot be substituted for one another.

Useful checks include reconciling participant and record counts before and after each rule; checking that the stated reason totals agree with the unique number excluded; and confirming that cutoff comparisons use the same time zone and availability definition. Examine whether exclusions cluster by site, treatment, visit, or participant. A cluster can reveal a systematic data-capture issue or a rule with unintended consequences. Such a check does not by itself justify changing the rule; it prompts verification that the written criterion was applied as intended.

There is no universally correct population or cutoff independent of the analysis question. A narrowly defined set can omit relevant information, while a broad set can include data that do not answer the intended question. The specification also cannot guarantee that collected data are complete or error-free. Its purpose is to make the evidence basis and exclusions explicit, not to turn data limitations into certainty.

Common Pitfall: Defining the Set After Seeing the Results

A common mistake is to remove a participant or concentration because its inclusion changes the parameter estimates or makes a diagnostic plot look less favorable, then describe the result as if the rule had been objective and established in advance. That reasoning confounds data eligibility with the result of the analysis. An unusual value is not necessarily erroneous, and an unfavorable influence is not itself evidence that a record is invalid.

The correction is to define inclusion and exclusion rules in terms of study membership, data validity, and the analysis question; apply them consistently; and record the specific reason for each exclusion. If a rule was not specified or a decision requires judgment, describe the basis transparently rather than presenting it as a mechanical criterion. This tutorial concerns defining the evidence set; any later decision about plan changes or departures from the plan should be documented as such.