Operationalizing Eligibility Criteria in Causal Designs

Why Eligibility Must Be Operational

A protocol may say that a study concerns “adults with recently diagnosed disease who are appropriate candidates for either treatment.” That description conveys a clinical idea, but it does not yet tell an analyst which records to include. Different reviewers could interpret “adult,” “recently diagnosed,” or “appropriate candidate” differently. In an observational data set, those differences can change who is selected and therefore the population represented by the causal contrast.

Operationalizing eligibility means translating each clinical condition into a rule that can be applied consistently to a specified person, using information available at a specified time. The rule should identify the relevant data source, event or measurement, threshold, time window, and handling of missing or conflicting records. The goal is not to replace clinical judgment with arbitrary code. It is to make the intended clinical judgment explicit enough to be reviewed, reproduced, and applied without using information that arises after baseline.

The subscript \(i\) allows baseline times to differ across people. A diagnosis recorded on a calendar date before a person’s \(t_{0i}\) may qualify as prior history; the same diagnosis recorded after \(t_{0i}\) is a post-baseline event for that person. Eligibility is therefore not just a list of diagnoses or thresholds. It is a set of conditions tied to a time origin.

From Clinical Intent to Executable Criteria

A useful criterion is specific enough that two analysts with the same data and codebook would reach the same decision. For each proposed rule, clarify five elements: who or what it concerns, the clinical definition, the time window, the data source, and what happens when the information is absent or inconsistent. A criterion that says “no recent treatment” is incomplete until “recent,” “treatment,” and the relevant records are defined.

Eligibility rules can be expressed as an intersection of conditions. If a protocol has \(K\) criteria, write \(C_{ik}=1\) when individual \(i\) satisfies criterion \(k\), and \(C_{ik}=0\) otherwise. When every criterion is required, overall eligibility is:

Because each \(C_{ik}\) is either zero or one, the product is one only when every required condition is met. If a criterion is unknown because its required information is missing, do not silently treat “unknown” as either satisfied or failed. Prespecify whether the record is ineligible, excluded pending review, or handled under a separate missing-information rule. The correct choice depends on the clinical question and data system, but it must be stated and reported.

Clinical terms often need measurement rules. “Disease” could mean a diagnosis code, a validated algorithm, a laboratory threshold, or a clinician’s diagnosis. “Stable” could refer to a measured value within a range or a documented assessment. “No prior use” could mean no record in the available database or a confirmed absence of use. These are not interchangeable. A database rule establishes eligibility as operationally measured; it does not guarantee that the underlying clinical state has been measured perfectly.

Temporal rules also need an unambiguous boundary. Specify whether the window includes its endpoints, which dates define the event, and how to handle time stamps when only dates are available. For example, “in the 180 days before baseline” should state whether an event exactly 180 days earlier counts. If the data record an event date and a service date, specify which date governs. These details matter especially when records are close to baseline.

Baseline Eligibility Versus Later Events

A person’s eligibility should ordinarily be decided using information available by baseline. A condition observed before baseline can be used even if it is clinically relevant to later care. By contrast, an event occurring after baseline is not a baseline eligibility characteristic. Excluding a person because that person later has an outcome, switches treatment, develops a contraindication, or fails to remain observable conditions inclusion on future experience.

The distinction is about time relative to each person’s baseline, not about whether the event is “old” in a general sense. A diagnosis discovered after baseline may describe a condition that existed earlier, but it is not automatically baseline information. If a protocol intends to use a retrospective test or later adjudication to establish a baseline condition, it must specify that measurement procedure and justify how it identifies the condition as of baseline. Otherwise, the rule risks relying on information that would not have been available at the eligibility decision.

This distinction protects the meaning of the target population. Suppose the intended target is the eligible population at baseline. If inclusion instead requires remaining event-free for 30 days, the selected group is no longer simply that baseline population. It is a group selected partly by what happened after baseline. In a causal comparison, such selection may depend on treatment, prognosis, or both. It can make groups incomparable even when the original assignment was randomized, and it changes the population and question being studied.

A Short Justification: Eligibility Defines the Target

Let \(Y_i^1\) and \(Y_i^0\) denote the potential outcomes under the two treatment conditions, and suppose the scientific question concerns the average contrast among people eligible at baseline. The target can be written as:

This expression makes the role of the eligibility rule explicit: it determines which individuals are included in the conditioning population. Changing a threshold, time window, or missing-data rule can change who has \(E_i=1\), and consequently can change the target, even if the outcome model and estimation procedure are left untouched.

Now suppose an analyst adds a post-baseline condition \(S_i=1\), such as “no hospitalization during the first 30 days,” and reports a contrast among those with \(E_i=1\) and \(S_i=1\). The resulting target is conditioned on a later event, not just baseline eligibility. If treatment can affect \(S_i\), or if prognosis affects both \(S_i\) and the outcome, the selected population can differ across treatment conditions. Thus the added rule does not merely clean the baseline cohort; it changes the causal question and may introduce selection bias. The justification is temporal: \(E_i\) is fixed from baseline information, while \(S_i\) is realized after baseline and may be affected by treatment or prognosis.

Making the Rule Auditable

Before applying eligibility criteria to a full data set, check the rules against boundary cases and inspect the resulting cohort. A compact audit should ask whether each rule can be computed from the specified fields, whether the time comparison is correct, and whether missing or conflicting records have a defined disposition. It should also verify that no post-baseline variable is being used inadvertently.

Useful diagnostics include counts excluded by each criterion, the number with unknown eligibility, and checks for dates that fall exactly on a boundary. Examine whether one criterion unexpectedly removes a large share of the cohort; that may reflect a data-capture limitation, a coding mismatch, or an unintended interpretation. If criteria are applied sequentially, report the sequence or provide counts that clarify how exclusions overlap. The final eligible sample count alone cannot show whether a rule behaved as intended.

Alternatives, Limitations, and Common Mistakes

A narrow, specific rule can improve clinical coherence but reduce the number of eligible records and limit applicability to people outside the restricted group. A broad rule can retain more records but may combine people with materially different clinical states. Neither is automatically superior: the rule should match the intended question, and the resulting target population should be described honestly. Eligibility criteria are also distinct from later statistical adjustment. Adjusting for a baseline characteristic does not make an otherwise ineligible person meet a protocol’s clinical definition.

Operational rules inherit limitations of their data. An absence of a diagnosis code is not necessarily evidence that the disease is absent; it may mean the diagnosis was not recorded. A dispensing record may not establish that medication was taken. A laboratory result may be measured at a clinically unusual time. These limitations should be assessed and reported, not hidden by a precise-looking algorithm. In some settings, clinical adjudication or sensitivity analyses using alternative reasonable definitions may be appropriate, provided the primary rule is specified and the alternatives are not chosen to favor a desired result.

• Mistake: excluding people for events after baseline. The wrong reasoning is, “This person later had a contraindication, so they were not eligible.” The correction is to decide baseline eligibility using the prespecified baseline information; later contraindications are follow-up events unless the protocol explicitly defines a different design.
• Mistake: treating no recorded evidence as evidence of absence. A blank record may mean unknown, not “no.” Specify how missing data are handled and distinguish absence of a record from confirmed absence of the condition.
• Mistake: leaving time windows implicit. “Recent use” or “current diagnosis” can produce inconsistent decisions. State a reference date, duration, endpoints, and event-date field.
• Mistake: assuming eligibility is only a data-cleaning step. Changing eligibility changes the population defining the target. Describe who qualifies and who does not, in clinical as well as operational terms.