Why Eligibility Rules Need to Be Explicit
Before analyzing a data set, an analyst must decide which observations belong to the study population defined by the research question and study design. Written criteria such as “adults enrolled during the recruitment period” are meaningful only when they can be applied consistently to individual records. If two analysts interpret the same phrase differently, they may prepare different analysis data sets before any statistical calculation begins.
This tutorial focuses on translating eligibility criteria into row-level decisions. A key distinction is whether a record is outside the intended study population because of a design rule, or whether its status cannot be determined because the data may be incomplete, invalid, or inconsistent. Those situations should not be silently combined.
The analysis unit is the entity represented by one row for the decision being made. It might be a person, visit, household, school, or specimen, depending on the data structure and study question. Eligibility rules must apply to the correct unit: a criterion about a participant should not be applied as if each visit were a separate participant, unless the design specifically defines visits as the units.
Translate Prose Into Row-Level Logic
A usable rule names the field or fields required, specifies the permitted values or range, clarifies relevant dates and units, and states how multiple conditions combine. For instance, “adult participants recruited in spring” is not yet operational. It needs a defined age threshold, a specified date field, and the exact start and end dates of “spring.”
In logical notation, let \(I_1,\ldots,I_k\) be the required inclusion conditions and \(E_1,\ldots,E_m\) the specified exclusion conditions. A fully determined eligible row satisfies every inclusion condition and none of the exclusions:
Here, \(\bigwedge\) means “and,” while \(\neg E_\ell\) means that exclusion condition \(E_\ell\) is false. This expression applies only when the values needed to evaluate the conditions are known and trustworthy. A missing value is not automatically equivalent to a condition being false. For example, a missing history field does not establish that a participant has no history covered by an exclusion criterion.
Criteria should be specified before examining how many records they remove or how results change. This helps prevent eligibility decisions from being adjusted to obtain a preferred analysis. The rule should also identify the relevant time point: age at consent may differ from age at diagnosis, and a recruitment date may differ from a data-entry date.
Worked Applications
Worked Example: Defining Eligibility for a Hypothetical Campus Survey
Suppose a hypothetical survey aims to describe first-year students who completed a survey between September 1 and September 30, inclusive. Each row represents one student. The written design criteria are: enrolled as a first-year student during the survey period, and a completed survey date within that period. The data contain five records:
| Record | First-year enrollment? | Survey date | Completion status |
|---|---|---|---|
| A | Yes | September 12 | Complete |
| B | No | September 12 | Complete |
| C | Yes | October 2 | Complete |
| D | Yes | September 30 | Complete |
| E | Missing | September 18 | Complete |
The row-level rule is: retain a student if first-year enrollment is confirmed and the survey date is on or after September 1 and on or before September 30. The endpoints are included because the design says “inclusive.”
Record A meets both conditions, so it is eligible. Record B is design-ineligible because the recorded enrollment category confirms that the student is not first-year. Record C is design-ineligible because October 2 is later than September 30. Record D is eligible: September 30 is included by the stated boundary rule. Record E cannot yet be classified because its enrollment status is missing. It is unresolved, not proven ineligible; the enrollment information needs review or retrieval.
The completion-status field is shown for context, but it is not one of the stated criteria. Unless the study protocol defines a completion rule, an analyst should not add one informally. If a completion requirement is part of the design, it must be stated explicitly and applied consistently.
Worked Example: Separating a Protocol Exclusion From a Data Error
Consider a hypothetical observational study of adults with a documented diagnosis of condition Q, enrolled from January 1 through June 30. The protocol excludes people who had the condition before age 18. Each row represents one person. For this example, assume age at diagnosis and enrollment date are recorded in years and calendar dates, respectively.
| Record | Age at diagnosis | Enrollment date | Diagnosis documented? |
|---|---|---|---|
| F | 16 | March 4 | Yes |
| G | 24 | April 9 | Yes |
| H | −2 | May 1 | Yes |
| J | Missing | February 14 | Yes |
| K | 31 | July 3 | Yes |
The inclusion conditions are documented diagnosis of Q and enrollment from January 1 through June 30, inclusive. The design exclusion is diagnosis before age 18. Record F meets the diagnosis and enrollment conditions but has diagnosis at age 16, so it is design-ineligible under the protocol. Record G meets both inclusion conditions and does not meet the exclusion, so it is eligible.
Record K is outside the enrollment period and is design-ineligible, even though its other values are plausible. Records H and J require different handling. A diagnosis age of −2 is not a plausible age; it is a data-quality problem, not evidence that the person meets the protocol’s “before age 18” exclusion. Record H should be placed on quality hold while the value is checked. Record J has no diagnosis age, so the exclusion condition cannot be evaluated; its eligibility is unresolved until the information is obtained or the study’s missing-information procedure is applied.
This distinction matters because “confirmed diagnosis before 18” is a substantive study criterion, whereas an impossible recorded age is evidence that the data need investigation. Treating every unusual value as an exclusion could remove people for reasons not specified by the design.
Worked Example: Combining Several Conditions for Each Row
A hypothetical program evaluation includes one row per clinic visit. The study defines an eligible visit as one occurring at a participating clinic, on or after April 1 and before July 1, for a patient aged at least 18 on the visit date. The dates therefore include April 1 and exclude July 1. The participating clinics are North and South.
| Visit | Clinic | Visit date | Age at visit |
|---|---|---|---|
| V1 | North | April 1 | 18 |
| V2 | East | May 10 | 42 |
| V3 | South | June 30 | 17 |
| V4 | North | July 1 | 35 |
| V5 | South | May 2 | Missing |
The rule is the conjunction of three conditions: clinic is North or South; visit date is at least April 1 and earlier than July 1; and age at visit is at least 18. V1 satisfies all three, including both specified date boundaries, and is eligible. V2 fails the participating-clinic condition, so it is design-ineligible. V3 is at a participating clinic on an included date but is under 18, so it is design-ineligible. V4 fails the date condition because July 1 is the excluded upper boundary. V5 meets the clinic and date conditions but has no age value; eligibility is unresolved rather than confirmed.
The unit here is the visit, not necessarily the patient. If one patient has several visits, each visit can be evaluated against the stated visit-level rule. Whether to include all eligible visits or select a single visit per patient is a separate design decision and must not be invented during filtering.
Why Unknown Values Must Stay Distinct
A rule can return “yes,” “no,” or “cannot determine” when the required data are incomplete. Keeping these outcomes distinct prevents a common logical error: interpreting “not recorded as meeting a condition” as “known not to meet the condition.” For a positive inclusion criterion, a missing value does not confirm inclusion. For an exclusion criterion, a missing value does not confirm absence of the exclusion. The correct next step depends on the protocol and the source data.
A short justification follows directly from the rule. Eligibility requires evidence that every inclusion condition is true and every exclusion condition is false. If one required value is unknown, then at least one part of that conjunction has not been evaluated. Therefore, the available record does not establish eligibility. It also does not necessarily establish ineligibility: if all known conditions are satisfied, the remaining uncertainty could be resolved in either direction when the missing information is obtained.
Data-quality review and eligibility review may therefore interact, but they answer different questions. Eligibility asks whether the unit belongs to the study population under the design. Quality review asks whether the recorded values are sufficiently reliable to apply that design. A corrected value may later establish eligibility or ineligibility; until then, the record should retain a clear unresolved or quality-hold status rather than receive a guessed classification.
Writing and Documenting a Reproducible Rule
A practical eligibility specification should record the study unit, each inclusion condition, each design exclusion, the fields used to evaluate them, and the precise boundaries and units. It should also state what happens when a necessary field is missing or invalid. The wording should make clear whether several conditions are joined by “and” or “or,” and how date endpoints are handled.
State what one row represents for this eligibility decision, such as one participant or one visit.
Write each inclusion and exclusion criterion separately, with fields, thresholds, units, categories, and date boundaries.
Apply the same logical rule to every row. Record a design-based exclusion only when reliable information establishes that a criterion is failed or an exclusion is met.
Mark missing, implausible, or conflicting values for resolution rather than silently treating them as eligible or ineligible.
Record the rule version and the reason for each classification so the decision can be reviewed and reproduced.
This record of the decision is useful even when the rule is simple. A label such as “excluded” alone does not show whether a row failed an age threshold, fell outside a date window, or was removed because a value could not be verified. Stating the reason keeps design choices distinguishable from data problems and makes later review more reliable.
Common Mistakes
- Adding criteria after looking at results. The wrong reasoning is that a rule is acceptable if it produces a cleaner or more convenient analysis. The correction is to define criteria from the study question and design, then apply them consistently.
- Using vague boundaries. “During spring” or “adult” is not sufficiently precise for row-level classification. State dates, endpoint inclusion, age threshold, and the time point at which age is measured.
- Calling missing information a negative answer. A blank field does not establish that an inclusion condition is false or an exclusion condition is absent. Mark the decision unresolved unless the protocol provides a specific rule.
- Confusing a data error with a design exclusion. An impossible or conflicting value is a reason to check data quality, not automatically a reason to exclude the unit under the study design.
- Applying a person-level rule to the wrong unit. If each row is a visit, specify whether the rule applies to every visit or to the person as a whole. Do not let the row structure silently determine the study population.
Check Your Understanding
For each question, explain the rule or distinction that determines the answer.
- A study includes participants aged 18 or older on the date of enrollment. What information must be specified to apply this rule consistently to each row?
- A record has a missing value for a characteristic named in an exclusion criterion. Why is it not automatically correct to classify the record as eligible?
- A participant has a confirmed characteristic that the study protocol explicitly excludes. Is this a design-based exclusion or a data-quality problem? Explain.
- A recorded age is −4 years. What is the appropriate classification issue to address before deciding whether the participant meets an age criterion?
- A data set has one row per visit, but eligibility is described in terms of participants. What must be clarified before applying the rule?