Eligibility Criteria and the Target Population

Eligibility Defines Who the Trial Is About

An emulated trial does not automatically describe everyone represented in its data. Its conclusions concern the people who would qualify under the trial’s eligibility rules. A clinically sensible rule can make the question precise, but it can also narrow the target population substantially. The important task is therefore not simply to list criteria: it is to state what population those criteria select and why.

The target trial protocol frames a causal question; eligibility specifies which people enter that question. For an emulation, the rule must be applicable using information that could be observed before treatment assignment. A rule that depends on later events can select people based on what happens after the causal question begins, changing the comparison the trial is intended to represent.

Definition: Let $L_{\mathrm{pre}}$ denote the information recorded before assignment, and let $E$ be an eligibility indicator, with $E=1$ for eligible people and $E=0$ otherwise. An eligibility rule is a prespecified function $g$ such that $E=g(L_{\mathrm{pre}})$. The target population is the population distribution conditional on $E=1$, provided $P(E=1)>0$. Inclusion criteria specify conditions that must be met; exclusion criteria specify conditions that rule a person out.

“Observable before assignment” is a substantive requirement, not just a convenient data-processing choice. The information must be defined at a point before assignment, available or ascertainable for the people being considered, and measured in a way that can be applied consistently. For example, “has a recorded diagnosis of condition X by the qualifying date” is more operational than “has condition X,” which could mean a clinical diagnosis, a self-report, or a laboratory threshold. The rule should state which definition is intended.

Eligibility does not itself make a study randomized, remove confounding, or guarantee that every eligible person appears in the data. It defines the population for the causal question. Whether the observational data support valid estimation of that question requires additional assumptions and methods.

State the Rule and the Resulting Population

A usable eligibility specification identifies the information needed to apply each criterion, the relevant measurement window, and how ambiguous or missing values are handled. A rule such as “adult patients with a recorded diagnosis of asthma before the qualifying date” is more reproducible than “patients with asthma.” If a laboratory threshold is used, state the threshold and units. If two measurements are required, say how their dates relate to the qualifying date.

Inclusion and exclusion rules work together. If $I(L_{\mathrm{pre}})$ indicates that all inclusion criteria are met and $X(L_{\mathrm{pre}})$ indicates that at least one exclusion criterion is met, then one common form is $E=I(L_{\mathrm{pre}})\{1-X(L_{\mathrm{pre}})\}$. This representation is useful only if “met” and “unknown” are distinguished: treating a missing measurement as though it passed a criterion can silently change the selected population.

The target population is not merely a list of diagnoses. It has a distribution of age, disease severity, prior care, and other characteristics induced by the rule. Even if an overall effect is reported as one number, that number averages effects across this particular distribution.

Worked Example: Applying a Clinical Eligibility Rule

Suppose a study asks about adults with moderate chronic kidney disease at a qualifying date. Its prespecified rule is: age at least 18 years; two recorded estimated glomerular filtration rate values, each from 30 through 59 mL/min/1.73 m², measured at least 90 days apart and both before the qualifying date; no dialysis recorded before that date. The data contain 1,000 people screened.

The first criterion is age. Of the 1,000 people, 920 are at least 18, so 80 fail this inclusion criterion. Among the 920 adults, 210 have two qualifying values that meet both the range and timing requirements. The other 710 do not meet the laboratory inclusion rule, either because a value is outside the range or because the required pair is not documented.

Of the 210 adults who meet the laboratory rule, 14 have dialysis recorded before the qualifying date. The final eligible count is $210-14=196$. Thus the rule selects 196 people, not all 1,000 screened people and not all 920 adults.

The target population is adults with the specified repeated laboratory evidence of moderate disease and no prior dialysis, as represented by the source population from which these people were screened. It is not automatically all adults with chronic kidney disease: people with a single qualifying value, missing values, or other disease stages are outside this rule’s target population. That distinction follows from the criteria, not from the treatment effect estimate.

How Eligibility Determines the Average Effect

Let $Y^1$ and $Y^0$ denote the potential outcomes under two specified assignments, and suppose both are well-defined for the people in the population under discussion. The eligible-population average effect is $\Delta_E=E[Y^1-Y^0\mid E=1]$. This is a population quantity: it averages individual contrasts over the people satisfying the eligibility rule. For the expectation to be finite, assume $P(E=1)>0$ and $E[|Y^1-Y^0|\mid E=1]<\infty$.

Theorem: Suppose $L_{\mathrm{pre}}$ takes values in a finite or countable set, $E=g(L_{\mathrm{pre}})$, $P(E=1)>0$, and the potential outcomes have finite conditional means in eligible strata with positive probability. Then $$ \Delta_E = \sum_{l:g(l)=1} \left\{E[Y^1\mid L_{\mathrm{pre}}=l]-E[Y^0\mid L_{\mathrm{pre}}=l]\right\} P(L_{\mathrm{pre}}=l\mid E=1). $$

Proof. Write $D=Y^1-Y^0$. Because $E$ is determined by $L_{\mathrm{pre}}$, the eligible population consists exactly of the values $l$ for which $g(l)=1$. Partitioning the event $E=1$ by those values gives $$ E[D\mid E=1] = \sum_{l:g(l)=1} E[D\mid L_{\mathrm{pre}}=l,E=1]P(L_{\mathrm{pre}}=l\mid E=1). $$ For every $l$ in this sum, $g(l)=1$, so knowing $L_{\mathrm{pre}}=l$ already implies $E=1$. Therefore $E[D\mid L_{\mathrm{pre}}=l,E=1]=E[D\mid L_{\mathrm{pre}}=l]$. By linearity of conditional expectation, this is $E[Y^1\mid L_{\mathrm{pre}}=l]-E[Y^0\mid L_{\mathrm{pre}}=l]$. Substitution proves the result. $\square$

For continuously distributed baseline information, the same reasoning is expressed as an integral over the eligible values of $L_{\mathrm{pre}}$, weighted by their conditional distribution in the eligible population. The key point is unchanged: the average depends both on the stratum-specific effects and on the distribution of strata selected by the criteria.

Worked Example: A Rule Changes the Average Effect

Consider a source population of 2,000 people divided into two baseline groups. A narrower eligibility rule selects 1,200 people in group 1 and excludes the remaining 800 people in group 2. Suppose the mean potential-outcome difference $E[Y^1-Y^0\mid L_{\mathrm{pre}}\text{ in group 1}]$ is 5 units, and the corresponding difference in group 2 is $-1$ unit.

Within the eligible population, the group 1 proportion is $1,200/1,200=1$, so the eligible-population effect is $1(5)=5$ units. In the broader source population, the group proportions are $1,200/2,000=0.6$ and $800/2,000=0.4$. Its average effect is therefore $$ 0.6(5)+0.4(-1)=3-0.4=2.6\text{ units}. $$

The difference between 5 and 2.6 is not an arithmetic inconsistency. The two averages refer to different population distributions. Neither number is universally “the” effect without specifying the population. Excluding group 2 may be clinically justified, but the narrower estimate then answers a narrower question.

Changing a Rule Changes the Target Population

The effect of a stricter rule can be made explicit when one eligible set is contained within another. Let $E$ denote a narrow rule and $G$ a broader rule, with $E=1$ implying $G=1$. Assume both populations have positive probability and the relevant conditional effects are finite. Define $q=P(E=1\mid G=1)$.

Result: The average effect under the broader rule is the weighted average of the effect among those satisfying the narrow rule and the effect among those in the broader rule who fail the narrow rule: $$ E[Y^1-Y^0\mid G=1] = qE[Y^1-Y^0\mid E=1] +(1-q)E[Y^1-Y^0\mid G=1,E=0]. $$

Derivation. Again let $D=Y^1-Y^0$. Within $G=1$, the events $E=1$ and $E=0$ partition the population. By the law of total expectation, $$ E[D\mid G=1] = E[D\mid E=1,G=1]P(E=1\mid G=1) + E[D\mid E=0,G=1]P(E=0\mid G=1). $$ Since $E=1$ implies $G=1$, the first conditional expectation is $E[D\mid E=1]$. Also $P(E=0\mid G=1)=1-q$. Substitution gives the stated identity. If $q=1$, the broader and narrower populations coincide up to probability zero, and the identity reduces to their shared effect. $\square$

This result explains why adding or removing an exclusion criterion is not a neutral editing decision. It can change the mix of people whose effects are averaged, even when the outcome and the treatment contrast are unchanged. If the effects are the same in both subpopulations, broadening the rule does not change the average; if they differ, it generally does.

Worked Example: Broadening an Age Criterion

A narrow rule requires age 65 or older. A broader rule requires age 50 or older. Suppose 300 of the 500 people meeting the broad rule are at least 65, so $q=300/500=0.6$. Suppose the mean effect is 2 units among those at least 65 and 6 units among those aged 50–64.

The broad-rule population contains 200 people aged 50–64, so its proportion is $200/500=0.4$. Applying the decomposition gives $$ E[Y^1-Y^0\mid \text{age}\geq 50] =0.6(2)+0.4(6)=1.2+2.4=3.6\text{ units}. $$

The narrow-rule effect is 2 units, while the broad-rule effect is 3.6 units. The change is exactly the contribution of the newly included age group. A report of either estimate should say which age rule defines the target population.

Why Eligibility Must Use Pre-Assignment Information

A post-assignment condition is not simply a more detailed baseline rule. If eligibility depends on a later event, it may select people differently across assignment groups and on the basis of events related to outcomes. Conditioning on that selection can create an association even when assignment has no causal effect. The following example isolates the issue.

Worked Example: A Future Event Creates a False Contrast

Let $A$ be a randomized binary assignment, with $P(A=1)=P(A=0)=0.5$. Let $U$ be an independent binary risk factor, with $P(U=1)=P(U=0)=0.5$. Suppose the outcome is $Y=U$ under either assignment, so each person’s potential outcomes satisfy $Y^1=Y^0=U$. The true average effect is therefore $E[Y^1-Y^0]=0$.

Now, incorrectly define eligibility using an event observed after assignment: $E=1$ if $A=1$ and $U=1$, or if $A=0$ and $U=0$. Each of the four $(A,U)$ combinations has probability $0.25$, so $P(E=1,A=1)=0.25$ and $P(E=1,A=0)=0.25$.

Among selected people assigned $A=1$, the rule forces $U=1$, hence the mean outcome is 1. Among selected people assigned $A=0$, it forces $U=0$, hence the mean outcome is 0. The selected-group difference is $1-0=1$, despite the true causal effect being zero.

The error is not repaired by calling the selected people a target population: this rule does not define membership using pre-assignment information. It selects on a condition jointly determined by assignment and risk. Eligibility should instead be specified using information available before assignment, then applied consistently regardless of the assignment received.

Diagnostics, Alternatives, and Limitations

Before analyzing outcomes, audit the eligibility rule as a data definition. A practical review asks whether every criterion has a clear operational definition, whether its required information is available before assignment, and whether the same algorithm would classify the same person the same way in another dataset. Tabulate the number eligible and excluded at each step, including the reasons for exclusion. Compare baseline characteristics across the eligible and excluded groups to understand how the rule changes the represented population.

These checks can reveal problems without proving that the causal analysis is valid. Large numbers of missing measurements may make the eligible group a selected subset of the intended clinical group. Different sites may use different diagnostic codes or measurement schedules. A very restrictive rule may leave few eligible people or remove clinically important subgroups. Conversely, a very broad rule may include people for whom the scientific question is not meaningful. Eligibility should reflect the intended question, not be adjusted after inspecting which rule produces a preferred result.

The main alternative is to define a broader or narrower target population. A broader rule can improve relevance to a wider group but may include people with different effects or less comparable care. A narrower rule can focus the question on a clinically specific group but limits applicability. Neither choice is inherently superior; they answer different questions. If a rule is changed, state the new population and recalculate or otherwise re-estimate the quantity for that population rather than interpreting the old estimate as though its target had not changed.

A further limitation is that eligibility based on observable information can still be measured imperfectly. A diagnosis may be absent from the record despite being present, or a laboratory value may be missing for reasons related to health status. Clear definitions make the rule reproducible, but do not guarantee that recorded eligibility perfectly represents the clinical state. Also, an effect for the eligible population does not automatically generalize to people excluded by the rule; that requires additional evidence and assumptions.

Common pitfall: “The data include people who eventually meet the clinical definition, so I can define eligibility using that information.” This reasoning can condition on post-assignment events and change group composition. Define eligibility from information available before assignment, document how unknown values are handled, and describe the resulting target population explicitly.

Check Your Understanding

Try each question, then check the solution immediately below it.

A rule requires a diagnosis recorded before assignment and excludes people with a missing diagnosis date. Is missingness being treated as eligible or ineligible, and why must the rule say so?
A broad population contains 40% of group A and 60% of group B. Their mean effects are 3 and 1 units, respectively. What is the broad-population mean effect?
A narrower rule selects 75% of a broad eligible population. The mean effect is 4 units in the narrow group and 0 in the broad group’s remainder. Find the broad-group mean effect.
Why is excluding people based on a condition first observed after assignment problematic for defining trial eligibility?

Solutions: (1) The rule makes people with missing diagnosis dates ineligible, because it excludes them; the specification must say this so that missingness is not silently treated as proof that the diagnosis was absent or present. (2) The weighted average is $0.4(3)+0.6(1)=1.8$ units. (3) The broad effect is $0.75(4)+0.25(0)=3$ units. (4) Post-assignment information can be affected by assignment or related to outcomes, so selecting on it can make selected groups differ for reasons that distort their comparison; eligibility should use pre-assignment information.