Why Check Feasibility Before Emulation?
A target trial protocol is only useful for an emulation if the available data can represent its essential events in the right people and in the right order. The dataset need not reproduce a randomized trial’s data collection exactly, but it must support a defensible reconstruction of eligibility, treatment strategies, baseline covariates, outcomes, and follow-up. If a crucial event is absent, recorded too coarsely, or measured only after it should have occurred, careful modeling cannot recover the missing information by itself.
A feasibility check is a design audit, not an effect estimate. It asks whether the proposed protocol maps to observed data well enough to be implemented and interpreted. It also distinguishes a limitation that can be addressed by linking another data source from one that requires changing the target question. The earlier material on eligibility and treatment classification, covariate measurement windows, and unequal outcome observation is useful context; the task here is to check those pieces together before committing to an emulation.
Map the Protocol to the Data
Let \(t=0,\ldots,K\) index the protocol’s decision and follow-up times. For an individual, write \(E\) for eligibility at baseline, \(A_t\) for treatment at time \(t\), \(L_t\) for covariates, \(Y_t\) for outcome status, and \(C_t\) for whether follow-up has ended by time \(t\). These symbols describe protocol-relevant variables; actual data may store them in claims, clinical records, registries, or linked sources.
For each element, record its operational definition, the data source, its time resolution, and any missingness or validation concerns. In particular, verify that eligibility can be determined using information available at baseline, treatment initiation is distinguishable from treatment history, confounders are measured before the decisions they are meant to inform, and outcomes can be identified during the specified follow-up. These are checks of data adequacy and temporal ordering, not substitutes for causal assumptions.
| Protocol element | Feasibility question | Common warning sign |
|---|---|---|
| Eligibility and time zero | Can each criterion and the start of follow-up be reconstructed for every candidate? | Eligibility depends on a later record, or no consistent time zero can be assigned. |
| Treatment strategies | Can the relevant treatment, initiation, and changes be distinguished at the needed times? | Records show only a broad exposure label or an interval that obscures sequence. |
| Covariates | Are important baseline and time-varying covariates measured before the decisions they relate to? | Only post-decision measurements exist, or measurement timing is unknown. |
| Outcome | Can the protocol outcome and its occurrence time be ascertained in the population under study? | Outcome capture depends strongly on unobserved care or has no usable event date. |
| Follow-up and censoring | Can the end of observation and loss to follow-up be distinguished? | Records stop without indicating whether follow-up ended administratively or was lost. |
This audit is not a demand for perfect data. Rather, it makes the consequences of imperfections explicit. A diagnosis code might be a practical proxy for a clinical criterion, but the audit should record that it is a proxy and assess whether its timing and validity suit the question. Likewise, missing values are not automatically disqualifying; their importance depends on what is missing, when, and for whom.
Two Basic Feasibility Results
The first result formalizes why unrecorded or ambiguous protocol events cannot simply be repaired by more elaborate analysis.
Proof. A quantity is identified from the observed-data distribution if it has the same value under every full-data description compatible with that distribution. By assumption, there are two compatible descriptions with the same observed-data distribution and different target values. Any procedure using only the observed records receives the same input distribution under both descriptions, so it cannot determine which target value is correct. Therefore the target quantity is not identified from those records alone. \(\square\)
This is a general data-feasibility principle, not a claim that every missing field makes an analysis impossible. Other measured information or justified assumptions may resolve some gaps. But those additions must be stated and defended; a model cannot distinguish possibilities that the observed records leave indistinguishable.
A second result concerns time resolution. Suppose an event is recorded only as occurring in a calendar interval, such as a month, rather than on a date or time.
Derivation. Let the interval be \([d_1,d_2]\), and let the two true event times be \(T_1\) and \(T_2\). The observed record reports only that \(T_1,T_2\in[d_1,d_2]\). The same recorded pair of intervals is compatible with \(T_1<T_2\), for example \(T_1=d_1\) and \(T_2=d_2\), and with \(T_2<T_1\), for example \(T_2=d_1\) and \(T_1=d_2\). Hence the recorded intervals do not determine the ordering. More precise records or an explicit, defensible ordering convention would be needed. \(\square\)
This matters when an outcome could occur before treatment, when eligibility must precede assignment, or when a covariate must precede a treatment decision. Choosing an ordering convention does not make the true sequence known; it makes an additional assumption that should be reported.
Worked Feasibility Audits
Worked Example: Eligibility and Treatment Dates in Claims
A proposed study compares initiation of Drug A with initiation of Drug B among adults with a recorded diagnosis. The protocol requires continuous enrollment for 365 days before initiation, no use of either drug during that period, and follow-up beginning on the initiation date. A claims extract contains enrollment spans, outpatient diagnoses, and dispensing dates. In a pilot of 1,000 people flagged as potentially eligible, 920 have a complete enrollment history, 760 of those have an observed diagnosis before a dispensing date, and 700 of those have 365 drug-free days that can be checked.
Eligibility reconstruction. Continuous enrollment can be checked for 920 of the 1,000 flagged people, so 80 cannot be assessed with this extract. Among the 920 assessable people, 760 have the diagnosis recorded before treatment. Thus the required order is verifiable for those 760, but not for the remaining 160. Of the 760, 700 have a full observable window for checking prior drug use. The other 60 lack sufficient history to confirm the washout criterion.
Decision. The extract can potentially support the protocol for the subset whose enrollment, diagnosis timing, and prior treatment history are all observable, but the 300 other flagged people cannot simply be declared eligible: some have incomplete enrollment, some lack a recorded diagnosis before treatment, and some lack the required look-back. The analyst should determine whether the missing history is recoverable through another source and whether restricting to assessable people changes the target population. The counts diagnose a feasibility problem; they do not show that the retained 700 are representative or that a causal effect is identified.
Worked Example: Monthly Dates and a Required Event Order
A protocol defines baseline covariate \(L_0\) as the most recent measurement in the 30 days before treatment initiation. In an electronic record extract, treatment and laboratory events are stored only by calendar month. For a patient, both treatment initiation and the relevant laboratory result are recorded in April. The protocol’s 30-day window would include a laboratory test on March 20 if treatment began April 10.
The April labels do not establish whether the laboratory test occurred before or after initiation. For instance, April 2 for the test and April 20 for treatment is consistent with the recorded data, as is April 20 for the test and April 2 for treatment. In the first ordering, the test may be a baseline covariate if it falls within the 30-day window; in the second, it is post-initiation and cannot serve as that baseline measurement.
Decision. The data do not reliably establish the required covariate timing for this patient. The team could obtain more precise timestamps, define a protocol that uses only measurements known to precede the treatment month if that is clinically and scientifically acceptable, or choose another data source. It should not silently treat every same-month laboratory value as baseline.
Worked Example: Outcome Capture and Follow-Up
Researchers propose estimating one-year risk of hospitalization after treatment initiation. They have an outpatient electronic record system that captures hospitalizations at their own health system. Of 500 patients, 450 remain registered in the system for a full year, while 50 have no recorded visits after month 4. The database has no linked external-hospital claims and does not indicate whether those 50 moved, changed providers, or remained event-free.
For the 450 patients with a full year of observable registration, the database can potentially ascertain a hospitalization recorded at that health system during the year. For the other 50, absence of a later hospitalization record does not establish absence of hospitalization: their observation status is unknown after month 4. Therefore the proposed one-year outcome and follow-up cannot be reconstructed uniformly for all 500 from this source alone.
Decision. The study needs to distinguish complete follow-up from loss of observable contact. Linkage to claims or a hospitalization registry might resolve the gap. Alternatively, the target question or population could be revised, with the implications for selection and interpretation stated clearly. Counting unrecorded hospitalizations as zero would impose an unjustified assumption that everyone remained under complete outcome surveillance.
Diagnosing Problems and Choosing a Response
A useful audit produces concrete evidence rather than a general label such as “the data are messy.” Report the number of records for which each required element is observable, the time precision available, and the reasons records fail a check. Inspect whether missingness clusters by site, calendar period, treatment group, or patient characteristics. Such patterns can warn that the usable records represent a selected subset, even when each individual field appears plausible.
The appropriate response depends on the failure. The main alternatives are not interchangeable:
- Improve or link the data. Seek a source with enrollment history, validated event dates, treatment records, or outcomes that the current source lacks. Confirm that linkage is permissible and accurate.
- Refine the protocol. A narrower eligibility definition, a different follow-up horizon, or a more measurable outcome may be scientifically acceptable. The revised question must be stated as a new target, not presented as though it were the original one.
- Proceed with explicit assumptions. This can be defensible when the assumption is meaningful, supported by external information, and its consequences can be examined. It is not a substitute for acknowledging that the data do not directly resolve the issue.
- Stop the emulation. If a central element cannot be reconstructed and no credible data source or assumption addresses it, the proposed emulation is not supported by the available records.
This audit differs from design-based assessment of treatment overlap. Feasibility asks whether the protocol’s variables and events can be measured and ordered from the records at all. Overlap asks a different question about whether observed treatment patterns support comparisons across strategies. Passing one check does not imply passing the other.
A common mistake is to treat a large sample size or a sophisticated model as a remedy for an undefined time zero, unknown treatment date, or outcome that is not observed after loss to follow-up. The correction is to separate measurement feasibility from statistical estimation: first establish what the records mean and when events occurred; only then decide whether an estimator is appropriate. If distinct plausible event histories fit the same records but imply different protocol classifications, report that ambiguity rather than concealing it through coding choices.
Check Your Understanding
For each problem, decide what the data permit and what the next step should be.
- A protocol requires a diagnosis before treatment initiation. Both are recorded in the same quarter, with no dates. Can their order be confirmed?
- A study defines eligibility using a one-year treatment washout, but the database begins six months before time zero. What is missing?
- A hospital record has no outcome entry after a patient stops visiting that hospital. Does this establish that the patient had no outcome?
- What is the difference between checking whether treatment dates are recorded and assessing whether treatment strategies have adequate overlap?
Solutions. (1) No. The quarter-level records are compatible with either order; obtain more precise dates or revise the operational definition with an explicit justification. (2) The database cannot verify the full washout period for that patient. Seek longer history, use a justified alternative source or definition, or exclude the patient under a transparent rule and consider the resulting target population. (3) No. The person may have had the outcome elsewhere or may no longer be observed; obtain linked outcome data or determine follow-up status before treating no record as no event. (4) Recording-date feasibility asks whether treatment events can be reconstructed at the necessary times. Overlap concerns whether the observed treatment patterns support comparisons between strategies; it is a separate design assessment.