Feasibility Is a Protocol-Specific Question
A protocol can be carefully specified and still be impossible to implement in a particular data source. The records may not identify enough eligible people, distinguish the treatment strategies, provide required information before treatment begins, or capture outcomes for the required follow-up. A feasibility assessment asks whether the data can represent the trial as written—not whether the treatment works.
The audit must therefore start with the protocol. It specifies who is eligible, how and when treatment strategies are assigned, which baseline information is required, what outcome is measured, and how long participants are followed. The earlier material on baseline information requirements motivates checking each required variable against the data source. Here the emphasis is on whether those requirements can be met at a useful scale and with sufficient coverage.
Feasibility is not the same as causal identification. A large, complete dataset cannot by itself establish that all confounders have been measured or that the causal assumptions are true. Conversely, a protocol may be conceptually sound but infeasible in records that do not capture treatment initiation or the outcome. Keep these judgments separate: first ask whether the intended comparison can be represented in the data; then assess the assumptions needed for causal interpretation.
Audit the Data in the Order the Protocol Uses Them
A practical audit follows the participant’s path through the emulated trial. At each stage, state the protocol rule, identify the exact data fields and dates needed to implement it, and count how many records pass. A funnel is useful because an apparently large database can shrink substantially once eligibility, treatment ascertainment, covariate requirements, and follow-up are applied.
Apply the protocol’s inclusion and exclusion rules using information available by the specified time zero. Count eligible people and, where relevant, eligible treatment-start dates. Check that the required dates and clinical criteria can actually be identified in the records.
Determine whether both strategies can be recognized and whether eligible people receive each strategy. Inspect counts in clinically relevant covariate groups as well as overall. A large total cannot compensate for a group in which one strategy is absent.
For every required covariate, verify that values refer to the period before treatment assignment and are recorded with adequate completeness and timing. A value recorded after time zero does not automatically serve as baseline information.
Check whether the specified outcome is represented, how it is dated, and whether capture is complete in the relevant population. A code or measurement is useful only if it corresponds to the outcome definition in the protocol.
Compare each participant’s observable time with the required horizon. Identify data-end dates, disenrollment, transfer, or other mechanisms that end outcome observation, and determine whether censoring can be distinguished from absence of an event.
For each stage, report a denominator and a numerator. For example, “baseline smoking status was available for 520 of 800 eligible participants” is interpretable; “smoking data were mostly complete” is not. Counts should be stratified by treatment strategy where possible, since overall completeness can conceal differences between groups. Also record the calendar period and source used for each query: coverage can change over time or across sites.
Worked Examples of Feasibility Checks
Worked Example: Finding the Usable Treatment Groups
Suppose a hypothetical database contains 12,000 people screened for a new-user comparison. Applying the protocol’s eligibility rules leaves 4,800 eligible people. Among them, records identify 620 who initiate strategy A, 1,880 who initiate strategy B, and 2,300 who initiate neither strategy during the protocol’s assignment window.
The fraction of eligible people with an identifiable initiation of either strategy is
Thus, 52.1% of eligible people initiate one of the two strategies within that window. Among initiators, the strategy A share is
So the records identify 620 strategy A initiators and 1,880 strategy B initiators, with A accounting for 24.8% of observed initiators. These counts show that both strategies occur overall, but they do not yet establish useful treatment variation in every important subgroup. The next audit would cross-tabulate treatment by key baseline characteristics and check that initiation dates support the protocol’s time-zero rule. The 2,300 people initiating neither strategy are not automatically evidence of a data failure: their role depends on how the protocol defines assignment and eligibility.
Worked Example: Checking Required Baseline History
A hypothetical protocol requires age, disease severity, and smoking status measured before time zero. There are 800 eligible initiators: 400 in each strategy. Age is available for 784 people, severity for 720, and smoking status for 520. A direct query finds all three variables available for 480 people, including 260 in strategy A and 220 in strategy B.
The joint completeness fraction is
Therefore, 60.0% of eligible initiators have all three required baseline variables recorded. The strategy-specific fractions are
Joint availability is 65.0% for strategy A and 55.0% for strategy B, a 10.0 percentage-point difference. The separate availability rates cannot be multiplied to infer joint completeness: the missingness patterns may overlap. The direct joint count is the relevant audit result. It also signals that simply proceeding with complete records could produce a selected subset and different group composition. Before proceeding, the investigator should establish why these fields are missing, whether another source contains them, and whether the protocol can be implemented without changing its scientific requirements. This audit does not, by itself, justify treating missing values as harmless.
Worked Example: Can the Data Support a One-Year Outcome?
A hypothetical study requires 365 days of outcome observation after time zero. The database closes on December 31, 2025. Of 400 eligible initiators, 240 have time zero on or before December 31, 2024, and 160 enter later. Assume for this calculation that all 240 earlier entrants remain observable in the database through the required period; the later entrants cannot have 365 days of administrative observation by the data close.
If \(D\) denotes the database end date, \(t_i\) a participant’s time zero, and \(\tau=365\) days the required horizon, full administrative follow-up is possible only when
The number meeting this date condition is 240, so the fraction of all eligible initiators with the calendar opportunity for a full year is
Thus 60.0% have the calendar opportunity for a complete year under the stated assumption. This is an upper bound on full observation if some people disenroll, transfer, or otherwise become unobservable earlier. Calendar opportunity must not be confused with confirmed follow-up or with the number of observed outcomes. The audit should separately count actual observable time and determine whether the outcome is captured throughout that time.
Why the Follow-Up Check Works
A simple date calculation clarifies why a database’s overall span is not enough. Let \(F_i\) be the time participant \(i\) remains observable after time zero, and let \(\tau\) be the protocol’s required follow-up horizon. The participant has complete outcome observation for that horizon only if \(F_i\geq\tau\). If observation ends at the earlier of administrative closure and loss of data coverage, then
where \(D\) is the administrative end date and \(L_i\) is the date the participant’s outcome data cease to be observable. Since the minimum is at least \(\tau\) exactly when both arguments are at least \(\tau\),
This is a feasibility condition, not a rule that all analyses must exclude anyone without complete follow-up. The protocol’s prespecified censoring and missing-outcome rules determine how incomplete observation is handled. The audit establishes how often and why observation ends before the required horizon, so that the planned approach can be judged against the actual data.
Diagnostics, Alternatives, and Limits
Warning signs include a steep drop from screened to eligible records; treatment dates that cannot be aligned with time zero; few or no initiators in important clinical groups; key covariates recorded only after treatment; outcome codes that do not distinguish the protocol’s outcome; and follow-up ending earlier for one strategy or subgroup. Report these patterns directly rather than hiding them in a single total. If feasible, repeat basic counts across calendar periods and sites to reveal whether apparent support depends on a narrow part of the database.
The response to a shortfall depends on its source. If a different database contains the same population and the necessary fields, obtaining those data may preserve the protocol. If the data source cannot capture a required feature, the investigator may need to revise the protocol so it asks a scientifically defensible, observable question, or decide not to emulate that trial with these records. A revision should be explicit: changing eligibility, treatment definitions, outcomes, or follow-up changes what trial is being represented. It is not a harmless data-cleaning choice.
| Approach | What it answers | What it does not establish |
|---|---|---|
| Protocol feasibility audit | Whether the data can operationalize the specified trial elements, and where records are lost | That the causal assumptions hold or that an effect estimate is unbiased |
| Power or precision calculation | Whether a proposed analysis may estimate an effect with a desired level of precision under stated assumptions | That treatment, covariates, outcomes, or follow-up are actually measured as required |
| Causal analysis | An effect estimate under its design and identification assumptions | That an unmeasured protocol element can be recovered from data that never recorded it |
A count that seems “large enough” is not a universal feasibility threshold. Adequacy depends on the planned comparison, the distribution of treatment and covariates, outcome frequency, and the analysis. Feasibility checks inform decisions about whether to proceed and what limitations to expect; they do not replace a design-specific precision assessment. Nor can a favorable audit verify absence of unmeasured confounding, correct outcome measurement, or all other requirements for a causal conclusion.
Common Mistakes
- Using the database’s total size as evidence of feasibility. The relevant number is the number remaining after applying the actual protocol rules, with usable information for the comparison.
- Checking treatment variation only overall. Both strategies may be common in the full cohort but one may be absent in a clinically important group. Inspect subgroup counts rather than relying on pooled totals.
- Equating a recorded value with a baseline value. A covariate measured after time zero may reflect treatment or subsequent events. Confirm that its measurement timing meets the protocol.
- Calling lack of a recorded outcome “no outcome.” Missing capture and a verified absence of the event are different data states. Audit the observation system and apply the protocol’s outcome rules.
- Treating feasibility as proof of validity. Enough records and complete fields do not establish causal assumptions. Feasibility and causal identification are separate assessments.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A source has 6,000 records, but only 900 meet eligibility and 300 have an identifiable initiation of either protocol strategy. Which count is most relevant to assessing treatment-group feasibility, and what else should be checked?
- In a cohort of 500 eligible participants, covariate A is available for 450 and covariate B for 400. Is joint availability necessarily 80%? Explain.
- A protocol needs 180 days of follow-up. A participant enters 100 days before the database closes. Can administrative data provide the full horizon? State the calculation.
- Both treatment strategies and all required covariates are well represented. Does that establish that the causal effect is identified? Why or why not?
Solutions. 1. The 300 initiators are the directly relevant count for whether the strategies can be identified in this eligible population, but their division between strategies and their distribution across important covariate groups must also be checked. 2. No. The overlap is unknown. Joint availability could be at most 400 and, from these margins alone, as low as \(450+400-500=350\); query the joint records directly. 3. No: administrative observation is \(100\) days, and \(100<180\). 4. No. Availability supports implementing the protocol, but does not establish the causal assumptions, including whether relevant confounding has been addressed or whether measurements are valid.