From a Causal Question to a Usable Protocol
A research question can sound precise while leaving crucial design choices unstated. “Does treatment reduce hospitalization?” does not yet say which population is of interest, what treatment is being compared with what alternative, or when and how hospitalization will be assessed. Those choices define the question a study is intended to answer. A protocol table makes them visible before an analyst commits to a data source or an estimation method.
The earlier material on target-trial components introduced the decisions that define a hypothetical randomized trial. Here, the practical task is to place those decisions beside their observational-data counterparts. The table does not make observational records equivalent to randomized assignment; it shows what the emulation would need to measure, what it can measure, and where important gaps remain.
For example, let \(S=1\) indicate membership in a specified target population, and let \(Y^1\) and \(Y^0\) denote the outcomes that would occur under two specified treatment strategies. An average treatment effect for that population is
The protocol states what \(S\), the two strategies, and \(Y\) mean. The observational data supply recorded variables that may help identify eligible people, their treatment histories, and their outcomes. These are different roles: the target population and causal contrast define the estimand; the dataset does not define them merely by containing certain columns.
Building the Table
A useful table has at least three kinds of information: the intended protocol decision, its proposed representation in the data, and the information needed to check that representation. Adding a status column—such as “available,” “partly available,” or “not available”—makes unresolved issues easy to spot. The example below uses a hypothetical question about a new medication and 180-day hospitalization. Its broad population description is illustrative, not a complete operational eligibility definition.
| Protocol decision | Intended target-trial specification | Observational-data counterpart | Information needed to assess alignment |
|---|---|---|---|
| Target population | Adults with the condition for whom either strategy is a reasonable option | People represented in the health-system records who meet the eventual criteria | Source-population coverage; recorded characteristics needed to identify the intended population |
| Treatment strategies | Begin the new medication versus follow the specified comparison strategy | Prescription and dispensing records, plus records describing the comparison strategy | Whether records distinguish the strategies and establish when they began |
| Assignment | Assignment to one of the two strategies under the hypothetical trial’s stated mechanism | Observed treatment choice, not randomized assignment | Factors influencing treatment choice and their measurement before treatment |
| Time zero and follow-up | A common start of eligibility, assignment, and outcome follow-up; assess outcomes through day 180 | Dates for eligibility, treatment, and outcome records | Date precision and whether the data support a coherent common start and follow-up window |
| Outcome | Hospitalization within 180 days, defined according to the protocol | Encounter or claims records that can identify hospitalization | Outcome definition, capture completeness, and availability of follow-up |
| Causal contrast | Difference in 180-day hospitalization risk under the two strategies in the target population | Data used to estimate that contrast, with the target population and strategies retained | Whether the analysis and available population correspond to the stated contrast |
| Analysis plan | A prespecified approach appropriate to the stated estimand and design | Measured covariates, treatment and outcome data, and the planned estimation procedure | Required variables, missingness, overlap, and assumptions for the chosen approach |
The table is a design aid, not a guarantee of validity. In particular, a field labelled “treatment” does not by itself show that the observed treatment history corresponds to the intended strategy. The same caution applies to eligibility, outcomes, and time zero. Each proposed counterpart must be checked against the meaning and timing of the protocol decision.
Why the Mapping Must Be Explicit
The central logic is a chain of correspondence. The protocol defines a causal quantity using intended strategies and outcomes; the dataset records variables that are supposed to represent those strategies and outcomes. If the recorded treatment variable represents a materially different intervention, or the recorded outcome covers a different time window, the analysis may answer a different question even if its calculations are flawless.
This is not merely a documentation preference. Suppose the intended strategy is to begin medication at a defined point, but the data only indicate whether a person ever received a prescription during a broad period. The record may classify exposure using information that occurs after the intended start. The protocol and measured variable then do not align in timing or meaning. An analysis of that variable cannot automatically be interpreted as the effect of the intended strategy.
A short justification for the table’s row-by-row structure follows from the estimand. In \(\tau=E(Y^1-Y^0\mid S=1)\), changing \(S\) changes the population over which the average is taken; changing either treatment strategy changes the potential outcomes being contrasted; and changing the definition or assessment period of \(Y\) changes the outcome itself. Therefore, data alignment must be assessed for each of these ingredients, rather than assumed from an overall claim that the dataset is suitable.
The table also distinguishes design decisions from data availability. A decision should be made because it defines the scientific question. The data column then reports whether and how that decision can be represented. If the intended population cannot be identified, the responsible response is to refine the data plan, find another source, or explicitly revise the target question—not to let convenience silently redefine the population.
Worked Example: Drafting a Protocol-to-Data Table
Hypothetical question: Among adults beginning care for a newly diagnosed condition, what is the effect of starting medication A rather than the comparison strategy on hospitalization risk within 180 days? Assume a health system has patient records, prescription data, encounter dates, and hospital records.
First, state the causal target in words: a 180-day risk difference for the specified population under the two strategies. For a binary outcome, one possible expression is
where \(Y^a=1\) means hospitalization within 180 days under strategy \(a\), and \(S=1\) denotes membership in the intended target population. The unit is a difference in risk, usually reported in percentage points.
Next, fill the rows rather than jumping directly to a model. The health system’s patient records may help locate people receiving care for the condition, but the protocol still needs eligibility criteria precise enough to identify \(S=1\). Prescription data may represent medication A, but the analyst needs to know whether they record prescribing, dispensing, or both and whether the dates are adequate. Hospital records may represent the outcome, but the analyst needs to verify that hospitalizations outside the system are captured if they matter to the target definition.
The proposed table therefore identifies both potential matches and unresolved information: patient records are a possible population source; prescription records are a possible treatment source; encounter dates are needed to check timing; and hospital records are a possible outcome source. It would be premature to mark all rows “aligned” before checking the criteria, date conventions, and capture of out-of-system events. No causal effect is calculated from this description because it supplies no outcome data and does not establish those alignments.
The completed first draft is useful precisely because it separates what the investigators intend from what the data source can substantiate. It also directs concrete questions to the data team before analysis begins.
Audit Feasibility Without Letting It Redefine the Question
Once a draft exists, audit each row. Ask whether the relevant information exists, whether it is measured at the right time, whether its definition matches the protocol, and whether it is available for the people the target population is meant to describe. Keep a record of uncertain definitions and missing fields. A table with a visible “unknown” is more informative than one that hides uncertainty in a broad variable label.
State the population, strategies, outcome, follow-up horizon, and causal contrast in words before selecting variables or methods.
Name the data source, variable, and relevant date or measurement, not merely a general label such as “EHR data.”
Record what is available, what needs validation, and what is absent or ambiguous. Do not treat missing protocol information as a statistical nuisance.
Improve the data, revise the protocol with scientific justification, or acknowledge that the desired question is not supported by this source.
Worked Example: Checking Outcome Availability
Suppose a hypothetical source contains 120 people: 64 recorded as starting one strategy and 56 recorded in the comparison group. At the data review stage, an outcome status through the planned horizon is available for 58 of the 64 and 50 of the 56, respectively. These counts describe data completeness; they do not establish a treatment effect.
For the first group, the observed outcome-status proportion is
For the comparison group, it is
Overall, 108 of 120 people have the outcome status recorded, so
The table should mark outcome follow-up as partly available, with 12 people lacking recorded status. The percentages alone do not tell us whether missingness is related to treatment, health, or outcome, and they do not show that the event definition is valid. The next checks are whether the planned horizon can actually be observed, why statuses are missing, and whether records capture events outside the system. A complete-case analysis or a missing-data method would require additional assumptions; neither should be selected merely because the table reveals missingness.
Distinguishing the Target From a Convenient Association
A common error is to define the question after inspecting the available columns: “We have a treatment indicator and an outcome, so their difference is the treatment effect.” That reasoning skips the protocol. The observed treatment groups can differ in prognosis, and the recorded exposure or outcome may not match the intended strategies or time window.
Worked Example: A Crude Difference Is Not the Protocol Effect
In a hypothetical dataset, 3 of 30 people recorded as receiving medication A are hospitalized within 180 days, while 14 of 70 people in the comparison group are hospitalized. The recorded risks are
The crude risk difference is \(0.10-0.20=-0.10\), or \(-10\) percentage points. This describes the difference between the observed groups in these hypothetical data. It is not, by itself, the causal risk difference in the target-trial protocol. The groups may differ in baseline characteristics, the treatment records may not represent the intended strategies, and the data may not identify the intended population or follow-up correctly. The protocol table makes those distinctions explicit; an estimation method cannot repair a mismatch that has not been understood.
A protocol table is preferable to relying only on a narrative because it places design decisions and data support side by side. It is also different from an analysis plan: the analysis plan explains how the specified causal contrast will be estimated, while the protocol table first clarifies what contrast is intended and whether the necessary information is represented. The table does not replace substantive knowledge, validation, or assumptions about confounding and measurement.
Common Mistakes and Limitations
- Using the dataset to define the target by default. Available records may cover only part of the population of scientific interest. State that limitation and its implications rather than silently treating the observed sample as the intended target.
- Treating labels as sufficient definitions. “Treatment,” “baseline,” and “hospitalization” need meanings and timing. A variable name is not evidence that the measurement matches the protocol.
- Equating observed treatment with randomized assignment. Observational treatment choice is not random assignment. The table records this difference and identifies information needed to understand it; it does not eliminate confounding.
- Filling every cell with confident language. “Unknown” or “not captured” is a useful finding. It signals a need for validation, a revised design, or a narrower claim.
- Choosing a method before defining the contrast. Matching, weighting, or regression cannot determine which population and treatment strategies the scientific question should concern.
The table’s main limitation is that a well-organized protocol cannot guarantee that its assumptions hold or that its data are accurate. It is a disciplined way to expose choices, test feasibility, and prevent an analysis from drifting away from its intended causal question. Some decisions will require further clinical or operational detail; the table should make those needs visible rather than prematurely settling them.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- What is the difference between a protocol decision and its observational-data counterpart?
- A study’s target is adults meeting a stated condition, but its database contains only people with a recorded prescription. Why might defining the target population from this database change the question?
- A planned outcome is hospitalization within 180 days, but the data contain only a yearly hospitalization indicator. What table entry should flag this, and why?
- In a hypothetical dataset, 12 of 40 people in one group and 9 of 60 in another experience an event. Calculate the crude risk difference and state what it does—and does not—mean.
- Name two checks needed before treating a prescription variable as a representation of a treatment strategy.
Solutions. (1) The protocol decision defines the intended design or causal target; the counterpart names the recorded information proposed to represent it. (2) Prescription-based inclusion may omit eligible people without a recorded prescription and can select the population based on observed treatment-related information, so it may no longer represent the original target. (3) Mark the outcome representation as mismatched or insufficient: an annual indicator does not establish whether the event occurred within the specified 180-day horizon. (4) The risks are \(12/40=0.30\) and \(9/60=0.15\); the crude difference is \(0.30-0.15=0.15\), or 15 percentage points. It is an observed-group difference, not automatically a causal effect. (5) Check what the record means (for example, prescribing versus dispensing) and whether its dates identify the start of treatment in the protocol’s intended time frame.