Comparing Two Target Trial Protocols

Why Compare Protocols?

A target trial protocol makes a causal question operational. It specifies who is eligible, which strategies are compared, when assignment and follow-up begin, and what outcome contrast is of interest. Two protocols can concern the same clinical decision yet define different target populations or different interventions. Their estimates may therefore differ even when both are implemented correctly.

The useful comparison is not simply “which protocol gives the larger effect?” First ask whether both protocols target the same causal contrast. Then identify which design choices account for any difference. This audit depends on aligning eligibility with time zero, avoiding selection created by future information, and distinguishing a population-average effect from effects within covariate strata.

Definition: Write a protocol as $P=(E_P,g_{P,1},g_{P,0},t_P,\psi_P)$, where $E_P$ is its eligibility rule, $g_{P,1}$ and $g_{P,0}$ are its two treatment strategies, $t_P$ is time zero, and $\psi_P$ is its target estimand. Let $S_P=1$ indicate eligibility under protocol $P$. If both protocols compare the same binary treatment at a common horizon, write the protocol-specific risk-difference estimand as $\Delta(P)=E[Y^1-Y^0\mid S_P=1]$. If strategies or outcome definitions differ, use their corresponding potential outcomes instead; the resulting estimands need not be the same causal contrast.

Here $Y^a$ denotes the outcome that would be observed under treatment $a$, defined relative to the protocol’s intervention and follow-up. For a binary outcome, $\Delta(P)$ is a risk difference. Its meaning depends on the population selected by $S_P$, as well as on the strategies and outcome horizon. A protocol comparison must therefore record all four design choices, not just the treatment labels.

Compare Each Protocol on Its Own Terms

Start by writing both protocols side by side. In particular, verify that eligibility can be assessed at time zero using information available then. Requiring a participant to survive, remain enrolled, or avoid an event after time zero in order to qualify can select on future experience. It may also create immortal time if that period is incorrectly credited to one treatment strategy. The earlier material on selection bias and protocol audits gives the relevant temporal warning: define eligibility before assignment and follow-up, rather than using future treatment or outcome information to classify baseline membership.

Protocol element	Questions for comparison
Eligibility	Who is included, and can inclusion be determined at time zero? Does one rule select a different mix of patients or a different subset within covariate strata?
Strategies	Are the interventions the same, including initiation, continuation, switching, and any grace period? Do the strategies correspond to the intended decision?
Time zero	Do eligibility, strategy assignment, and start of outcome follow-up coincide? Are baseline covariates measured before treatment assignment?
Estimand	Is the target a risk, risk difference, or another summary? Is it defined for the eligible population of that protocol and at the same horizon?

Before interpreting estimates, state which differences matter substantively. A narrower eligibility rule may make the estimate more relevant to a specific decision, while reducing generalizability. A different treatment strategy may answer a genuinely different question. A different time zero may change both the eligible population and the risk period. None of these changes is automatically a defect; the issue is whether the protocol matches the intended decision and whether the contrast is described accurately.

Worked Example: A Composition Difference With Shared Stratum Effects

Suppose two protocols compare the same treatment strategies at the same time zero and one-year horizon. Eligibility differs only in the proportions of two baseline strata, $Z=0$ and $Z=1$. Assume the treatment effect within each stratum is the same in both eligible populations: $\tau_{A,0}=\tau_{B,0}=0.10$ and $\tau_{A,1}=\tau_{B,1}=0.04$. Protocol A has $w_A(0)=0.70$, $w_A(1)=0.30$; protocol B has $w_B(0)=0.20$, $w_B(1)=0.80$, where $w_P(z)=P(Z=z\mid S_P=1)$.

The protocol-specific effects are obtained by standardizing the stratum effects to each protocol’s eligible population:

\Delta(A)=0.70(0.10)+0.30(0.04)=0.070+0.012=0.082, \qquad \Delta(B)=0.20(0.10)+0.80(0.04)=0.020+0.032=0.052.

Thus $\Delta(A)-\Delta(B)=0.030$, or three percentage points. Under the stated shared-effect condition, this difference is entirely due to the stratum weights: A includes more people in the stratum with the larger effect. This calculation does not show that either protocol is biased. It shows that different target populations can have different average effects even when stratum-specific effects are shared.

What Can Be Decomposed, and Under Which Conditions?

Let $Z$ be a discrete baseline variable with strata that cover the eligible populations. Define the protocol-specific stratum effect and stratum weight by

\tau_{P,z}=E[Y^1-Y^0\mid Z=z,S_P=1], \qquad w_P(z)=P(Z=z\mid S_P=1).

For every stratum with $w_P(z)>0$, the law of total expectation gives the exact identity below. It does not require the within-stratum effects to be equal across protocols.

Result: If the eligible population is partitioned by the values of $Z$, then $$ \Delta(P)=\sum_{z:w_P(z)>0} w_P(z)\tau_{P,z}. $$ For protocols A and B, if every stratum with positive weight in either protocol has a well-defined effect under both eligible populations, then $$ \Delta(A)-\Delta(B) =\sum_z\{w_A(z)-w_B(z)\}\tau_{A,z} +\sum_z w_B(z)\{\tau_{A,z}-\tau_{B,z}\}. $$ The first term is a composition component using A’s stratum effects as the reference; the second is a within-stratum effect component. This decomposition depends on the chosen reference protocol and is an accounting identity, not a unique causal attribution.

To prove the first identity, partition the event space for the eligible population into the disjoint strata $Z=z$. For strata with positive weight, conditional expectation gives

E[Y^1-Y^0\mid S_P=1] =\sum_z E[Y^1-Y^0\mid Z=z,S_P=1]P(Z=z\mid S_P=1) =\sum_z w_P(z)\tau_{P,z}.

For the second identity, add and subtract $\sum_z w_B(z)\tau_{A,z}$:

\begin{aligned} \Delta(A)-\Delta(B) &=\sum_z w_A(z)\tau_{A,z}-\sum_z w_B(z)\tau_{B,z}\\ &=\sum_z\{w_A(z)-w_B(z)\}\tau_{A,z} +\sum_z w_B(z)\{\tau_{A,z}-\tau_{B,z}\}. \end{aligned}

The expression requires effects to be defined wherever they are used. If a stratum is absent from one eligible population, its conditional effect in that population is not defined by the data-generating distribution. Do not silently insert a value for it. Restrict a comparison to common support, or state an additional extrapolation model that defines the missing stratum effect. This identity and its proof use only finite sums; for continuous $Z$, the corresponding statements require conditional expectations and integrability, with sums replaced by integrals.

Important condition: The simpler formula $\Delta(A)-\Delta(B)=\sum_z\{w_A(z)-w_B(z)\}\tau_z$ follows only when $\tau_{A,z}=\tau_{B,z}=\tau_z$ for every stratum with positive weight in either population. Sharing the same stratum labels is not enough: eligibility may select different people within a stratum, changing its average treatment effect.

Worked Example: Same Stratum, Different Eligible Subsets

There is only one value of $Z$, so both protocols have weight one in that stratum. Within it are two equally common baseline subtypes, $U=0$ and $U=1$. Their individual-average treatment effects are $0.20$ and $-0.10$, respectively. Protocol A includes both subtypes; protocol B includes only $U=1$. Assume these effects apply to the eligible members of each subtype.

Under A, the stratum effect is the average over its equally represented subtypes:

\tau_{A,z}=0.50(0.20)+0.50(-0.10)=0.10-0.05=0.05.

Under B, all eligible people have $U=1$, so $\tau_{B,z}=-0.10$. Each protocol has the same $Z$-stratum weight, $w_A(z)=w_B(z)=1$, but $\Delta(A)=0.05$ and $\Delta(B)=-0.10$. The difference is $0.15$, and it cannot be explained by different weights on $Z$. Eligibility changed the composition within the stratum and therefore changed the stratum-specific effect. Applying the shared-$\tau_z$ formula would incorrectly predict a zero difference.

Identification Is Protocol-Specific

For a protocol with a baseline treatment $A\in\{0,1\}$, let $X$ denote measured baseline covariates available before time zero. A standard identification result states the conditions under which its eligible-population mean potential outcome can be recovered from observed data. The conditions must be assessed for each protocol and its own eligible population.

Result: Suppose, within $S_P=1$, (1) consistency holds, so $Y=Y^A$; (2) conditional exchangeability holds, $Y^a\perp A\mid X,S_P=1$, for $a=0,1$; (3) positivity holds, $P(A=a\mid X=x,S_P=1)>0$ for each treatment and almost every $x$ in the eligible population; and (4) the relevant expectations exist. Then $$ E[Y^a\mid S_P=1] =\sum_x E[Y\mid A=a,X=x,S_P=1]P(X=x\mid S_P=1) $$ when $X$ is discrete. For continuous $X$, replace the sum by an integral with respect to its distribution among eligible individuals.

For the proof, condition first on $X$ within the eligible population:

E[Y^a\mid S_P=1] =\sum_x E[Y^a\mid X=x,S_P=1]P(X=x\mid S_P=1).

Conditional exchangeability allows the inner expectation to be replaced by $E[Y^a\mid A=a,X=x,S_P=1]$. By consistency, among those with $A=a$, $Y^a=Y$; hence this equals $E[Y\mid A=a,X=x,S_P=1]$. Substitution proves the formula. Positivity ensures the conditional observed mean is defined for the treatment group at each relevant $x$. Subtracting the formula for $a=0$ from that for $a=1$ identifies $\Delta(P)$. These assumptions do not make two protocol estimands equal: each protocol may have a different distribution of $X$, different eligibility, or different strategies.

Worked Example: Time Zero Changes the Target Population

Consider a treatment decision at hospital admission and a one-year death outcome. Protocol A defines eligibility at admission and starts follow-up immediately. Protocol B instead requires patients to survive 30 days after admission before entering, then starts its one-year follow-up on day 30. Suppose 100 admitted patients meet A’s admission criteria; 20 die in the first 30 days, and 80 survive to day 30.

A’s target population contains all 100 eligible patients, including the 20 who die during the first month. B’s target population is the 80 day-30 survivors. If B is emulated by selecting only these survivors, it addresses a conditional question about people alive at day 30, not the admission-time question A addresses. Its one-year risk also begins later. The two risks cannot be interpreted as competing answers to the same admission-time decision without additional definitions and assumptions.

If a treatment group in B is classified by receiving treatment at any point during the first 30 days, but its follow-up is credited from admission, patients must remain alive long enough to receive the treatment. That classification can create immortal-time bias. A coherent day-30 protocol instead makes eligibility and strategy assignment at day 30 and starts follow-up then; it answers the survivor question. To answer the admission question, eligibility, assignment strategies, and follow-up must be aligned at admission, with treatment strategies defined from that time.

Choosing Between Protocols

A defensible choice follows the intended decision. If the decision concerns treatment for all patients meeting admission criteria, a protocol that restricts eligibility to later survivors does not directly answer it. If the decision is explicitly about treatment among survivors at a later clinical review, that later time zero may be appropriate. Likewise, initiation and sustained-use strategies are not interchangeable: one concerns starting treatment, while the other concerns a treatment course. Comparing their estimates as if they were estimates of one common effect obscures the decision each represents.

Useful diagnostics include checking whether eligibility uses only information available at time zero; tabulating how each rule changes the eligible population and baseline covariate distribution; checking treatment overlap within each protocol’s eligible population; and comparing follow-up horizons, outcome definitions, and strategy details. Large changes in the target population or little treatment overlap can make an estimate unstable or prevent identification. Sensitivity analysis for protocol choices can show how conclusions vary across plausible definitions, but it does not make different estimands identical.

Common mistake: “The protocols use the same $Z$ strata, so any effect difference must come from different stratum weights.” This is wrong because eligibility can select different people within a stratum, changing $\tau_{P,z}=E[Y^1-Y^0\mid Z=z,S_P=1]$. The correction is to retain protocol-specific stratum effects in the exact decomposition, and use a weights-only comparison only after justifying equal stratum-specific effects across the eligible populations.

Protocol choice has limits. A well-matched protocol cannot compensate for unmeasured confounding, outcome measurement error, inadequate positivity, or unobserved loss to follow-up. Nor does an estimate for one eligible population automatically generalize to another. If the scientific aim is to transport an effect, that requires additional assumptions about how treatment effects vary between the populations and whether relevant effect modifiers are measured. The first task remains to describe the target question correctly.

Check Your Understanding

Answer each question, then compare with the brief solutions.

Protocols A and B have identical stratum effects $0.12$ and $0.02$. The high-effect stratum has weights $0.25$ in A and $0.75$ in B. Find each protocol’s average effect and the difference A minus B.
Why does having the same values of $Z$ in two protocols not by itself justify a weights-only decomposition?
Name one temporal defect in a protocol that requires survival after time zero for eligibility, and state a coherent alternative if the intended question concerns survivors at that later time.
State the three central identification conditions used in the baseline-treatment result and what each contributes.

Solutions. (1) A’s effect is $0.25(0.12)+0.75(0.02)=0.03+0.015=0.045$. B’s is $0.75(0.12)+0.25(0.02)=0.09+0.005=0.095$. Thus A minus B is $-0.050$. (2) Eligibility may select different subgroups within $Z$, so $\tau_{A,z}$ and $\tau_{B,z}$ need not agree. (3) Eligibility based on future survival selects on post-time-zero information; it may also induce immortal-time bias if treatment classification uses that period. For a survivor question, define eligibility, assignment, and follow-up together at the later time. (4) Consistency links observed and potential outcomes; conditional exchangeability removes confounding given measured baseline covariates; positivity ensures each treatment is represented at relevant covariate values.