When Better Prognosis Looks Like Better Treatment
Suppose a trial enrolls participants over several months while diagnosis or care improves. Participants who enter later may then have a lower event risk, regardless of which treatment they receive. If an adaptive allocation rule assigns a larger share of later participants to the experimental arm, that arm can have a lower overall event rate even when treatment itself changes no one’s outcome. The observed difference combines treatment assignment with a difference in when participants entered.
The issue is not that a declining event risk necessarily invalidates randomization. At each enrollment, the next assignment may still be randomized according to the protocol. Rather, adaptive assignment probabilities can vary with enrollment history, making the treatment arms’ temporal compositions different. When calendar time is prognostic, those compositions matter for a crude comparison that pools all participants.
This model describes a hypothetical setting, not a real trial. To interpret the calculations, assume the event is measured the same way in every period, outcomes are available for all participants, and the stated period-specific risks apply to both assignments. For calculations based on adaptive probabilities, also assume the period sizes and allocation probabilities are as specified; the resulting arm compositions describe their large-sample proportions. These assumptions isolate the effect of improving prognosis and differing enrollment-time composition.
How Temporal Composition Creates a Crude Difference
First consider two periods, early and late, with event risks \(p_E\) and \(p_L\), where \(p_E>p_L\). Let \(f_1\) be the proportion of treatment-arm participants enrolled late, and \(f_0\) the proportion of control-arm participants enrolled late. If the period-specific risks are the same in both arms, the pooled risks are weighted averages of the two period risks.
Subtracting gives a direct expression for the apparent treatment-minus-control risk difference:
The formula shows both ingredients. A decline in event risk means \(p_E-p_L>0\); more late enrollment in the treatment arm means \(f_1>f_0\). Together these make the crude difference negative, which looks like a treatment benefit when fewer events are favorable. If either ingredient is absent—no prognostic change, or equal temporal composition—this particular difference is zero under the model.
Worked Example: Same Period Risks, Different Pooled Risks
In a hypothetical trial, the early-period event risk is 40% and the late-period risk is 8% in both arms. Among 100 treatment participants, 25 enrolled early and 75 late. Among 100 control participants, 75 enrolled early and 25 late. Suppose the event counts match the stated risks exactly within each period.
Treatment arm. The early events are \(25(0.40)=10\); the late events are \(75(0.08)=6\). Thus the pooled event risk is \(R_1=(10+6)/100=0.16\), or 16%.
Control arm. The early events are \(75(0.40)=30\); the late events are \(25(0.08)=2\). Thus \(R_0=(30+2)/100=0.32\), or 32%.
The crude risk difference is \(0.16-0.32=-0.16\), a difference of \(-16\) percentage points. Yet the risks are identical between assignments in each period: 40% early and 8% late. The apparent benefit comes entirely from the treatment arm having more participants in the lower-risk late period.
The composition formula confirms the calculation. Here \(f_1=0.75\), \(f_0=0.25\), and \(p_E-p_L=0.40-0.08=0.32\), so \((f_0-f_1)(p_E-p_L)=(0.25-0.75)(0.32)=-0.16\).
From Adaptive Probabilities to Arm Composition
A response-adaptive design can create temporal imbalance when its probability of assigning treatment changes over enrollment. For example, let \(q_t=P(A_i=1\mid T_i=t)\) be the treatment-assignment probability in period \(t\). If period sizes are equal, the fraction of treatment assignments coming from a particular period is proportional to \(q_t\); the corresponding control fraction is proportional to \(1-q_t\). This is a composition calculation, not a claim that every realized trial will have exactly those fractions.
Worked Example: Increasing Treatment Allocation During Improving Prognosis
Consider four equally sized enrollment periods in a hypothetical trial. Event risks for either assignment are 40%, 30%, 20%, and 10%, respectively. The adaptive treatment-assignment probabilities are 0.25, 0.40, 0.60, and 0.75. Thus treatment assignment becomes more common as enrollment proceeds and prognosis improves.
Across the four equally sized periods, the treatment-assignment weights are \(0.25,0.40,0.60,0.75\), with sum \(2.00\). The expected treatment-arm risk, in the large-sample allocation proportions, is
For control, the allocation weights are \(1-q_t\): \(0.75,0.60,0.40,0.25\), also summing to \(2.00\). Therefore,
The expected crude difference is \(0.2075-0.2925=-0.0850\), or \(-8.50\) percentage points. Within every period the event risk is identical under treatment and control. The pooled difference arises because the treatment weights emphasize later, lower-risk periods, while the control weights emphasize earlier periods.
These calculations use allocation probabilities to find the limiting composition, not to assert exact realized counts. In a finite trial, random assignment produces variation around those proportions. If period sizes are unequal, the appropriate weights also include the number enrolled in each period. Specifically, if period \(t\) contains \(n_t\) participants, treatment weights are proportional to \(n_tq_t\), and control weights to \(n_t(1-q_t)\).
A Second Check on the Direction and Scale
The same mechanism can be checked with two periods and a different set of values. This is useful because a risk difference and a relative risk describe the apparent contrast on different scales; neither scale removes the underlying temporal imbalance.
Worked Example: An Apparent Relative Risk Reduction
Suppose early event risk is 50% and late event risk is 30% for either assignment. In a hypothetical large trial, the treatment arm is 60% late-enrolled and the control arm is 20% late-enrolled.
The treatment-arm risk is \((1-0.60)(0.50)+0.60(0.30)=0.20+0.18=0.38\), or 38%. The control-arm risk is \((1-0.20)(0.50)+0.20(0.30)=0.40+0.06=0.46\), or 46%.
The crude risk difference is \(0.38-0.46=-0.08\), or \(-8\) percentage points. The crude relative risk is \(0.38/0.46=0.8261\), approximately 0.826. It would be tempting to describe this as a 17.4% lower observed event risk in the treatment arm, since \(1-0.8261=0.1739\). That is a description of the pooled data, not evidence of a treatment effect: the risks within both periods are equal across assignments.
As a check, the composition formula gives \((0.20-0.60)(0.50-0.30)=-0.40(0.20)=-0.08\), exactly the pooled risk difference.
Derivation and Interpretation
The two-period result follows by writing each arm’s pooled risk as the risk among early participants times the early fraction, plus the risk among late participants times the late fraction. Since the early fraction is one minus the late fraction,
This establishes the sign without requiring a treatment effect: if \(f_1>f_0\) and \(p_E>p_L\), then \(R_1-R_0<0\). Reversing either inequality reverses the sign. For instance, if prognosis worsens over time while treatment receives more late assignments, the compositional difference points toward apparent harm instead. If risk is flat over time, unequal late fractions alone do not create this bias.
The result concerns a marginal, pooled comparison. It does not say that every participant’s prognosis improved, or that treatment caused the trend. Nor does it establish that any particular adaptive algorithm must produce this pattern. The key conditions are a prognostic time trend and a difference between arms in their distribution over time.
What to Check and What the Calculation Does Not Show
A practical first check is to tabulate enrollment period alongside assignment and outcomes. Compare the proportions enrolled early and late in each arm, and examine whether the event rate changes over the same periods. A plot or table that shows both a declining event rate and a treatment arm concentrated later makes the mechanism plausible. If the arm compositions are similar, or the event rate does not change, this specific explanation is less compelling. Such checks identify a possible source of a crude contrast; they do not by themselves prove why the assignment pattern arose.
The main alternatives answer different questions. A fixed allocation probability that does not respond to accumulating outcomes does not deliberately shift allocation over time, though random finite-sample differences in enrollment timing can still occur. A pooled, unadjusted comparison is simple but combines within-period outcome patterns with each arm’s period composition. Comparisons that account for enrollment time, or designs that balance assignment across enrollment periods, address this temporal imbalance more directly; their assumptions and implementation require separate consideration.
The numerical results also have limits. They assume the period-specific risks are known or accurately represented and that outcomes are observed consistently. In a finite study, the observed risks fluctuate around the underlying risks. Missing outcomes, changing event definitions, or an incorrect description of the time trend can alter the calculation. Moreover, this example assumes no treatment effect at any time; it does not analyze a setting where treatment effects themselves vary over calendar time.
Check Your Understanding
Try each calculation using the stated no-treatment-effect model. Full solutions follow the questions.
- Early and late event risks are 35% and 15%. The treatment arm is 70% late-enrolled and the control arm is 30% late-enrolled. Find the pooled risk in each arm and the crude risk difference.
- Using the setting in question 1, what would the crude risk difference be if both arms were 50% late-enrolled? Explain.
- With equal-sized periods, treatment probabilities are 0.20 and 0.80 in two periods. Event risks are 30% and 10%. Calculate the limiting pooled risk in each arm and the crude difference.
- State two conditions needed for improving prognosis to create an apparent treatment benefit through this mechanism. Give one reason the observed difference might not equal the calculated large-sample difference in a finite trial.
Solutions. (1) Treatment risk is \(0.30(0.35)+0.70(0.15)=0.105+0.105=0.210\), or 21.0%. Control risk is \(0.70(0.35)+0.30(0.15)=0.245+0.045=0.290\), or 29.0%. The crude difference is \(0.210-0.290=-0.080\), or \(-8.00\) percentage points. (2) Equal late fractions give \(R_1-R_0=(0.30-0.30)(0.35-0.15)=0\). Both pooled risks are \(0.5(0.35)+0.5(0.15)=0.250\), or 25.0%. (3) Treatment weights are 0.20 and 0.80, so \(R_1=\{0.20(0.30)+0.80(0.10)\}/1.00=0.140\). Control weights are 0.80 and 0.20, so \(R_0=\{0.80(0.30)+0.20(0.10)\}/1.00=0.260\). The difference is \(-0.120\), or \(-12.0\) percentage points. (4) Event risk must decline over enrollment, and the treatment arm must have a larger late-enrolled fraction than control. In a finite trial, random variation can make the realized arm counts and event rates differ from their large-sample proportions.