Why Pandemic Phase Matters for a Platform Trial
REMAP-CAP was designed as a platform trial for severe community-acquired pneumonia and was adapted to study COVID-19 during the pandemic. Its design allowed multiple treatment domains to be evaluated within a common trial structure. But the clinical context in which participants were treated did not remain fixed: evidence about effective care accumulated, treatment availability changed, and the background risk of severe outcomes shifted. These changes matter when interpreting a treatment comparison, even when treatment assignment within the trial was randomized.
For example, an intervention tested as an addition to usual care early in the pandemic may later be used alongside different background therapies. Meanwhile, changes in who is hospitalized, disease severity, vaccination, circulating variants, and clinical practice can alter the outcome risk in both randomized groups. A platform result therefore needs a clearly specified comparison and time context. The relevant question is not simply whether a treatment “worked,” but what treatment strategy was compared with what contemporaneous standard care, in which eligible population, and over which period.
A phase label is a summary, not a biological mechanism. The actual differences between periods may include several factors at once. Consequently, an apparent difference in treatment effect by phase does not, by itself, identify which changing factor caused it.
Define the Comparison Before Pooling
Let \(A=1\) denote assignment to the intervention strategy and \(A=0\) assignment to the control strategy. Let \(Y=1\) indicate the adverse outcome of interest, such as death by a stated follow-up time. For phase \(k\), define \(p_{ak}=P(Y=1\mid A=a,\text{ phase }k)\), interpreted here as the phase-specific outcome risk under randomized assignment. The phase-specific risk difference is \(RD_k=p_{1k}-p_{0k}\); negative values favor the intervention for an adverse outcome.
This comparison has a causal interpretation when treatment assignment is randomized within the relevant phase, the assigned strategies are sufficiently well defined, outcome ascertainment is comparable, and follow-up is handled appropriately. Randomization protects the comparison from confounding within the randomized population; it does not make the populations enrolled in different calendar periods identical.
The control strategy must also be described accurately. If usual care changes, “control” may not mean the same bundle of care throughout the trial. For an add-on intervention, the contrast is generally the intervention plus the background care available in that setting versus background care alone. If a background treatment becomes routinely available, the later contrast may be the new intervention added to that treatment, not the intervention versus the earlier standard of care.
A pooled effect requires an explicit target. If \(w_k\) are nonnegative weights summing to one, a phase-standardized risk difference is:
Equal weights target an equal mixture of the represented phases. Weights based on the phase distribution of eligible patients in a specified target population answer a different question. Neither is automatically the “right” pooled effect: the weights determine the population to which the average applies.
How Changing Care Affects Interpretation
REMAP-CAP’s COVID-19 evaluations took place while knowledge and clinical practice were developing rapidly. For instance, evidence supporting corticosteroid use in severe COVID-19 changed the care context for later participants. The trial’s immunomodulatory comparisons, including evaluation of interleukin-6 receptor antagonists, must therefore be read in the context of the background therapies participants received. A result for an intervention used alongside corticosteroids does not automatically estimate its effect when used without them.
Three distinct changes should be kept separate:
- Change in baseline risk: the probability of the outcome under control may rise or fall across phases.
- Change in the treatment strategy: the meaning of “intervention plus usual care” can change when usual care changes.
- Change in the enrolled population: eligibility, referral, disease severity, or prior immunity can shift over time.
These changes can alter an estimated treatment effect, its precision, or its applicability. They also make a simple comparison of outcomes in early and late participants unreliable: calendar time is not randomized, so differences between periods can reflect many concurrent changes.
Worked Example: Why Nonconcurrent Controls Can Mislead
Hypothetical data. Suppose a trial evaluates the same strategy in an early and a later phase. In the early phase, 40 of 100 control participants and 30 of 100 intervention participants have the adverse outcome. In the later phase, the corresponding counts are 10 of 100 and 5 of 100.
The phase-specific risks and risk differences are:
The intervention groups have 10 and 5 percentage-point lower adverse-outcome risks in the early and later phases, respectively. With equal phase weights, the standardized risk difference is:
Now suppose an analyst ignores phase. Among the intervention participants, 80% are from the later phase; among controls, only 20% are from the later phase. The pooled risks are \(0.20(0.30)+0.80(0.05)=0.10\) for intervention and \(0.80(0.40)+0.20(0.10)=0.34\) for control. The crude difference is \(0.10-0.34=-0.24\), or 24 percentage points in favor of intervention. That is not the equal-phase estimate of \(-0.075\). The discrepancy arises because the groups have different mixtures of periods with different baseline risks. In an actual randomized comparison, allocation should be interpreted according to its design and the observed data; this hypothetical imbalance illustrates why calendar-time composition must be checked, especially when using nonconcurrent controls or pooling across changing periods.
A Short Justification for Phase Standardization
The standardization formula has a direct interpretation. Imagine first calculating the intervention and control risks separately within each phase. To target a population with phase weights \(w_k\), average each strategy’s risks using the same weights, then subtract:
Using the same weights for both strategies prevents the contrast from being driven merely by one group containing more participants from a low-risk or high-risk phase. This calculation does not remove bias from unmeasured differences within a phase, repair a poorly defined treatment strategy, or establish that the phase-specific effects apply to a different population. It only makes the averaging target explicit.
Worked Example: A Standard-Care Change and a Defined Target
Hypothetical data. Suppose an add-on treatment is evaluated before and after a new background therapy becomes common. In the earlier phase, adverse-outcome risks are 32% with the add-on and 40% with control. In the later phase, when both groups generally receive the newer background therapy, risks are 14% and 20%, respectively. Assume the target population is 25% earlier-phase and 75% later-phase patients.
The phase-specific risk differences are \(-0.08\) and \(-0.06\). Standardizing to the stated target gives:
The estimated standardized risk difference is \(-6.5\) percentage points for the specified phase mixture. This is an average of two comparisons against the control strategy available in each phase. It is not an estimate of the add-on treatment’s effect in a world where background care never changed. If the treatment is used in the later phase on top of the newer background therapy, the later phase-specific contrast is the more directly relevant one for that setting.
The interpretation also depends on whether the treatment strategies are sufficiently consistent within each phase. If the newer background therapy was received by only some participants, report and examine that variation. A phase label alone cannot establish that everyone received the same care.
Worked Example: The Effect Scale Changes the Story
Hypothetical data. In an early phase, suppose adverse-outcome risk is 40% under control and 30% under intervention. In a later phase, the risks are 10% and 5%. The absolute risk differences are:
On the relative risk scale, intervention risk divided by control risk is \(0.30/0.40=0.75\) early and \(0.05/0.10=0.50\) later. Thus, the later phase has a smaller absolute reduction but a larger relative reduction. These statements are not contradictory; they describe effects on different scales. Whether the effect appears to vary by phase depends partly on the chosen scale, as well as on sampling uncertainty. A phase-by-treatment interaction should therefore be defined on a stated scale rather than described vaguely as “different effects.”
Assumptions, Diagnostics, and Failure Modes
The strongest interpretation comes from randomized comparisons using concurrent controls within the relevant period. For a phase-specific causal interpretation, check that assignment was randomized as designed, each strategy had a meaningful chance of assignment for eligible participants, the endpoint and follow-up definition were comparable, and missing outcomes or protocol deviations do not undermine the analysis. Treatment availability may cause a positivity problem: if an intervention or control strategy is no longer available in a phase, the trial cannot directly estimate their randomized contrast there.
To transport an effect from one phase to another, additional assumptions are needed. Measured effect modifiers should be captured, the treatment strategies should be comparable, and the data should support estimating outcomes under both strategies for the target population. Merely adjusting for calendar time does not guarantee these conditions. If important factors such as disease severity or background treatment differ and are not accounted for, a cross-phase comparison may not be transportable.
Useful diagnostics include tabulating enrollment and randomized treatment counts by phase; plotting recruitment, control event rates, and treatment availability over time; checking endpoint definitions and follow-up completeness; and reporting background therapy use by randomized group and phase. Examine phase-specific estimates with uncertainty intervals, not just a pooled estimate. A noisy phase-specific estimate is not proof of no effect, and a difference between point estimates is not proof of interaction. Interpretation should also consider whether phase definitions were planned or selected after viewing the outcomes.
| Approach | What it estimates or uses | Main caution |
|---|---|---|
| Concurrent phase-specific comparison | Randomized treatment contrast among participants in one phase | May be imprecise when a phase has few participants |
| Phase-standardized estimate | Average of phase-specific effects for stated phase weights | Depends on the weights and adequate information in each phase |
| Unstratified pooled comparison | One contrast across all participants | Can obscure changing baseline risk, strategy context, or phase composition |
| Nonconcurrent external control | Comparison with control participants recruited in another period | Calendar-time differences can confound the comparison |
A common mistake is to say, “The platform randomized treatment, so the pooled comparison is valid regardless of when participants enrolled.” Randomization supports a causal comparison for the randomized strategies and population represented by the design; it does not make the standard of care, participant mix, or treatment effect constant across time. The correction is to show concurrent, phase-specific contrasts where possible and to define any pooled target and assumptions explicitly.
The main limitation is that calendar phases often bundle many changes, and the number of participants may not support precise interaction estimates. Splitting the data into many periods can also produce unstable estimates. Conversely, pooling can conceal clinically important changes. Adaptive allocation and treatment-arm availability should be considered when interpreting the data, but calendar-time trends alone cannot establish that adaptation caused an outcome difference. The analysis must retain the randomized comparison and distinguish prespecified decisions from exploratory explanations.
Name the intervention, control strategy, outcome, follow-up, eligible population, and whether the question concerns one phase or a mixture.
Show how background care, participant characteristics, and treatment availability changed across the periods being compared.
Use the randomized, contemporaneous comparison where available and report uncertainty for each phase.
Choose and report phase weights, explain the effect scale, and state the assumptions needed for interpretation beyond the observed phases.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Why might a pooled comparison across early and late participants be misleading if control-group outcome risk changed over time?
- In two phases, the intervention and control risks are 18% and 25% in phase 1, and 8% and 12% in phase 2. What is the equal-phase standardized risk difference?
- Why does randomization within a phase not prove that an effect observed in that phase applies to a later phase with different standard care?
- What is the difference between a phase-specific randomized comparison and a comparison of outcomes between calendar phases?
Solutions. (1) If treatment groups or compared data sets contain different proportions of each phase, their pooled risks can reflect changing baseline risk and phase composition as well as treatment. (2) The phase differences are \(0.18-0.25=-0.07\) and \(0.08-0.12=-0.04\). Thus, \(0.5(-0.07)+0.5(-0.04)=-0.055\), or a 5.5 percentage-point lower risk under intervention for an equal mixture of the phases. (3) Randomization supports the comparison in the enrolled phase, but transport requires assumptions about differences in population, background care, treatment strategy, and effect modifiers. (4) A phase-specific randomized comparison contrasts assigned strategies among participants in that phase. A comparison between phases contrasts groups enrolled at different times, who may differ in care and other characteristics; calendar phase itself was not randomized.