RECOVERY's Usual-Care Comparator Over Time

Why the Comparator Changed

In a long-running platform trial, “usual care” describes the care available to control participants at a particular time and place. It does not necessarily name one stable treatment. For RECOVERY, hospitalized patients with COVID-19 were randomized while knowledge, clinical practice, and available treatments were changing. Consequently, usual care could include different background treatments or supportive-care practices at different points in recruitment.

One consequential change followed RECOVERY’s own dexamethasone result, announced in June 2020: corticosteroid treatment became part of usual care for many hospitalized patients who needed oxygen or ventilation. That did not mean every patient received dexamethasone, or that practice changed identically at every hospital on one date. It did mean that the clinical context of later randomized comparisons differed from that of earlier ones. Other practices, including respiratory support and use of other medicines, also evolved during the pandemic.

The key interpretive question is therefore not simply, “What was the control treatment?” It is: what care policy did random assignment to usual care represent for this comparison, in this population, during this recruitment period? Answering that question helps distinguish a change in the comparator’s context from a change in the experimental treatment’s effect.

What Randomization Does—and Does Not—Protect

Within a randomized comparison, assignment makes treatment groups comparable on average at the point of randomization. If experimental-treatment and control participants are recruited over the same period and assigned under the same randomization procedure, changes in background care do not by themselves remove the benefit of randomization. Participants in both groups are exposed to the clinical context of that period; the principal contrast is the difference created by randomized assignment.

This protection is specific to the comparison being made. Randomization does not make control participants recruited months apart exchangeable, freeze background care, or guarantee that an effect in one period applies to another. Nor does it make an experimental treatment’s effect constant across periods. For example, an experimental treatment might work differently when control participants are also receiving an effective background therapy. That possibility is effect modification, not automatically a failure of randomization.

A useful way to formalize the issue is to define \(Y=1\) as death by a specified follow-up time, and let \(p_{1j}\) and \(p_{0j}\) be the risks of death in the experimental-treatment and usual-care groups, respectively, in period \(j\). Two common period-specific contrasts are the risk difference \(RD_j\) and risk ratio \(RR_j\).

A negative \(RD_j\) indicates a lower death risk with the experimental treatment in that period; an \(RR_j\) below 1 indicates the same direction on the relative scale. These are not interchangeable summaries. If usual-care mortality falls over time, a similar relative effect can correspond to a smaller absolute reduction in deaths.

What Does a Pooled Result Mean?

A pooled treatment estimate combines information from participants recruited in multiple periods. Its meaning depends on the analysis scale, the period distribution of participants, and how the estimate is calculated. In a simple setting, a pooled risk for a group can be written as a weighted average of its period-specific risks, with weights determined by that group’s distribution of participants across periods. More generally, regression or time-to-event analyses may use different weighting and modeling assumptions.

If the randomized treatment effect is genuinely similar across periods, a pooled estimate can summarize the average effect across the trial’s recruited population and care contexts. It should not automatically be interpreted as the effect under one timeless version of usual care. If treatment effects differ by period, a single pooled number can conceal that variation; its value then depends partly on how much information each period contributes.

A short justification makes the role of weighting explicit. Let \(w_j\) be the proportion of participants in a target population who belong to period \(j\), with \(\sum_j w_j=1\). The population-average risk under each assignment is the weighted average of the period-specific risks. Therefore, the population-average risk difference is the weighted average of the period-specific risk differences, provided the same period weights are used for both assignment groups.

The same simple identity does not hold for the risk ratio: the ratio of two weighted average risks is generally not the weighted average of the period-specific risk ratios. Thus, stating the effect measure and the population to which a pooled estimate refers is part of its interpretation, not a technical detail.

Reading RECOVERY Results Across Recruitment Periods

The chronology of usual care matters when describing what a RECOVERY comparison answers. For example, a treatment comparison recruited before widespread corticosteroid use in eligible patients and one recruited after that practice became common do not necessarily share the same background-care context. The later control group may receive treatments that were unavailable, uncommon, or not yet established earlier. An overall estimate can still be valid for its randomized comparison, but its interpretation should identify the population and period mix it summarizes.

In a platform trial, different treatment questions may also have different recruitment windows. A control group for one comparison should not be assumed to represent a fixed, universal RECOVERY control cohort. Where comparisons share contemporaneous controls, the relevant contrast is with those controls under the care context at that time. When interpreting results from different arms, check whether recruitment periods overlap and whether eligibility and baseline populations are comparable before treating their estimates as direct head-to-head evidence.

A treatment-by-period analysis can assess whether estimated effects differ across prespecified or clinically motivated periods. It is important to compare the treatment effects themselves, with their uncertainty, rather than to conclude that effects differ merely because one period’s result is statistically significant and another’s is not. Such analyses may be imprecise, especially when subdividing the trial; an apparent difference can reflect random variation. They are most informative when period definitions have a clinical rationale and the analysis reports estimates and confidence intervals, not just separate \(p\)-values.

Diagnostics, Alternatives, and Limitations

When reviewing a result across changing usual care, first establish the recruitment dates and whether experimental and control participants were randomized contemporaneously. Then examine period-specific participant characteristics, control-group care, and outcome risks. Changes in these summaries can reveal that the clinical context shifted; they do not alone show bias in a contemporaneous randomized contrast. Finally, check whether the reported pooled estimate corresponds to the intended population and effect measure.

• Changing control risk: a marked change in control outcomes may reflect altered patient mix, disease severity, clinical care, or other calendar-time changes. It is not itself evidence that the randomized treatment effect changed.
• Changing background treatment: describe important co-interventions and whether their use differed across periods. If treatment assignment changes access to those co-interventions, interpretation may concern a treatment strategy rather than the isolated biological effect of one medicine.
• Unequal period representation: a pooled estimate can be driven more by periods contributing more participants or events. Consider whether that weighting matches the population relevant to the question.
• Small subgroups: splitting data by calendar period reduces precision. A noisy subgroup estimate should not be treated as a reliable trend without considering its interval and the interaction analysis.

The main alternative to an unqualified pooled estimate is to report period-specific estimates alongside a pooled estimate, with a clearly described period definition and an interaction assessment when justified. Another option is standardization: calculate period-specific risks and combine them using weights chosen to represent a stated target population. These approaches answer different questions. A pooled trial estimate describes the average over its analysis population and weighting; a standardized estimate targets the chosen period mix. Neither can guarantee transport to future practice, where patient eligibility, variants, immunity, and care may differ.

A common mistake is to compare the early control mortality with later control mortality and conclude that the experimental treatment became less effective because “the control improved.” That reasoning compares groups from different calendar contexts rather than comparing randomized groups within each context. The correction is to preserve the randomized contrast within the relevant periods, then assess whether those treatment contrasts differ and state what pooled population they represent.