Why the Overall Result and the Responder Question Are Different
A large randomized trial can estimate the average effect of a treatment in its enrolled population with considerable precision. That does not automatically establish whether a smaller, biologically plausible subgroup benefits. The distinction matters when a treatment is proposed for patients thought to be especially likely to respond, such as patients who have not yet developed detectable antibodies or patients with impaired immune function.
In the RECOVERY comparison, hospitalized patients with COVID-19 were randomly assigned to usual care or usual care plus high-titre convalescent plasma. The primary outcome for this interpretation is death from any cause within 28 days after randomization. The central question is not whether plasma had a plausible mechanism, but what the randomized comparison showed about mortality—and how far that result can be extended to proposed responder groups.
The Mortality Comparison
RECOVERY assigned 5,795 patients to convalescent plasma and 5,763 to usual care. The reported 28-day mortality was about 24% in each group: 1,398 deaths in the plasma group and 1,408 in the usual-care group. The trial’s reported mortality rate ratio was 1.00, with a 95% confidence interval from 0.93 to 1.07 and a p-value of 0.93.
A rate ratio of 1 represents equal mortality rates in the two groups. A ratio below 1 favours plasma for mortality; a ratio above 1 favours usual care. The point estimate of 1.00 therefore indicates no estimated mortality advantage in the overall randomized population. The confidence interval is also important: it is compatible with a modest relative reduction or increase in mortality, rather than proving that the true effect is exactly zero. The large sample makes the overall estimate informative, but it does not turn a non-significant result into proof of equivalence.
Worked Example: Reconstructing the Crude Mortality Risks
Use the reported death counts and randomized group sizes to calculate the crude 28-day mortality risks and their difference. These calculations describe the observed proportions; they need not match the trial’s reported model-based rate ratio exactly.
The risk in the plasma group is
or 24.12%. The risk in the usual-care group is
or 24.43%. The crude risk difference, plasma minus usual care, is
which is approximately \(-0.31\) percentage points. In these data, the observed mortality proportion was slightly lower with plasma, but the difference is small relative to its sampling uncertainty. The crude risk ratio is approximately \(0.2412/0.2443=0.987\), close to 1. The trial’s reported rate ratio of 1.00 is the appropriate published estimate to quote; small differences from a ratio of crude proportions can arise from the analysis method and rounding.
A useful rough translation of the reported relative confidence interval makes the scale of uncertainty easier to see. If the usual-care risk is taken as 24.43%, multiplying it by 0.93 and 1.07 gives approximately 22.72% and 26.14%. On this simplified scale, the interval corresponds to roughly a 1.71 percentage-point reduction through a 1.71 percentage-point increase in mortality. This is an illustration, not a separately reported absolute-risk confidence interval: it holds the comparison risk fixed and translates the relative limits approximately.
What the Overall Estimate Does—and Does Not—Show
The primary randomized comparison addresses the average effect of offering plasma to the hospitalized patients enrolled in the trial, under the care and treatment conditions in place during that trial. Randomization supports a causal comparison between assigned groups, subject to ordinary issues such as adherence and missing outcome information. It does not mean that every patient received the same co-interventions, nor does it make the result automatically applicable to every stage of illness or every plasma product.
For the overall population, the near-one rate ratio and its confidence interval provide no evidence that convalescent plasma reduced 28-day mortality. The p-value of 0.93 means that, under the statistical null hypothesis and the model used, data at least this incompatible with a difference were not unusual. It is not the probability that the null hypothesis is true, and it does not measure the probability that plasma has no effect.
A non-significant result is also not a formal equivalence finding. Equivalence requires a prespecified margin defining how large a difference would be clinically unimportant, along with an analysis showing that the confidence interval lies within that margin. RECOVERY’s reported confidence interval is evidence about the range of relative effects compatible with the data; it should be interpreted directly rather than replaced by the statement that the treatments are “the same.”
Antibody-Negative Patients and Proposed Responders
One biologically motivated subgroup was patients without detectable SARS-CoV-2 antibodies at randomization. The rationale is understandable: donor antibodies might be more useful when a recipient has not yet developed a detectable antibody response. In this subgroup, the reported mortality rate ratio was 0.96, with a 95% confidence interval from 0.85 to 1.07. The interval includes 1 and does not establish a mortality benefit. The trial reported no statistically significant evidence that the treatment effect differed according to baseline antibody status; the interaction p-value was 0.34.
An interaction test addresses a different question from a within-subgroup test. The within-subgroup estimate asks whether the effect in that subgroup is distinguishable from no effect. The interaction test asks whether the estimated effects differ between subgroups. A non-significant interaction does not prove that effects are identical; it indicates that the data did not provide strong evidence of a difference. Likewise, a subgroup estimate that is not statistically significant does not prove that the subgroup has no benefit.
Worked Example: Reading the Antibody-Negative Estimate
Suppose a reader sees the reported antibody-negative subgroup rate ratio of 0.96 (95% CI 0.85 to 1.07) and concludes, “Plasma reduced mortality by 4% in patients without antibodies.” Check that interpretation.
The point estimate is 4% below 1 on the relative scale, since \(1-0.96=0.04\). But that is an estimate, not a demonstrated benefit. The confidence interval includes 1, and it is compatible with effects ranging from a 15% relative reduction to a 7% relative increase. Thus the result does not establish benefit in this subgroup, and it also does not rule out every potentially beneficial effect.
The interaction p-value of 0.34 is not a test of whether the antibody-negative subgroup alone benefited. It tests whether the estimated effect differs by antibody status. Since it is not small, the trial did not find convincing evidence of different effects by this characteristic. The correct summary is that the subgroup estimate was uncertain and that there was no statistically significant evidence of treatment-effect heterogeneity by baseline antibody status.
Proposals for other responder groups, including patients with impaired immune function, require the same discipline. A biological explanation can motivate a subgroup hypothesis, but it is not itself evidence of a clinical mortality benefit. The overall trial result cannot exclude a distinct effect in every small or narrowly defined group. Conversely, the possibility of such an effect cannot be presented as if RECOVERY had established it. A credible responder claim needs a clearly defined population, an appropriate randomized comparison, enough outcome information for useful precision, and evidence that the effect differs meaningfully from the effect in other patients.
Why a Large Trial Can Still Leave Subgroup Uncertainty
The overall sample size is not the sample size for every subgroup. Dividing participants according to baseline characteristics reduces the number of patients and deaths contributing to each estimate. Mortality estimates are driven especially by the number of outcome events, so a subgroup can have limited precision even within a very large trial. If many subgroups are examined, apparently unusual estimates can also arise by chance; selective emphasis on a favourable subgroup after seeing the results makes the evidence less reliable.
The following short calculation illustrates why a large overall study can estimate an overall risk difference more precisely than a much smaller subgroup can estimate its own difference. For independent randomized groups with binary outcomes, the estimated risk difference is \(\hat{p}_1-\hat{p}_0\). Its approximate standard error is
Each term is the estimated variance of a group’s sample proportion; independence of the randomized groups means the variances add. The denominators are the numbers of patients contributing to the relevant comparison. Restricting the calculation to a subgroup reduces those denominators, usually increasing the standard error. This explains why the overall trial can give a comparatively precise answer about its enrolled population while leaving a smaller responder-group estimate uncertain.
Worked Example: Why Subgroup Size Matters
Consider a clearly hypothetical comparison in which mortality is about 24% in both groups. With 5,000 patients per group, the approximate standard error of the risk difference is
That is about 0.85 percentage points. If only 500 patients per group contribute to a proposed subgroup, with the same illustrative risks, then
The subgroup standard error is about 2.70 percentage points, over three times as large. These hypothetical numbers are not RECOVERY subgroup results; they show why the overall trial’s size should not be mistaken for equally precise evidence about every subset.
Comparison, Diagnostics, and Limits
The randomized overall comparison is the strongest basis for the average mortality conclusion in the enrolled population. A subgroup analysis can address whether effects vary across patient characteristics, but it is generally less precise and is vulnerable to multiplicity, sparse events, and data-driven selection. Observational comparisons of plasma recipients and non-recipients may add context, but treatment selection and differences in illness severity can confound their results; they do not replace the randomized comparison.
When reading a claimed responder result, check the following before interpreting it:
- Definition: Was the subgroup characteristic measured before treatment and specified clearly, or was the group chosen after inspecting outcomes?
- Precision: How many patients and deaths contributed? Is the confidence interval narrow enough to distinguish clinically important benefit from little or no effect?
- Interaction: Was treatment-effect heterogeneity assessed directly, rather than inferred because one subgroup’s p-value was below a threshold and another’s was not?
- Multiplicity: Were many candidate subgroups examined? A single favourable estimate among many comparisons may be a chance finding.
- Applicability: Does the trial population, timing, and intervention match the patients and plasma product to which the claim is being applied?
The mortality result is specific to the enrolled hospitalized population, the trial’s treatment context, and the intervention tested. It does not settle every question about plasma in a different disease stage, with a different product, or in a separately defined patient group. Those possibilities are limits on generalization, not evidence that the overall trial’s randomized mortality comparison should be disregarded.
Worked Example: Evaluating a Proposed Responder Claim
A hypothetical report states that a treatment “works in immunocompromised patients” because the estimated mortality rate ratio in that subgroup is 0.82, while the overall trial rate ratio is 1.00. The subgroup interval is 0.60 to 1.12, and no interaction result is given. What can be concluded?
The subgroup point estimate is compatible with a potentially meaningful relative reduction, but the interval also includes 1 and extends to a possible increase. The subgroup result alone therefore does not establish a mortality benefit. The overall estimate being near 1 does not disprove a subgroup effect, but the difference between the two point estimates is not sufficient evidence that effects differ. A treatment-by-subgroup interaction analysis, considered with the subgroup’s size, prespecification, and multiplicity, is needed to assess that claim. On these facts, “works in immunocompromised patients” is too strong; the estimate is hypothesis-generating and uncertain.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- What does a reported mortality rate ratio of 1.00 mean in the overall RECOVERY comparison?
- Why does a 95% confidence interval of 0.93 to 1.07 not prove equivalence?
- What question does a treatment-by-antibody-status interaction test address?
- Why can a large overall trial leave the effect in a proposed responder subgroup uncertain?
- What evidence would be needed to support a claim that plasma reduces mortality in a narrowly defined responder group?
Solutions. (1) The estimated mortality rates were equal on the reported relative scale; it is not proof that the true effect is exactly zero. (2) Equivalence requires a prespecified clinically acceptable margin and a confidence interval contained within that margin; the reported interval alone is not such a test. (3) It assesses whether treatment effects differ across antibody-status groups, not whether one subgroup’s effect alone differs from no effect. (4) Subgroup analyses include fewer patients and often fewer deaths, which increases uncertainty; multiple subgroup searches can also produce chance findings. (5) A clearly defined and preferably prespecified group, adequate randomized outcome data, useful precision, and credible evidence of a treatment effect—and of meaningful difference from other groups—would be needed.