Why Screen Before a Larger Comparison?
An outpatient treatment for acute COVID-19 may be promising in the laboratory yet fail to improve outcomes in patients, or cause harms that outweigh a possible benefit. Testing every candidate immediately in a large trial would use substantial time and resources before establishing whether a candidate is sufficiently safe and active to justify that investment. ACTIV-2 addressed this problem through a phase 2/3 outpatient evaluation: smaller randomized assessments screened candidates, and candidates meeting protocol-defined criteria could proceed to larger comparisons.
The essential distinction is between screening evidence and confirmatory evidence. Early safety, viral RNA, or symptom findings can help decide whether further evaluation is justified. They do not, by themselves, establish that treatment prevents hospitalization or death. A later, larger comparison can estimate clinical outcomes more precisely, but it remains subject to its own design, conduct, and interpretation.
ACTIV-2 (A5401) was an outpatient platform evaluation of investigational COVID-19 therapies. Its master protocol allowed candidate interventions to be assessed through randomized comparisons, generally against placebo, while the platform could evaluate more than one candidate over time. Phase 2 assessments included safety and measures of antiviral activity, such as SARS-CoV-2 RNA, as well as clinical measures such as symptoms. Larger phase 3 comparisons focused on clinical outcomes, including progression to severe illness such as hospitalization or death. The exact eligibility criteria, endpoints, analysis populations, and progression rules depended on the protocol version and the candidate being evaluated.
What an Early Signal Can Tell Us
A phase 2 result is useful only when the outcome and comparison are clear. For a binary outcome—such as viral RNA below a specified threshold at a particular visit—let \(\hat{p}_T\) be the observed proportion in the treatment group and \(\hat{p}_C\) the observed proportion in the control group. The difference \(\hat{p}_T-\hat{p}_C\) estimates the absolute difference in outcome proportions. Its meaning depends on how the outcome is defined: a higher proportion may be favorable for viral clearance but unfavorable for an adverse event.
The usual approximate standard error below assumes independent randomized groups and a binary outcome measured consistently in both groups. The normal approximation is most reliable when each group has enough outcome events and non-events; sparse data require methods designed for small counts. Randomization supports a treatment-group comparison, but it does not repair biased outcome measurement, substantial differential missingness, or deviations from the planned analysis.
An approximate 95% confidence interval is the observed difference plus or minus 1.96 standard errors. It describes the range of values compatible with the data under the approximation and model assumptions; it is not a 95% probability statement about the fixed treatment effect. In a screening stage, the interval helps show both the direction and the uncertainty of an early signal. A protocol may use other prespecified rules as well, including safety review, futility boundaries, or an endpoint-specific threshold.
Worked Example: A Hypothetical Phase 2 Viral-Activity Signal
Hypothetical data: Suppose 34 of 50 participants assigned to an outpatient therapy and 20 of 50 assigned to placebo have viral RNA below a prespecified threshold at a scheduled assessment. Assume participants are independently randomized, the outcome is measured at the same visit in both groups, and the analysis includes all participants with outcome data under a prespecified approach.
The observed proportions are \(\hat{p}_T=34/50=0.68\) and \(\hat{p}_C=20/50=0.40\). The estimated difference is \(0.68-0.40=0.28\), or 28 percentage points in favor of treatment for this defined virologic outcome.
The approximate 95% interval is \(0.28\pm1.96(0.0957)\), which is \(0.28\pm0.1875\), or approximately \((0.0925,0.4675)\). Under the stated assumptions, these data suggest a favorable early signal on the measured virologic outcome. The interval is wide, and the result does not establish that participants feel better or are less likely to be hospitalized. Whether this signal warrants progression depends on the protocol’s prespecified criteria and the full safety and clinical evidence.
Safety Is Part of the Gate
A favorable activity estimate cannot be considered separately from safety. Early-stage trials may be too small to characterize uncommon harms: observing no serious adverse events does not show that the true risk is zero. Safety review therefore considers the event types, timing, severity, exposure, and comparison group, as well as whether further monitoring or stopping is appropriate. It is not simply a count of whether any event occurred.
Worked Example: Interpreting Zero Observed Events
Hypothetical data: Suppose no serious adverse events are observed among 50 participants who received a candidate therapy. For a simple illustration, assume each participant has the same independent probability \(p\) of experiencing the event during the specified follow-up, and ascertainment is complete. The probability of seeing zero events is \((1-p)^{50}\).
A one-sided 95% upper confidence bound solves \((1-p)^{50}=0.05\). Thus \(p=1-0.05^{1/50}\approx0.0582\), or about 5.8%. The observation of zero events is compatible, under this model, with an event risk as high as roughly 5.8% at this confidence level. It is reassuring only in a limited sense; it cannot rule out clinically important uncommon harms.
In practice, safety summaries account for the event definition, observation period, participant exposure, and the comparison group. This simple calculation is not a substitute for the trial’s planned safety analyses.
ACTIV-2’s staged approach made safety and early activity relevant to whether a candidate should move forward. A progression decision is a governance decision informed by evidence, not a mechanical declaration that a drug works. A candidate may stop because of an unfavorable safety profile, insufficient activity, operational changes, or other prespecified considerations. Conversely, proceeding means that the evidence justified further testing under the protocol, not that clinical benefit had already been demonstrated.
How the Larger Comparison Changes the Question
In a larger phase 3 comparison, the central question shifts from whether early evidence merits further evaluation to whether the intervention improves a clinically meaningful outcome enough to matter. For outpatient COVID-19, progression to severe disease is more directly patient-relevant than a laboratory measurement, but such outcomes may be uncommon. The number of participants and the follow-up period must therefore be adequate for the event rate and effect the study is designed to detect.
The absolute risk difference is often useful for clinical interpretation. If \(\hat{p}_T\) is the observed proportion experiencing an unfavorable outcome in the treatment group and \(\hat{p}_C\) the corresponding control proportion, a negative \(\hat{p}_T-\hat{p}_C\) favors treatment. The standard error calculation is the same as above, but interpretation now concerns the clinical endpoint and its units—for example, percentage points of participants hospitalized or dead by day 28. A risk ratio may also be reported, but relative effects alone can obscure the absolute event reduction.
Worked Example: A Hypothetical Phase 3 Clinical Outcome
Hypothetical data: Suppose 15 of 500 treatment-group participants and 30 of 500 placebo-group participants are hospitalized or die by day 28. Assume independent randomized groups, complete and comparable outcome ascertainment, and a prespecified analysis. The event risks are \(15/500=0.030\) and \(30/500=0.060\).
The estimated treatment-minus-placebo risk difference is \(0.030-0.060=-0.030\), a 3.0 percentage-point lower observed risk in the treatment group. The estimated relative risk is \(0.030/0.060=0.50\), a 50% relative reduction in the observed risk.
The approximate 95% interval for the risk difference is \(-0.030\pm1.96(0.01308)\), or approximately \((-0.0556,-0.0044)\). Equivalently, the estimated absolute risk reduction is 3.0 percentage points, with an approximate interval from 0.44 to 5.56 percentage points. This hypothetical result is compatible with a lower 28-day event risk under treatment, subject to the assumptions and analysis used. It does not imply that every patient benefits, and the relative reduction should be read alongside the absolute risks.
Why Stage Transitions Need Careful Interpretation
A seamless platform can make evaluation more efficient, but “seamless” should not be taken to mean that all phase 2 observations automatically count as confirmatory phase 3 evidence. Whether early participants contribute to a later primary analysis, and how the transition is handled statistically, must be specified in the protocol. If the same data influence both a decision to expand and a final test, ordinary analyses may not have their advertised error rates unless the design accounts for that adaptation.
The distinction matters because a screening result can be selected precisely for being promising. Estimates selected this way tend to overstate effects on average, a form of selection bias sometimes called the winner’s curse. Phase 3 therefore needs a valid comparison and a prespecified analysis, with appropriate control of type I error where the design combines stages or makes multiple comparisons. A favorable phase 2 estimate is a reason to learn more, not a guarantee that the estimate will replicate.
Several diagnostic questions help identify trouble in a staged evaluation:
- Was the endpoint fixed and measured consistently? Changes in visit timing, assay procedures, or outcome definitions can make stage-to-stage results difficult to compare.
- Were safety data sufficiently mature? Limited follow-up or small exposure may miss delayed or uncommon harms.
- Did the control group represent the same context? Changes in standard care, circulating variants, vaccination, or background event rates can alter outcomes over calendar time.
- Were missing outcomes or departures from treatment balanced? Differential loss to follow-up can bias a comparison even after randomization.
- Were many candidates or endpoints considered? Chance favorable findings become more likely as the number of comparisons grows, making prespecified decision rules and transparent reporting important.
A stable concurrent placebo group can help preserve interpretability when several candidates are evaluated, but it does not remove every challenge. Shared controls require careful accounting for calendar time and eligibility, and separate candidate comparisons should not be treated as if they were direct head-to-head randomized comparisons unless they were designed that way.
Comparison with Other Evaluation Strategies
| Approach | Primary strength | Key limitation |
|---|---|---|
| Small phase 2 screening followed by larger evaluation | Limits investment in candidates with weak early activity or concerning safety findings | Early estimates are imprecise and can exaggerate effects that trigger progression |
| One large trial for every candidate from the outset | Can provide direct, adequately powered clinical-outcome evidence without a screening transition | Requires substantial resources even for candidates with little activity or unacceptable harms |
| Uncontrolled early study | Can describe feasibility, tolerability, and outcome patterns quickly | Cannot reliably separate treatment effects from natural recovery, selection, or changes in care |
ACTIV-2’s staged platform approach combined the efficiency of early screening with randomized comparisons. Its trade-off is that different stages answer different questions and need not support the same strength of conclusion. The phase 2 experience is useful for deciding what to investigate next; the larger comparison is needed to estimate clinical effects with greater precision.
Common Mistake and Key Takeaway
Wrong reasoning: “The phase 2 treatment group had better viral results, so the therapy was proven to prevent hospitalization.” This substitutes an early biological outcome for a later clinical outcome and skips the uncertainty in the small-stage estimate.
Correction: State exactly which outcome was measured, in which participants, and at what time. Treat a phase 2 signal as evidence about that endpoint and as input to a prespecified progression decision. Make claims about hospitalization or death only from an appropriate clinical-outcome comparison, with its uncertainty and design limitations made clear.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Why can an early viral RNA signal justify further evaluation without establishing a reduction in hospitalization?
- In a trial, 24 of 80 treatment participants and 16 of 80 control participants meet a favorable binary endpoint. Calculate the observed difference in proportions, treatment minus control.
- What does observing zero serious adverse events in a small treatment group fail to establish?
- Why should a reader check whether phase 2 participants are included in the phase 3 primary analysis?
- Name two reasons that a phase 3 event rate might differ from the rate observed during phase 2 recruitment.
Solutions. (1) Viral RNA is a virologic outcome, not hospitalization; a favorable result may indicate antiviral activity but does not prove a clinical benefit. (2) The proportions are \(24/80=0.30\) and \(16/80=0.20\), so the difference is \(0.30-0.20=0.10\), or 10 percentage points in favor of treatment for this endpoint; uncertainty would require a standard error or interval. (3) It does not establish zero risk or exclude uncommon or delayed harms; the sample and follow-up may be too limited. (4) If data used to guide expansion also enter the final analysis, the design and analysis must account for that adaptation to support valid inference. (5) Examples include changes in circulating variants, vaccination, standard care, eligibility, or calendar time.