Zoledronic Acid's Negative Result in STAMPEDE

Why a large null result needs careful reading

A large randomized trial can provide strong evidence against a treatment benefit that is sufficiently large and precise enough to be excluded by its confidence interval. It cannot, merely by reporting a non-significant test, prove that the treatment has no effect. The zoledronic acid comparison in STAMPEDE is a useful case: its survival result was close to no difference, while its interval still allowed both some benefit and some harm.

STAMPEDE evaluated several additions to long-term androgen-deprivation therapy (ADT) for men with prostate cancer. In the zoledronic acid comparison, the clinically important questions were distinct: did adding zoledronic acid improve overall survival, and did it reduce skeletal-related events? A null result for one outcome does not automatically answer the other.

The STAMPEDE comparison and its outcomes

STAMPEDE was a multi-arm, multi-stage platform trial. In the comparison of interest, men were assigned to standard care or to standard care with zoledronic acid. The drug was a bone-targeted treatment; the primary outcome for the reported comparison was overall survival. Skeletal-related events (SREs) were also assessed as a secondary outcome. They concern clinically important complications involving bone, rather than death itself.

Random assignment supports a causal comparison of the assigned treatment strategies, provided the analysis preserves that assignment and outcome ascertainment is sufficiently complete. The reported treatment effect is therefore best read as the effect of being assigned to the zoledronic-acid strategy in the trial population, not as an effect among only patients who completed every planned dose. The survival analysis accounts for differing follow-up times and censoring; its hazard ratio is not a ratio of median survival times.

For overall survival, the reported hazard ratio for zoledronic acid versus standard care was 0.94 (95% confidence interval 0.79 to 1.11; p=0.450). The point estimate is compatible with a modestly lower death hazard, but the interval includes 1 and extends above 1. The test did not provide statistically significant evidence of a survival benefit. For time to first SRE, the reported estimate was also not statistically significant: hazard ratio 0.89 (95% confidence interval 0.73 to 1.07; p=0.217). This does not establish that zoledronic acid has no effect on skeletal events; the interval includes both a potentially meaningful reduction and a small increase in hazard.

These are outcome-specific results. A treatment might fail to improve overall survival yet affect a nonfatal complication, or affect neither outcome detectably. The SRE analysis should not be substituted for the survival analysis, and a survival null does not settle the SRE question.

Reading a hazard ratio and its interval

A hazard is the instantaneous event rate among people who have not yet experienced the event, conditional on the survival model and the observed follow-up. A hazard ratio below 1 favours the treatment with respect to that event; a ratio above 1 favours the comparator. For a single hazard ratio to summarize the contrast over follow-up, proportional hazards is a useful working assumption: the relative hazards are approximately constant over time. If this assumption is seriously violated, one number may conceal time-varying effects, and survival curves or time-specific summaries become important.

The overall-survival estimate 0.94 means the estimated death hazard in the zoledronic acid group was 0.94 times that in the standard-care group under the fitted analysis. It does not mean that 6% fewer patients died, nor that survival time was 6% longer. The confidence interval 0.79 to 1.11 corresponds to effects ranging from a 21% lower hazard to an 11% higher hazard, on the hazard-ratio scale. Since 1 represents equal hazards, the interval does not exclude no difference.

A two-sided confidence interval and test at significance level \(\alpha=0.05\) are linked: when the 95% interval excludes 1, the corresponding two-sided test will generally reject \(H_0\); when it includes 1, the test will generally not reject. Here p=0.450 is far above 0.05, so the result is not statistically significant. The p-value is the probability, assuming the null model and analysis assumptions, of obtaining a test statistic at least as extreme as the observed one. It is not the probability that the null hypothesis is true.

What a large trial’s null finding can establish

A large trial is valuable not because its non-significant result proves zero effect, but because adequate information can make the confidence interval narrow enough to rule out effects of practical importance. The key question is therefore not only “Was p below 0.05?” but also “Which treatment effects remain compatible with the estimate and its uncertainty?” This distinction is especially important when a treatment is costly, burdensome, or associated with adverse effects: deciding whether the remaining possible benefit matters requires clinical judgment as well as statistical evidence.

STAMPEDE’s overall-survival interval does not support a claim that zoledronic acid produced a statistically significant survival improvement. It also does not exclude every potentially useful survival reduction: the lower limit, 0.79, remains compatible with a hazard reduction of about one fifth. Conversely, the upper limit allows a modestly higher hazard. Thus “no statistically significant survival benefit was demonstrated” is supported; “the treatment has no survival effect” is not.

A formal claim of equivalence or non-inferiority would require a prespecified margin defining how large a difference would be clinically unacceptable, as well as a design and analysis intended to evaluate that margin. A conventional superiority test with p=0.450 does not perform that task. Nor does a large sample turn a p-value into a direct measure of clinical importance: precision is read from the interval, while importance depends on the outcome and decision context.

Comparing survival and skeletal-event evidence

The SRE estimate in STAMPEDE was 0.89 with 95% confidence interval 0.73 to 1.07 and p=0.217. Its direction favours zoledronic acid, but its interval crosses 1. It is therefore reasonable to say that the comparison did not demonstrate a statistically significant reduction in time to first SRE. It is not reasonable to translate that into certainty that SRE risk was unchanged. The interval remains compatible with a substantial reduction as well as a small increase in hazard.

SREs and death differ in clinical meaning, frequency, and timing. A time-to-first-event analysis also answers a particular question: how long until the first qualifying event? It does not directly quantify the total number of recurrent events or the burden and severity of each event. When evaluating a secondary outcome, consider its definition, the precision of its estimate, and whether the analysis addresses the clinical quantity of interest.

Design context, diagnostics, and common pitfalls

In a multi-arm platform, the comparison must be identified precisely. A shared standard-care group can contribute to several randomized comparisons, but an estimate for zoledronic acid alone versus standard care is not automatically the estimate for a regimen combining zoledronic acid with another treatment. Pooling arms or interpreting a combination-arm result as the standalone effect requires justification by the design and analysis; it should not be done merely because the arms share a drug.

Before accepting a null result as informative, check whether the estimate and interval are reported for the intended population and outcome, whether follow-up and censoring are adequate, and whether the survival-model assumptions are plausible. Kaplan–Meier curves that separate and later converge, or cross, can signal non-proportional hazards. Also inspect the number of events and the width of the interval: a non-significant estimate with a wide interval is weak evidence against clinically important effects. For secondary outcomes, consider that multiple outcomes are being examined and that a single p-value does not describe the whole evidence base.

A common mistake is to argue: “The p-value is above 0.05, so zoledronic acid has no effect.” The correction is to report the estimated effect, its interval, the test result, and the remaining uncertainty. For STAMPEDE, the survival estimate was close to 1 and its interval included 1; the trial did not demonstrate an overall-survival benefit, but its interval did not prove an exact zero effect. The same discipline applies separately to SREs.

The comparison’s limits should remain visible. A hazard ratio is a relative measure, not an absolute difference in survival probability or event rate. The findings concern the trial’s assigned strategy, participants, background care, follow-up, and endpoint definitions. They do not by themselves establish what would happen in every population or under a different treatment schedule. A clinically useful decision may also depend on absolute risks, adverse effects, competing outcomes, and patient priorities.