Why Disease Setting Changes the Interpretation
STAMPEDE evaluated treatments in men with prostate cancer at different points in the disease course. In this setting, metastatic disease means cancer that has spread to distant sites at trial entry; nonmetastatic disease means distant metastases were not detected at entry. These groups differ in prognosis and clinical context. Consequently, an outcome reported in one setting is not automatically the right outcome, or the right estimate, for the other.
The question here is not simply whether an experimental treatment “worked.” It is what was measured, in which randomized comparison, and for which disease setting. For example, the STAMPEDE abiraterone comparison in metastatic disease reported an overall-survival benefit. In nonmetastatic disease, the pooled abiraterone comparisons reported benefits in metastasis-free survival and overall survival. These findings are related, but they come from different populations and trial comparisons, and the endpoints and follow-up summaries are not interchangeable.
Read the Endpoint Before the Estimate
Overall survival (OS) measures time until death from any cause. Metastasis-free survival (MFS) measures time until distant metastasis or death, as defined in the trial analysis. Failure-free survival (FFS) uses the trial’s specified definition of failure, which may include events other than death or distant metastasis. The event definition matters: an effect on FFS does not automatically mean the same effect on OS.
A hazard ratio (HR) compares event hazards over follow-up. For an endpoint where events are unfavorable, an HR below 1 favors the treatment arm, subject to the trial’s analysis and assumptions. It is a relative comparison of hazards, not a ratio of the percentages alive at a particular time. A confidence interval (CI) describes uncertainty in the estimated effect; if it includes 1, the data are compatible with no hazard difference as well as with effects in either direction allowed by the interval.
Absolute survival estimates answer a different, clinically useful question: what proportion of patients in each arm were alive, or free of a specified event, at a particular time? These estimates depend on follow-up and the population’s baseline risk. They should be reported with their time point and endpoint, not substituted for an HR.
Worked Example: Abiraterone in Metastatic Disease
In the STAMPEDE metastatic-disease comparison, adding abiraterone to standard treatment was reported to improve OS, with an HR of 0.63 and a 95% CI of 0.52 to 0.76. The reported 3-year OS estimates were 83% with abiraterone and 76% with the comparison treatment.
Relative interpretation. The estimated relative reduction in the death hazard is \(1-0.63=0.37\), or 37%. Transforming the CI limits in the same way gives \(1-0.76=0.24\) and \(1-0.52=0.48\). Thus, the interval corresponds to a relative hazard reduction of approximately 24% to 48%, under the model used for the HR.
Absolute interpretation. At 3 years, the difference between the reported OS estimates is \(83\%-76\%=7\) percentage points. This is a time-specific difference in estimated survival, not a 37-percentage-point gain and not a guarantee for every patient. The HR and the 3-year difference summarize the outcome in different ways.
Scope of the conclusion. These numbers support an OS benefit in the metastatic comparison. They do not establish the size of the effect in nonmetastatic disease, nor do they show that the effect differs between settings.
Worked Example: Abiraterone in Nonmetastatic Disease
In pooled STAMPEDE comparisons of abiraterone-based treatment in nonmetastatic disease, the reported MFS HR was 0.53 (95% CI 0.44 to 0.64). Reported 6-year MFS estimates were 82% in the abiraterone group and 69% in the comparison group.
The estimated relative reduction in the MFS event hazard is \(1-0.53=0.47\), or 47%. The CI corresponds to reductions from \(1-0.64=0.36\), or 36%, to \(1-0.44=0.56\), or 56%. Because MFS includes metastasis or death according to the trial definition, this is not an estimate of the effect on death alone.
The 6-year absolute difference is \(82\%-69\%=13\) percentage points in MFS. In plain terms, the estimated proportion alive without a recorded MFS event at that time was 13 percentage points higher in the abiraterone group. This summary is specific to the nonmetastatic comparisons and the reported time point.
The nonmetastatic analyses also reported an OS benefit; the pooled OS HR was 0.60 (95% CI 0.48 to 0.73). It is appropriate to report that result as evidence about OS in those comparisons. It would be inaccurate to treat the MFS HR of 0.53 as if it were the OS HR, or to assume that the metastatic and nonmetastatic estimates came from one identical trial comparison.
Comparing Settings Without Overclaiming
A useful comparison begins by laying out the population, intervention, comparator, endpoint, effect estimate, and follow-up for each result. Only after those are clear should the estimates be discussed side by side. In particular, comparisons are difficult when endpoint definitions differ, standard care changes, or populations enter the trial with different disease burden and baseline risk.
| Question | Metastatic comparison | Nonmetastatic comparison |
|---|---|---|
| What does the setting describe? | Distant metastases detected at entry | Distant metastases not detected at entry |
| Example abiraterone result | OS HR 0.63 (95% CI 0.52 to 0.76) | MFS HR 0.53 (95% CI 0.44 to 0.64); OS was also reported |
| Absolute summary cited here | 3-year OS: 83% versus 76% | 6-year MFS: 82% versus 69% |
| What the result supports | An OS treatment effect in the metastatic comparison | An MFS treatment effect in the nonmetastatic comparisons |
| What it does not establish alone | The treatment effect in nonmetastatic disease | The treatment effect in metastatic disease or an interaction between settings |
The table deliberately places an OS result beside an MFS result to show why a superficial numerical comparison is unsafe. The HRs, 0.63 and 0.53, are not directly comparable measures of the same endpoint. Even when the same endpoint is available in both settings, comparing two point estimates is not a formal test that the effects differ.
Worked Comparison: Testing for a Difference in Effects
The following numbers are hypothetical and are not STAMPEDE results. Suppose the same treatment and endpoint were evaluated in two independent disease-setting groups, producing HR 0.70 (95% CI 0.54 to 0.91) in metastatic disease and HR 0.86 (95% CI 0.66 to 1.12) in nonmetastatic disease. A direct comparison can be made on the log-HR scale. The interaction contrast is the difference between the two log HRs, equivalently the log of their ratio.
Assume the two estimates are approximately normally distributed on the log scale and independent, and use the rounded 95% CIs to recover approximate standard errors. For a 95% CI \((L,U)\) on the HR scale, the approximate standard error of the log HR is \([\log(U)-\log(L)]/(2\times1.96)\).
Under independence, the variance of a difference is the sum of the variances. Thus the standard error of the interaction contrast is:
The approximate test statistic is \(z=-0.2059/0.1895=-1.086\), giving a two-sided \(p\)-value of about 0.278. The 95% CI for the log contrast is \(-0.2059\pm1.96(0.1895)\), or approximately \((-0.577,0.166)\). Exponentiating gives a ratio of HRs of 0.814, with an approximate 95% CI of 0.561 to 1.181.
The interval includes 1, so this example does not provide strong evidence that the relative treatment effects differ by setting. It also does not prove that the effects are identical: the interval allows meaningful differences in either direction. This calculation is only appropriate under the stated independence and approximate normality assumptions; a trial’s prespecified interaction analysis is preferable when available.
Diagnostics, Alternatives, and Limitations
Before drawing a cross-setting conclusion, check whether the comparisons really address the same question. Review the population definitions, randomization, treatment and control regimens, endpoint definitions, follow-up, and analysis populations. Check whether the reported CIs are for the same effect measure and whether an interaction estimate and CI are reported. An HR for OS should not be combined or contrasted casually with an HR for MFS.
When an interaction estimate is available, report its effect measure and uncertainty, rather than relying on whether each setting-specific result separately reaches a significance threshold. “Significant in one setting but not significant in the other” does not imply a significant difference between settings. Conversely, a nonsignificant interaction test does not establish equal effects; it may simply be imprecise.
Alternative summaries answer different questions. Setting-specific HRs describe relative event rates over follow-up under the analysis model. Time-specific survival differences describe absolute outcomes at a selected time. Restricted mean survival time, when reported, summarizes average time alive or event-free through a prespecified horizon. None should be substituted for another without explanation. If the clinical decision depends on absolute benefit, the time point and baseline risk are especially important.
Cross-setting comparisons also have limits even when the statistical calculation is correct. Disease setting may be associated with other differences in prognosis, eligibility, treatment, or calendar period. A difference between trial cohorts can therefore reflect more than a biological change in treatment effect. In a MAMS platform, comparisons may also involve arms added at different times or different control-arm experience; the earlier material on STAMPEDE’s control arm across calendar time is relevant when assessing whether controls are contemporaneous and comparable.
The most defensible summary is therefore specific: state what treatment was compared, in which setting, for which endpoint, and with what uncertainty. Then distinguish evidence of benefit within each setting from evidence that benefit differs between settings.
Check Your Understanding
Answer each question using the endpoint, setting, and uncertainty in the result. Brief solutions follow.
- An HR of 0.63 for OS is reported in a metastatic comparison. What relative hazard reduction does it estimate, and what does it not mean?
- For nonmetastatic disease, the 6-year MFS estimates are 82% and 69%. Find the absolute difference and name the endpoint.
- Why is “significant in one setting, nonsignificant in another” not a test of treatment-effect heterogeneity?
- In the hypothetical comparison, the ratio of HRs is 0.814 with a 95% CI of 0.561 to 1.181. What conclusion is justified?
- Name two checks needed before comparing outcomes across disease settings.
Solutions. (1) It estimates a 37% relative reduction in the death hazard under the analysis model, not a 37-percentage-point increase in survival or a probability that an individual patient benefits. (2) \(82\%-69\%=13\) percentage points in 6-year MFS. (3) Separate tests assess each setting’s effect against its own null; heterogeneity concerns the difference between effects and requires a direct interaction contrast. (4) The interval includes 1, so there is not strong evidence of different relative effects; the wide interval does not establish equality. (5) Examples include confirming the same endpoint definition and effect measure, checking treatment and control regimens, and comparing follow-up and population eligibility.