Prediction Is Not the Same as Surrogacy
An intermediate outcome can be strongly associated with a participant’s later event and still be a poor basis for choosing which treatment arm to continue. These are different questions. The first asks whether participants with different intermediate outcomes tend to have different final outcomes. The second asks whether treatment effects on the intermediate outcome reliably indicate which treatment will improve the final outcome. Arm selection depends on the second question.
For notation, let \(Z\) be an intermediate outcome measured before the final endpoint, with larger values defined as favorable. Let \(Y_\tau\) indicate whether the final event has occurred by a clinically specified time \(\tau\), so \(Y_\tau=1\) denotes an event and lower event risk is favorable. In a MAMS trial, let \(a\) denote an active arm and \(0\) the shared control. This binary event notation is a way to explain the distinction; the same reasoning applies to other final-outcome summaries, including appropriately defined time-to-event measures.
This distinction extends the earlier material on prognostic association with the final endpoint. Prognostic information can help predict outcomes for participants, but selection requires evidence about how randomized treatment contrasts behave. In the notation above, an arm may improve \(Z\) relative to control without reducing final-event risk. That is why a predictor of prognosis is not automatically a valid surrogate for choosing treatment.
Why Within-Arm Association Can Mislead
The distinction can be made precise using the law of total covariance. Let \(A\) denote randomized treatment assignment. The overall association between \(Z\) and \(Y_\tau\) can be decomposed into an average of within-arm associations and an association between arm-specific means:
To see why, write each variable as its deviation from its arm-specific mean plus that mean. When multiplying the deviations and taking expectations, the cross-terms have expectation zero conditional on \(A\). The remaining terms are the average within-arm covariance and the covariance of arm-specific means, giving the displayed identity.
The first term can show that participants with favorable intermediate outcomes tend to have fewer final events within arms. The second term concerns differences between arms. If arms with higher mean \(Z\) also have higher final-event risk, this between-arm term goes in the unfavorable direction and can weaken or reverse the overall association. More importantly for selection, the within-arm term does not determine the between-arm pattern. This is the core reason that a prognostic intermediate outcome can mislead arm selection.
Individual-level association should be examined with care. A pooled association across all arms can partly reflect treatment assignment or differences in baseline risk rather than a relationship within comparable participants. Conversely, conditioning on a post-randomization intermediate outcome does not generally estimate a causal treatment effect: participants with the same observed \(Z\) in different arms need not be comparable. Predictive association is informative, but it is not proof that changing \(Z\) will change the final endpoint.
Worked Example: Strong Prognostic Association but No Final Benefit
Consider a hypothetical randomized comparison with 1,000 participants per arm. The intermediate outcome is early response, and the final outcome is an event by a specified follow-up time. In control, 300 participants respond; 30 responders and 280 nonresponders have the final event. In the active arm, 600 respond; 120 responders and 190 nonresponders have the event.
In control, the event risks are \(30/300=0.10\) among responders and \(280/700=0.40\) among nonresponders. In the active arm, they are \(120/600=0.20\) and \(190/400=0.475\), respectively. Response is prognostic within each arm: responders have lower event risk than nonresponders.
The active-arm response rate is \(600/1000=0.60\), compared with \(300/1000=0.30\) in control, a difference of \(0.60-0.30=0.30\), or 30 percentage points. But the overall event risks are \(310/1000=0.31\) in both arms: \((30+280)/1000=0.31\) and \((120+190)/1000=0.31\). Thus, selection based on the response rate would favor the active arm, while this example shows no reduction in final-event risk.
The example does not imply that this pattern is typical, nor does comparing event risks within observed response groups establish a causal mechanism. It demonstrates the narrower point: a strong within-arm prognostic relationship is compatible with an intermediate treatment effect that fails to reflect the final treatment effect.
Arm Ranking Is the Selection-Relevant Question
For a particular active arm \(a\), define the intermediate contrast as the difference in mean intermediate outcome and the final contrast as the difference in final-event risk:
With the stated outcome directions, a positive \(\Delta_Z(a)\) is favorable, while a negative \(\Delta_Y(a)\) is favorable. Selection based on \(Z\) is most useful when the contrasts on \(Z\) preserve the ordering of the arms’ effects on the final endpoint. Individual participants’ \(Z\)-to-\(Y_\tau\) association does not establish that ordering.
Worked Example: A Reversed Ranking of Three Arms
Suppose a hypothetical MAMS trial has three active arms and a common control. The table gives the response rate and final-event risk by arm. Assume that participant-level summaries within each arm show lower final-event risk among responders than among nonresponders.
| Arm | Response rate | Final-event risk |
|---|---|---|
| Control | 40% | 30% |
| A | 70% | 31% |
| B | 55% | 24% |
| C | 45% | 20% |
Relative to control, the response-rate differences are \(70\%-40\%=30\) percentage points for A, \(55\%-40\%=15\) points for B, and \(45\%-40\%=5\) points for C. Ranking by this intermediate outcome gives A, then B, then C.
The final-event risk differences are \(31\%-30\%=1\) percentage point for A, \(24\%-30\%=-6\) points for B, and \(20\%-30\%=-10\) points for C. Since lower risk is favorable, the final-outcome ranking is C, then B, then A. In this hypothetical result, selecting the arm with the largest response-rate difference would select A, even though its final-event risk is slightly higher than control. The stated within-arm prognostic association does not prevent this reversal.
In an actual trial, these are estimated contrasts with sampling error, not known population quantities. The example isolates the direction of the problem; it does not provide a general selection rule or account for the uncertainty introduced by selecting among arms.
Trial-level surrogacy is assessed from randomized treatment contrasts, not by treating every participant as an independent piece of evidence about how treatment effects relate. A single trial can provide several active-versus-control contrasts, but they share a control group and are statistically dependent. They are also drawn from the same trial context. A strong relationship among participants in that trial is not equivalent to evidence that the intermediate contrasts predict final contrasts in other treatment comparisons or settings.
Worked Example: Contrasts Across Hypothetical Trials
Consider three hypothetical randomized trials, each comparing one treatment with its own control. The intermediate outcome is response rate; the final outcome is event risk by the same follow-up horizon. Let \(\Delta_Z\) be the response-rate difference and let \(\Delta_Y\) be the treatment-minus-control event-risk difference.
| Trial | Control response | Treatment response | Control event risk | Treatment event risk |
|---|---|---|---|---|
| 1 | 40% | 60% | 30% | 27% |
| 2 | 35% | 55% | 25% | 29% |
| 3 | 50% | 65% | 32% | 24% |
For trial 1, \(\Delta_Z=60\%-40\%=20\) percentage points and \(\Delta_Y=27\%-30\%=-3\) points. For trial 2, \(\Delta_Z=55\%-35\%=20\) points but \(\Delta_Y=29\%-25\%=4\) points, an unfavorable final contrast. For trial 3, \(\Delta_Z=65\%-50\%=15\) points and \(\Delta_Y=24\%-32\%=-8\) points.
All three treatments improve response, but the final contrasts do not move consistently in the same favorable direction. These few hypothetical comparisons cannot validate or reject a general surrogate relationship; they show what trial-level evidence concerns: the pattern of treatment contrasts, rather than the association between response and event among participants. Reliable general conclusions require enough appropriate randomized comparisons and attention to how their populations, interventions, endpoint definitions, and follow-up differ.
Diagnostics, Alternatives, and Limitations
Before using an intermediate outcome to guide selection, examine both levels of evidence and the conditions that could distort either. Useful warning signs include:
- A mismatch in arm rankings. Arms that look most favorable on \(Z\) do not consistently look favorable on the final endpoint, or the ranking changes substantially across analyses.
- Association driven by pooling. The relationship between \(Z\) and the final endpoint weakens, disappears, or reverses when examined within arms or after accounting for important baseline prognostic factors.
- Dependence on follow-up or outcome definition. Intermediate and final outcomes are measured at different horizons, or event ascertainment and censoring differ enough to make contrasts hard to compare.
- Uncertain estimates or incomplete outcomes. Few events, small samples, missing intermediate measurements, and immature follow-up can make apparent treatment-effect rankings unstable.
- Limited evidence across comparisons. One MAMS trial, particularly with a shared control, offers limited independent information about how well intermediate effects predict final effects more broadly.
The principal alternative is to select using the final endpoint itself when it is sufficiently mature and precisely estimated at the selection look. This avoids relying on an unverified surrogate relationship, but may be impractical when final events accrue slowly. An intermediate outcome can still be useful as a predictor or as one component of a prespecified decision process; its use should be justified by evidence about treatment contrasts and by the consequences of a wrong selection. Selection uncertainty and the design’s operating characteristics also matter, but an association at the participant level cannot substitute for them.
No single measure of individual association can establish that an intermediate endpoint captures the treatment effects relevant to the final endpoint. Nor can a favorable intermediate contrast guarantee a favorable final contrast in every arm. The distinction is especially important when biological pathways differ across treatments, when treatments affect the final endpoint through routes not reflected in \(Z\), or when intermediate measurement is noisy or incomplete. These limitations do not make intermediate outcomes useless; they make the claim being made more specific.
Common Mistake
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions.
- In a randomized trial, responders have lower event risk than nonresponders within both arms. Does that establish that the arm with the larger response-rate increase will reduce event risk more? Why or why not?
- For an arm, the event risk is 22% and the control event risk is 28%. Calculate \(\Delta_Y\) and interpret its direction.
- What is the between-arm component in the total covariance decomposition, and why can it matter for selection?
- Why should three active-versus-control comparisons from one MAMS trial with a shared control not be treated as three fully independent trials when assessing trial-level surrogacy?
Solutions. (1) No. The within-arm association is individual-level prognostic evidence; it does not establish that treatment effects on response track treatment effects on event risk. (2) \(\Delta_Y=22\%-28\%=-6\) percentage points, a 6-point lower event risk in the active arm in this hypothetical comparison. (3) It is \(\operatorname{Cov}(E[Z\mid A],E[Y_\tau\mid A])\), the association between arm-specific means. It can oppose the within-arm association and reveal a treatment-effect pattern inconsistent with participant-level prediction. (4) The contrasts share the same control observations, so their estimates are statistically dependent; they also come from one trial context rather than independent settings.