The Surrogate Paradox in Arm Selection

Why an Improved Intermediate Outcome Can Mislead

In a multi-arm, multi-stage trial, an intermediate outcome may be available sooner than the final time-to-event endpoint. It can therefore inform decisions about which arms continue. But an arm can improve the intermediate outcome and still worsen the final endpoint. This reversal is called the surrogate paradox. It matters because an intermediate measure can be associated with the final outcome without reliably carrying the treatment effect on that outcome.

For clarity, suppose larger values of the intermediate outcome \(S\) are favorable. Let \(a=0\) denote control and \(a=1\) an experimental arm. For a time-to-event final endpoint, use restricted mean survival time (RMST) through a prespecified horizon \(\tau\): each participant’s survival time is truncated at \(\tau\), and the arm-level mean is the average of those truncated times. Define the treatment effects as the experimental-arm mean minus the control-arm mean. Thus, a positive RMST effect is favorable.

The paradox is about causal treatment effects, not simply a discrepancy between two observed averages. In a randomized comparison, randomization supports estimating the average effect of assignment on each endpoint. It does not, by itself, show that changing \(S\) would change \(R\), or that an effect on \(S\) predicts the effect on \(R\).

A Direct-Harm Mechanism

One route to reversal is that treatment improves the intermediate outcome through one pathway but harms the final endpoint through another. The following additive model makes the mechanism explicit. Suppose the treatment changes the intermediate outcome by \(\gamma\), and suppose the final RMST has a direct treatment component \(\theta\) and a component \(\beta\) per unit of \(S\):

Here \(R(a,s)\) is the mean final outcome under treatment \(a\) if \(S\) were set to \(s\). The effect mediated through the intermediate outcome is \(\beta\gamma\); the direct component is \(\theta\). Substitution gives the total treatment effect:

If \(\gamma>0\) and \(\beta>0\), the intermediate-mediated component is beneficial. Nevertheless, the total effect is harmful whenever \(\theta<-\beta\gamma\). The model assumes an additive relationship with no treatment-by-intermediate interaction; the equation is not a general identity for arbitrary data. It shows precisely how direct harm can outweigh mediated benefit when those assumptions are defensible.

Heterogeneity Can Also Produce a Reversal

An average improvement in \(S\) does not mean that every subgroup improves, and subgroup effects on \(S\) and the final endpoint need not have the same magnitude. Even if effects point in the same direction within each subgroup, averaging the effects can produce a reversal when the magnitudes differ. The next example uses equally sized subgroups so that the overall effects are simple averages.

When Can the Reversal Be Ruled Out?

A guarantee against reversal requires assumptions connecting treatment, the intermediate outcome, and the final endpoint. A useful strong sufficient condition is individual-level monotonicity: treatment does not lower the intermediate outcome for any participant, it does not directly reduce the final outcome at a fixed intermediate value, and a higher intermediate value does not reduce the final outcome under treatment. To state this precisely, let \(R_i(a,s)\) be participant \(i\)’s potential final outcome under treatment \(a\) and intermediate value \(s\), on the RMST scale. Assume for every participant and relevant \(s\):

• \(S_i(1)\ge S_i(0)\): treatment does not reduce the intermediate outcome.
• \(R_i(1,s)\ge R_i(0,s)\): at a fixed intermediate value, treatment does not harm the final outcome.
• \(R_i(1,s)\) is nondecreasing in \(s\): raising the intermediate outcome under treatment cannot reduce the final outcome.

Then treatment cannot reduce the final outcome for any participant. The proof compares outcomes in two steps: first raise the intermediate outcome under treatment, then compare treatment with control while holding the intermediate value fixed.

The first inequality uses \(S_i(1)\ge S_i(0)\) and monotonicity of \(R_i(1,s)\). The second uses the nonnegative direct effect at \(s=S_i(0)\). Taking expectations preserves the inequalities, so \(E\{R_i(1,S_i(1))\}\ge E\{R_i(0,S_i(0))\}\), and \(\Delta_R\ge0\). This is a sufficient condition, not a necessary one. It is also much stronger than observing a positive average effect on \(S\).

A less restrictive but still substantive guarantee follows from the additive model. If its assumptions hold, \(\gamma>0\), \(\beta\ge0\), and \(\theta\ge0\), then \(\Delta_R=\theta+\beta\gamma\ge0\). A homogeneous slope \(\beta\) and additive effects make the average decomposition straightforward. With interactions or heterogeneous effects, an average intermediate benefit and an average nonnegative direct effect do not, by themselves, guarantee a nonnegative total effect: the association between individual intermediate changes and their final-outcome effects also matters.

Assumptions, Diagnostics, and Limits

To interpret an intermediate-outcome relationship causally, one needs more than a regression of final outcome on intermediate outcome. For example, participants with better intermediate outcomes might also have better underlying prognosis. That association can make \(S\) predictive of \(R\) without showing that intervening to improve \(S\) would improve \(R\). A mediation interpretation typically requires consistency (observed outcomes agree with the potential outcomes under the treatment and intermediate value received), no interference between participants, suitable positivity, and adequate control of confounding of the intermediate–final relationship. Natural direct and indirect effect interpretations can require additional, untestable cross-world assumptions; they also require that treatment not create an uncontrolled confounder of the mediator–outcome relationship. Randomization of treatment alone does not supply these assumptions.

A practical diagnostic begins by checking the direction and scale of both effects, including whether a larger intermediate value is actually favorable. Then examine whether subgroup effects, plausible direct pathways, or intermediate-outcome measurement problems could undermine the proposed connection. For a time-to-event endpoint, the RMST horizon must be stated: effects through one horizon need not represent effects through another. Estimates are uncertain as well; a pair of point estimates with opposite signs does not establish that the true effects have opposite signs.

Intermediate-based ranking or screening is therefore not equivalent to comparing arms on the final endpoint. A predictive relationship can help forecast final outcomes, while a causal surrogate claim asks the stronger question of whether treatment effects on the intermediate outcome reliably capture treatment effects on the final outcome. When the assumptions needed to rule out reversal are not credible, the intermediate outcome can still inform an interim decision, but a favorable result should not be interpreted as proof of final-endpoint benefit. The final outcome and uncertainty in the intermediate–final relationship remain important.

The monotonicity and additive conditions above are sufficient conditions, not routine facts. They may fail if treatment has toxicity, if some participants’ intermediate outcomes worsen, if the intermediate outcome has different consequences across participants, or if effects change with treatment or intermediate level. No simple pattern in an observed trial can verify every individual-level potential-outcome assumption. The conditions are useful because they expose exactly what must be believed to rule out the paradox, not because they are automatically satisfied in MAMS trials.

Common Mistake