Predicting Clinical Treatment Benefit From a Surrogate

From a Biomarker Effect to a Clinical Treatment Prediction

A biomarker can respond strongly to treatment without reliably forecasting how much patients benefit clinically. The relevant prediction question is not whether a biomarker changes, but whether the treatment effects on the biomarker and the clinical outcome are related across randomized trials. Even a strong relationship does not make the biomarker a guaranteed substitute for clinical outcomes: a new treatment may have a clinical effect that differs from the relationship’s average prediction.

This tutorial focuses on predicting a new treatment’s clinical effect using a relationship validated across trials. It assumes that treatment effects have been oriented so that positive values mean improvement for both endpoints. For example, a decrease in a harmful biomarker can be recoded as a positive biomarker benefit. The clinical effect’s scale and units must also be explicit: a risk difference, a mean difference, and a standardized effect are not interchangeable.

A simple model for this relationship is

Here, \(\beta_0+\beta_1\Delta S_i\) is the mean clinical effect associated with a biomarker effect of \(\Delta S_i\), and \(\tau^2\) is the residual between-trial variance around that mean relationship. For a new trial with biomarker effect \(\Delta S_{\mathrm{new}}\), the predicted mean is \(\hat{\beta}_0+\hat{\beta}_1\Delta S_{\mathrm{new}}\). The actual clinical effect may differ from this mean because the new trial has its own residual deviation.

The model is useful only under substantive and statistical conditions. The validation trials should be sufficiently comparable to the intended prediction setting, and the new biomarker effect should lie within or near the range of effects represented in those trials. The relationship should be approximately linear over that range, with no important unmodeled pattern in the residuals. The residual distribution and variance assumptions should be reasonably appropriate if normal-theory prediction intervals are used. Crucially, the evidence must account for uncertainty in the estimated trial effects; a regression that treats noisy effect estimates as exact can misrepresent the relationship and its predictive uncertainty. The earlier material on measurement error in surrogate validation addresses that issue.

Why a Prediction Interval Is Wider Than an Interval for the Mean

An interval for the mean relationship at a specified biomarker effect concerns uncertainty about \(\beta_0+\beta_1\Delta S_{\mathrm{new}}\). A prediction interval concerns the clinical effect in a particular new trial. It must therefore include both uncertainty in that mean and the trial-to-trial residual variation \(\tau^2\). Reporting only an interval for the mean can make prediction look substantially more certain than it is.

For a linear model fitted to \(m\) trials, write \(x_0=(1,\Delta S_{\mathrm{new}})^\mathsf{T}\), and let \(\widehat{\mathrm{Cov}}(\hat{\beta})\) be the estimated covariance matrix of the regression coefficients. The estimated standard error for predicting a new clinical effect is

Under the normal linear-model assumptions, with residual variance estimated from the regression, a prediction interval is the predicted mean plus or minus the appropriate \(t\) critical value times \(SE_{\mathrm{pred}}\). For ordinary least squares with two fitted coefficients, the degrees of freedom are \(m-2\). In a meta-regression that accounts for sampling error and other variance components, the fitted model’s appropriate uncertainty calculation and degrees of freedom should be used rather than mechanically applying the ordinary least-squares formula.

Worked Predictions

Checking Whether the Prediction Is Trustworthy

A numerical prediction is only as useful as the evidence and model behind it. Before applying one, check the following:

• Range: Compare the new biomarker effect with the range observed in the validation trials. A prediction far outside that range is extrapolation; the fitted line may not continue there.
• Residual pattern: Plot trial-level residuals against fitted values and biomarker effects. Curvature, changing spread, or clusters can indicate that the simple linear, constant-variance model is inadequate.
• Influence: Check whether one trial dominates the fitted slope or prediction. Refit after leaving out influential trials and examine how much the prediction changes.
• Predictive performance: Where feasible, assess predictions for trials not used to fit the relationship, for example with leave-one-trial-out validation. Compare predicted and observed effects and assess calibration and interval coverage.
• Effect-estimate uncertainty: Confirm that sampling error in the treatment-effect estimates and relevant correlations has been incorporated. Treating imprecise trial effects as exact can distort the estimated relationship and make uncertainty appear smaller than warranted.

An ordinary regression line can be useful for communicating the basic relationship, but it is not always the best analysis. A meta-analytic or hierarchical regression can represent within-trial sampling error and between-trial variation explicitly. A nonlinear model may be appropriate when the evidence supports curvature, although added flexibility is difficult to justify with few trials. Direct clinical-outcome evidence remains essential when the relationship is weak, the prediction is highly uncertain, or the consequences of a wrong prediction are substantial.

Trial-level prediction is also distinct from showing that biomarker changes mediate an individual patient’s treatment benefit. A relationship across trial treatment effects does not establish an individual-level causal mechanism. Nor does the regression eliminate the possibility that treatment affects the clinical outcome through pathways not captured by the biomarker. These limitations matter especially when a point prediction is presented without its prediction interval or when it is interpreted beyond the trials supporting it.

Common Pitfall: Reporting the Mean as the New Trial’s Effect

Incorrect reasoning: “The regression predicts a clinical benefit of 0.50 percentage points, so the new trial will produce a 0.50-point benefit.” This treats the estimated mean relationship as if every trial lay exactly on the regression line.

Correction: The predicted mean is an average conditional on the model. A new trial can deviate from it by the residual amount represented by \(\tau^2\), in addition to uncertainty in the fitted relationship. Report a prediction interval, and explain the model and evidence to which it applies. If the biomarker effect itself is estimated with meaningful uncertainty, that uncertainty must also be propagated into the prediction rather than ignored.