From Trial-Level Association to a Threshold
When a surrogate endpoint is used to support a claim about clinical benefit, the relevant question is not only whether the two endpoints are associated across trials. It is also how much clinical treatment effect remains unexplained, and what surrogate treatment effect would be large enough for a new trial’s clinical effect to have a positive lower prediction limit. Trial-level \(R^2\) summarizes the first question; the surrogate threshold effect (STE) addresses the second.
These quantities describe relationships across randomized trials, not relationships between individuals within a trial. They depend on the treatment-effect definitions, direction, and scales used in every trial. For clarity, suppose larger values of both treatment effects represent greater benefit. If benefit is represented by a reduction in a clinical outcome, the effects must be coded so that this reduction has the agreed positive direction before fitting the regression.
The regression line summarizes the expected clinical effect at a given surrogate effect. The prediction interval is wider: it includes uncertainty about the estimated line and the residual variation among trials around that line. For an STE intended to address a new trial, that residual variation matters. A confidence interval for the mean line alone does not answer the same question.
Model, Assumptions, and Calculations
For a basic unweighted analysis, write the trial-level model as
where the \(x_j\) values are treated as fixed and the residuals are independent with a common variance. The residual represents the part of the clinical treatment effect not captured by the fitted linear relationship. These assumptions make the usual regression prediction interval available. In a real meta-analytic validation, estimated effects have different sampling uncertainties and may require a meta-analytic regression that accounts for them; the simple calculations below illustrate the core quantities, not a universal analysis recipe.
With \(m\) trials, the fitted line is \(\hat{y}=a+bx\). The trial-level coefficient of determination is
It is the fraction of the observed total sum of squares in the clinical effects accounted for by the fitted line. It is not the fraction of an individual patient’s clinical outcome explained by their biomarker, and it does not establish that changing the surrogate causes a clinical benefit.
Worked Example: Calculating Trial-Level R-Squared
Consider five hypothetical randomized trials. The table gives surrogate effects \(x_j\) and clinical effects \(y_j\), both in percentage points of benefit. The displayed clinical effects are rounded; calculations use the unrounded values shown by the fitted values plus residuals in the last column.
| Trial | \(x_j\) | Fitted clinical effect | Residual | \(y_j\), approximately |
|---|---|---|---|---|
| 1 | 1 | 0.6 | 0.63246 | 1.2325 |
| 2 | 2 | 1.8 | -0.63246 | 1.1675 |
| 3 | 3 | 3.0 | 0 | 3.0000 |
| 4 | 4 | 4.2 | -0.63246 | 3.5675 |
| 5 | 5 | 5.4 | 0.63246 | 6.0325 |
The fitted line is \(\hat{y}=-0.6+1.2x\). The residuals sum to zero and are uncorrelated with the centered \(x\) values, so the line has intercept \(-0.6\) and slope \(1.2\). Their squared sum is \(SSE=4(0.63246^2)=1.60\) percentage-points squared. The fitted values have mean 3.0, and their squared deviations sum to \(1.2^2(10)=14.4\). Since the residuals are orthogonal to the fitted values, the total sum of squares is \(SST=14.4+1.6=16.0\).
Therefore,
In this hypothetical data set, the fitted linear relationship accounts for 90.0% of the observed variation in clinical treatment effects across the five trials. This is a strong sample association, but five trials provide little information about how stable that proportion is in a broader collection of trials. A point estimate of \(R^2\) should therefore be accompanied by an uncertainty assessment, commonly a trial-level bootstrap or an interval method appropriate to the meta-analytic model.
Prediction Limits and the Surrogate Threshold Effect
Under the stated linear model, a two-sided 95% prediction interval for the clinical effect in a new trial at surrogate effect \(x\) is
where \(s=\sqrt{SSE/(m-2)}\). The “1” under the square root represents residual variation for a new trial; the other terms represent uncertainty in the estimated mean line. The lower prediction limit is the fitted clinical effect minus the displayed margin. Using the two-sided 95% interval’s lower endpoint is a common STE convention; the chosen interval level and convention should be stated.
The STE is found by solving for the smallest \(x\) for which that lower limit is zero or greater. This requires a positive benefit direction, a fitted relationship used over a defensible range, and a prediction interval that corresponds to the intended new-trial prediction. The equation can have more than one crossing if the fitted relationship is unusual; the definition then requires specifying the relevant domain and selecting the smallest qualifying value there.
Worked Example: Finding the Surrogate Threshold Effect
Continue with the five hypothetical trials. Here \(m=5\), \(\bar{x}=3\), \(S_{xx}=10\), \(SSE=1.60\), and \(df=m-2=3\). The residual standard deviation is \(s=\sqrt{1.60/3}=0.73030\) percentage points, and \(t_{0.975,3}=3.182446\). Thus the lower prediction limit is
The STE solves \(L(x)=0\). Equivalently, on the relevant domain where the fitted effect is positive, square both sides of
Expanding gives approximately
The positive root is \(x\approx2.6335\) percentage points. Substitution into the original, unsquared equation confirms the crossing. Thus, under this fitted model and its assumptions, the lower endpoint of the two-sided 95% prediction interval reaches zero at a surrogate benefit of about 2.634 percentage points. This is not a guarantee of benefit for every trial at or above that value, nor does it prove that the surrogate is causally responsible for the clinical effect.
The estimated threshold is itself uncertain because the regression coefficients, residual variation, and trial set are uncertain. A trial-level bootstrap can quantify this: resample trials as whole units, refit the regression in each resample, and recalculate the threshold using the same prediction-interval rule. Report the resulting distribution or a percentile interval, while noting that a small number of trials can make bootstrap intervals unstable. A threshold estimate alone is not a confidence interval for the threshold.
Worked Example: A Prediction Limit Near the Threshold
At \(x=2.50\) percentage points, the fitted clinical effect is
The prediction margin is calculated without rounding intermediate quantities:
The lower prediction limit is \(L(2.50)=2.40-2.57234=-0.17234\) percentage points, or approximately \(-0.172\) percentage points. Since it is below zero, this surrogate effect is below the estimated STE under the chosen rule. The result does not say the mean clinical effect is negative: the fitted mean is positive, but the prediction interval for a new trial includes effects below zero.
Uncertainty, Diagnostics, and Extrapolation
A regression slope and its uncertainty provide useful context for \(R^2\). In the hypothetical example, the slope standard error is \(SE(b)=s/\sqrt{S_{xx}}=0.23094\) clinical percentage points per surrogate percentage point. A 95% confidence interval for the mean-line slope is
Under the model, this interval describes uncertainty in the population regression slope, not the range of effects expected in a new trial. Prediction limits are wider because they also include residual trial-to-trial variation. Likewise, the value \(R^2=0.900\) is a sample summary, not a precise statement that exactly 90% of clinical-effect variation will be explained in future settings.
Inspect a scatterplot of trial effects and the fitted line before interpreting either quantity. Look for curvature, influential trials, unequal residual spread, and a threshold driven by one or two points. With few trials, a single trial can substantially change both \(R^2\) and the STE. Check that treatment-effect directions and scales are aligned, and that trial-level estimates are sufficiently comparable for the chosen model. A high \(R^2\) can coexist with a wide prediction interval if the clinical effects vary substantially around the line; it can also arise over a narrow range of effects without supporting useful predictions elsewhere.
Worked Example: Why Extrapolation Can Mislead
Suppose someone proposes \(x=7\) percentage points for a future use of the same surrogate, although the observed trial effects ranged only from 1 to 5. The fitted clinical effect is \(-0.6+1.2(7)=7.80\) percentage points. The prediction margin is
The lower prediction limit is \(7.80-3.88901=3.91099\), or approximately \(3.911\) percentage points. The model’s numerical lower limit is positive, but \(x=7\) lies outside the observed range. The calculation therefore relies on an untested continuation of the linear relationship and residual pattern; it should not be treated as validated evidence that a clinical benefit is assured. Extrapolation can produce an apparently reassuring threshold crossing even when the data contain no trials near the proposed surrogate effect.
The main alternatives answer different questions. A confidence band for the mean regression line describes uncertainty in the average clinical effect at a given surrogate effect, whereas the STE convention above uses a prediction limit for a new trial. A weighted or hierarchical meta-regression can incorporate differing within-trial precision and between-trial heterogeneity more appropriately than unweighted ordinary least squares. It may change both \(R^2\) and the threshold, so results should state the model and the definition of \(R^2\) used. Neither a high \(R^2\) nor a positive lower prediction limit establishes individual-level surrogacy, transportability to a different population, or causal mediation.
Check Your Understanding
Answer each question, then compare with the brief solutions.
- In a trial-level regression, what does \(R^2=0.75\) mean, and what does it not mean?
- Why does a prediction interval include an additional “1” inside the square root that is absent from the confidence interval for the mean line?
- In the worked data, is \(x=2.50\) above or below the estimated STE? Explain using the calculated lower prediction limit.
- Why should the proposed \(x=7\) prediction be treated cautiously even though its calculated lower limit is positive?
- Name one approach for assessing uncertainty in an estimated STE and one reason its result may be unstable.
Solutions. (1) It means the fitted linear relationship accounts for 75% of the observed between-trial variation in the clinical effects in that data set. It does not mean that 75% of an individual outcome is explained or that the surrogate causes clinical benefit. (2) A future trial has its own residual deviation from the population line, represented by that additional term; a mean-line confidence interval does not include this new-trial variation. (3) It is below the estimated STE: at \(x=2.50\), the lower prediction limit is approximately \(-0.172\) percentage points, below zero. (4) It is outside the observed range, so the result depends on extrapolating the fitted linear relationship and residual behavior. (5) Resample trials as whole units, refit the regression, and recalculate the threshold in each bootstrap sample. With few trials, resamples can omit influential trials or yield highly variable thresholds.