Why Surrogacy Claims Need Stress Tests
A biomarker can respond to treatment and predict a clinical outcome without reliably capturing the treatment’s clinical benefit. The association may partly reflect common causes of biomarker and outcome, a treatment pathway not represented by the biomarker, or choices about when and how the biomarker is measured. Randomization supports causal comparisons of treatment groups, but it does not by itself randomize a post-treatment biomarker. Consequently, associations between biomarker values and outcomes within treatment groups can remain confounded.
Negative-control outcomes and sensitivity analyses do not certify a biomarker as a surrogate. They ask a narrower, useful question: does the evidence remain credible when we probe assumptions that the primary analysis cannot verify directly? A negative control can reveal a pattern consistent with residual bias. Quantitative sensitivity analysis shows how strong a specified bias would need to be to change a conclusion. Repeating the analysis with alternative, scientifically defensible biomarker definitions tests whether the conclusion depends on an arbitrary measurement choice.
Let \(A\) denote randomized treatment, \(S\) a post-treatment biomarker, \(Y\) the clinical outcome, and \(Y_{NC}\) a negative-control outcome. Baseline covariates are denoted by \(X\), and \(U\) represents unmeasured common causes. A useful negative control should have a plausible reason to share relevant causes of \(S\) and \(Y\), such as disease severity or care-seeking behavior, while having no causal path from \(A\) or \(S\) to \(Y_{NC}\) during the study period. It should also be measured comparably enough that its associations are interpretable.
Those conditions are substantive assumptions, not properties established merely by naming an outcome “negative control.” If treatment affects the proposed control outcome, or if the control has no relevant shared causes with the clinical endpoint, its result may mislead. Randomization helps evaluate whether treatment itself appears associated with the control, but a null treatment comparison cannot rule out all bias in the biomarker–clinical outcome relationship.
Using a Negative-Control Outcome
A common diagnostic is to estimate the association of \(S\) with \(Y_{NC}\), adjusting for treatment assignment and prespecified baseline covariates \(X\), using a model appropriate for the outcome type. The target is not necessarily a causal effect of \(S\) on the control. Instead, the question is whether the association is unexpectedly strong given the proposed absence of a causal pathway. The estimate, its uncertainty, and the design assumptions should all be reported.
Worked Example: Interpreting a Negative-Control Association
In a hypothetical randomized trial, investigators study a standardized week-4 biomarker \(S\) and a clinical endpoint \(Y\). They also record a negative-control outcome, an unrelated laboratory abnormality that is not expected to respond to the treatment or to the biomarker pathway. After adjustment for treatment and baseline covariates, a linear model estimates an association of \(0.30\) control-outcome units per one standard deviation higher \(S\), with \(SE=0.10\). Assume the model’s standard-error approximation is adequate for this illustration.
The test statistic for a zero association is \(z=0.30/0.10=3.00\). A two-sided normal-reference p-value is approximately \(0.0027\). A 95% confidence interval is \(0.30\pm1.96(0.10)=0.30\pm0.196\), or \((0.104,0.496)\) control-outcome units per standard deviation of \(S\).
The interval excludes zero, and the estimated association is not small relative to its uncertainty. If the control outcome truly has no causal relationship to \(S\), this finding raises concern that shared causes, selection, or another source of bias contributes to biomarker–outcome associations. It does not show that the clinical association is biased upward by \(0.30\), nor does it identify a particular confounder. The control’s relevance and the plausibility of its no-effect assumption must be examined before interpreting the result.
A null result is also limited. A wide interval can include substantial associations, and a control that shares few of the relevant confounders may stay null even when the main analysis is biased. Thus, a negative-control analysis is most informative when its expected causal status and shared-bias rationale are stated before inspecting the result. Where feasible, use more than one scientifically justified control and explain why each is informative; a collection of poorly justified controls is not automatically stronger evidence.
Alternative Biomarker Definitions
The biomarker definition should follow the biological and clinical question. Plausible alternatives may include a prespecified measurement time, a summary over a biologically relevant interval, a threshold based on a clinically meaningful range, or a definition that handles values below quantification limits differently. The point is not to search many definitions until one gives the preferred result. Instead, specify a small set of defensible alternatives and compare whether key conclusions—such as direction, magnitude, and uncertainty of the treatment-related biomarker change and its relationship with the clinical endpoint—are stable.
Comparisons need care. Two definitions may use different scales or target different biological processes, so raw coefficients may not be directly comparable. Standardizing a continuous biomarker can help express associations per standard deviation, but it does not make distinct constructs identical. If multiple definitions are chosen after seeing the results, apparent robustness can be overstated; report that exploratory choice and avoid treating the most favorable estimate as confirmatory evidence.
Worked Example: Comparing Two Prespecified Biomarker Definitions
Suppose a hypothetical trial estimates a clinical treatment effect of \(-0.40\) outcome units, where negative values favor treatment. Investigators consider two prespecified, standardized biomarker definitions. Under a simplified linear decomposition with no treatment-by-biomarker interaction, the biomarker-associated component is the treatment effect on the biomarker multiplied by the outcome association per standard deviation of that biomarker.
For definition 1, the estimated treatment effect on the biomarker is \(-0.50\) standard deviations, and the estimated outcome association is \(0.70\) outcome units per standard deviation. The product is \((-0.50)(0.70)=-0.350\) outcome units. Relative to the total effect, its magnitude is \(0.350/0.400=0.875\), or 87.5%.
For definition 2, the corresponding estimates are \(-0.42\) standard deviations and \(0.60\) outcome units per standard deviation. The product is \((-0.42)(0.60)=-0.252\) outcome units, and the magnitude ratio is \(0.252/0.400=0.630\), or 63.0%.
Both definitions imply a biomarker-associated component in the treatment-benefit direction, but the estimated fractions differ materially. This is evidence that the numerical surrogacy summary depends on biomarker definition. The ratios are descriptive calculations under the stated simplified decomposition, not proof that the biomarker causally mediates those fractions. Uncertainty in all estimates and the assumptions behind the decomposition would also need to be considered in a full analysis.
Quantifying Sensitivity to Residual Confounding
A sensitivity analysis makes an untestable concern explicit by assigning it a parameter or range. One transparent illustration uses a linear outcome model. Let \(S_r\) and \(U_r\) be the biomarker and an omitted confounder after removing their linear associations with treatment and measured covariates. Suppose the outcome model, on this adjusted scale, is \(Y=\beta S_r+\gamma U_r+\varepsilon\), where \(\gamma\) is the outcome change per unit of \(U_r\), and \(\varepsilon\) is uncorrelated with \(S_r\). If \(U_r\) is omitted, the fitted biomarker coefficient is biased whenever \(S_r\) and \(U_r\) are associated.
This follows by substituting the outcome model into the covariance: \(\operatorname{Cov}(S_r,Y)=\beta\operatorname{Var}(S_r)+\gamma\operatorname{Cov}(S_r,U_r)+\operatorname{Cov}(S_r,\varepsilon)\). The last term is zero by assumption. Dividing by \(\operatorname{Var}(S_r)\) gives the expression above. The result is a sensitivity calculation, not a method for estimating an unknown confounder from nothing: the analyst must posit or bound \(\gamma\) and the residual association.
Worked Example: How Strong Must Residual Confounding Be?
In a hypothetical analysis, the observed biomarker coefficient is \(0.80\) clinical-outcome units per standard deviation of \(S\). For illustration, suppose the residual standard deviations of \(S\) and an omitted confounder \(U\) are both 1, their residual correlation is \(0.30\), and a one-unit increase in \(U\) changes the outcome by \(\gamma=0.50\) units. These are sensitivity assumptions, not estimates from the trial.
Because both residual standard deviations equal 1, \(\operatorname{Cov}(S_r,U_r)=0.30(1)(1)=0.30\), and \(\operatorname{Var}(S_r)=1\). The implied bias is \(0.50(0.30)/1=0.15\) outcome units per standard deviation of \(S\). Under this assumed positive confounding, the bias-adjusted coefficient is \(0.80-0.15=0.65\) outcome units per standard deviation.
Now consider a sensitivity range with \(\gamma\) from \(0.30\) to \(0.80\) and residual correlation from \(0.10\) to \(0.50\), keeping both residual standard deviations equal to 1. The smallest positive bias is \(0.30(0.10)=0.030\); the largest is \(0.80(0.50)=0.400\). The corresponding adjusted coefficients range from \(0.80-0.400=0.400\) to \(0.80-0.030=0.770\). In this range the association remains positive, but its magnitude varies substantially. If plausible assumptions allowed a bias of at least \(0.80\), the sign could be removed or reversed; that possibility would weaken the robustness of the original interpretation.
The direction matters: if the confounder’s associations with \(S\) and \(Y\) have opposite signs, the bias can be negative rather than positive. A credible report therefore shows the assumed direction and ranges, and explains why they are plausible in context instead of presenting one convenient adjusted value.
Checking Uncaptured Treatment Pathways
Residual confounding is not the only threat. A treatment may affect the clinical outcome through pathways not represented by the biomarker. Under a simple additive decomposition with no treatment-by-biomarker interaction, write the total treatment effect as \(\theta=\beta\Delta S+\delta\), where \(\Delta S\) is the treatment effect on the biomarker, \(\beta\) is the outcome change per biomarker unit under the model, and \(\delta\) is the remaining treatment effect not represented by that biomarker component. This equation is a sensitivity framework: it does not establish that the decomposition is causally identified.
For the second definition in the preceding example, the biomarker-associated component was \(-0.252\) outcome units. If the total treatment effect is \(-0.40\), the remaining component implied by this arithmetic is \(\delta=-0.40-(-0.252)=-0.148\) outcome units. A plausible nonzero remainder is a warning against claiming that this biomarker alone captures the entire treatment effect. If an analysis instead assumes \(\delta=0\), it should show how conclusions change over scientifically plausible values of \(\delta\), including values that oppose the biomarker-associated component.
This check complements negative controls: a negative-control outcome probes some forms of shared bias, while a direct-effect sensitivity parameter asks how much treatment influence outside the biomarker would alter the surrogacy conclusion. Neither analysis rules out other problems, including selection caused by conditioning on post-treatment variables, missing biomarker measurements, or a mismatch between the study population and the intended use population.
Diagnostics, Comparisons, and Limits
Useful diagnostics include checking whether the negative control could plausibly be affected by treatment, comparing its baseline and follow-up measurement patterns across groups, examining whether missingness differs, and reviewing whether the biomarker and control share the proposed causes. For alternative biomarker definitions, inspect distributions, timing, missingness, and influential observations as well as the final coefficients. For quantitative analyses, plot or tabulate adjusted conclusions across a grid of sensitivity values; a single assumed value can conceal a tipping point.
These checks answer different questions and are not substitutes for one another. Negative controls use observed outcomes as diagnostics but depend strongly on control validity. Alternative definitions probe measurement and specification choices but can encourage selective reporting if chosen after results are known. Quantitative sensitivity analyses expose dependence on assumptions but cannot verify those assumptions from the observed data alone. A robust conclusion is more credible when these distinct checks point in a compatible direction, yet robustness is not the same as proof of surrogacy.
| Approach | Primary question | Main limitation |
|---|---|---|
| Negative-control outcome | Does an association appear where the assumed causal pathway is absent? | A control may fail to share relevant bias or may have an unintended causal pathway. |
| Alternative biomarker definitions | Does the conclusion depend on a particular time point, summary, or threshold? | Definitions can target different constructs; post hoc selection undermines interpretation. |
| Quantitative sensitivity analysis | How strong must residual confounding or an unrepresented pathway be to change the conclusion? | Results depend on assumptions or ranges that data may not identify. |
Check Your Understanding
Answer each question using the stated assumptions. Brief solutions follow the questions.
- A biomarker is associated with a proposed negative-control outcome after adjustment. What can this finding support, and what can it not prove?
- In the omitted-variable model, let \(\beta_{\mathrm{obs}}=0.60\), \(\gamma=0.40\), residual correlation \(0.25\), and both residual standard deviations equal 1. Calculate the adjusted coefficient.
- Why is a nonsignificant negative-control association not evidence that residual confounding is absent?
- A treatment effect on a standardized biomarker is \(-0.30\), and the outcome association is \(0.50\) units per standard deviation. Calculate the biomarker-associated component under the simple linear decomposition.
- Name one reason estimates from two alternative biomarker definitions may not be directly comparable.
Solutions. (1) It supports concern about a shared bias or another violated assumption, but does not identify the bias or quantify its effect on the clinical endpoint. (2) The bias is \(0.40(0.25)(1)(1)/1=0.10\), so the adjusted coefficient is \(0.60-0.10=0.50\). (3) The interval may be wide, and the control may not share relevant confounders; a nonsignificant test does not establish a zero association. (4) The component is \((-0.30)(0.50)=-0.15\) outcome units. (5) The definitions may use different scales or represent different biological constructs.