Clinical Benefit Thresholds Under Sensitivity Analysis

Why Compare Sensitivity Results With a Clinical Threshold?

A treatment effect can be favorable without being large enough to matter to patients or clinicians. A prespecified minimum clinically important benefit makes that distinction explicit: it is the smallest effect on a specified scale judged clinically worthwhile for the target population and outcome. In sensitivity analysis, the estimate depends on assumptions about unobserved outcomes. Comparing each assumption-specific estimate with the same clinical threshold shows whether the estimated benefit is large enough under each assumption—not merely whether the estimate is positive.

This comparison answers a different question from comparing effects with zero. It also does not, by itself, establish that the true effect exceeds the threshold. The calculations below classify point estimates. Sampling uncertainty and uncertainty about missing-data assumptions remain distinct, as emphasized in the earlier material on Sampling Uncertainty Versus Assumption Uncertainty.

The inequality matters. Equality means the estimate meets the minimum, but does not strictly exceed it. A negative margin means the estimate falls short; a zero margin means it is exactly at the threshold; and a positive margin means it is above the threshold. If the effect is defined in a direction where smaller values indicate benefit, first state that direction clearly and use a consistently oriented benefit measure for the comparison. Do not apply the inequality mechanically to a contrast whose sign has the opposite meaning.

Assumptions Needed for a Meaningful Comparison

A threshold comparison is interpretable only when the threshold and the estimates refer to the same clinical question. The treatment-effect estimand must be held fixed across the sensitivity assumptions: the population, outcome, time point, handling of intercurrent events, and treatment contrast should not silently change. The threshold must also use the estimand’s scale and units. For example, a threshold of 3 points on a symptom scale cannot be compared directly with an effect reported as a standardized difference or a percentage without a justified conversion.

The threshold should be specified before examining the sensitivity results, or otherwise justified independently of the results. Choosing a threshold after seeing the estimates can make the classification appear more favorable than it is. Likewise, the set \(A\) should represent assumptions considered plausible for the analysis, rather than a collection selected simply because its estimates meet the threshold. These are substantive requirements: arithmetic cannot establish that a threshold is clinically appropriate or that a sensitivity set is plausible.

For a finite set of scenarios, report the comparison scenario by scenario. For a continuous sensitivity set, the minimum and maximum of the estimated effect over that set provide useful summaries, provided those extrema are defined or reliably evaluated. In either case, the same threshold is applied throughout. Averaging estimates across assumptions is generally not a substitute: an average can meet the threshold while individual plausible assumptions do not, and it may have no clear interpretation as the treatment effect under any one assumption.

Worked Comparisons

What Scenario-Wide Summaries Establish

For a finite set \(A\), a simple summary is the number or proportion of scenario-specific estimates meeting the threshold. This should accompany, not replace, the scenario values or margins, because a count alone hides how far estimates lie above or below the minimum. A useful related summary is the smallest margin across the set. If all assumptions are to be treated as relevant, then:

This equivalence follows directly from \(M(a)=\hat{\theta}(a)-\Delta_{\mathrm{clin}}\): the minimum margin is nonnegative exactly when no scenario has a negative margin. Similarly, if \(\max_{a\in A}M(a)<0\), every estimate is below the threshold. If the margins include both negative and nonnegative values, the point-estimate classification changes within the set. For a continuous set, the same logic applies when the minimum or maximum exists and is correctly evaluated.

These summaries describe sensitivity of the threshold classification to the specified assumptions. They do not establish that the assumptions exhaust all scientifically plausible possibilities. Nor does a scenario count assign probabilities to assumptions: unless a probability model over assumptions has been specified and justified, “three of four scenarios meet the threshold” is a count, not a probability that the clinical benefit is adequate.

How This Differs From Other Comparisons

A comparison with zero asks whether an estimate points toward benefit or harm under the chosen sign convention. A comparison with \(\Delta_{\mathrm{clin}}\) asks whether the estimated magnitude reaches a clinically meaningful minimum. An estimate can therefore be positive yet below the threshold. A confidence interval can also be compared with a threshold, but that is a different inferential statement from classifying the point estimate. Do not use a point-estimate threshold comparison as a substitute for an uncertainty analysis.

Comparing every assumption-specific estimate with the threshold preserves the connection between each result and its assumptions. A single representative scenario may be easier to summarize but can obscure changes in classification elsewhere in the plausible set. An average across scenarios may obscure them too, especially if some estimates fall below and others exceed the threshold. The earlier material on Robust Conclusions Across Assumption Sets concerns how conclusions vary over assumptions; the present comparison adds a prespecified clinical benchmark to that scenario-by-scenario assessment.

Diagnostics, Limitations, and Common Mistakes

Before interpreting a threshold table, check that the threshold was justified and prespecified, all estimates target the same estimand, and the outcome scale, units, time point, and benefit direction agree. Check the subtraction and the inequality at equality. For continuous sensitivity analyses, verify that the reported extrema genuinely cover the full plausible set. If a classification changes after a small change in the sensitivity parameter, report that fact rather than describing the result as uniformly above or below the threshold.

A common mistake is to say that every estimate “exceeds” the threshold when one equals it. The correction is to distinguish “meets or exceeds” (\(\geq\)) from “strictly exceeds” (\(>\)). Another is to declare clinically adequate benefit because an estimate is positive. The correction is to compare it with the actual minimum, not with zero. A third is to call the threshold classification statistically established because a point estimate is above the minimum. The correction is to state that the estimate exceeds the threshold under that assumption and keep inferential uncertainty separate.

Finally, a threshold is not a universal property of an outcome. Its clinical meaning can depend on the population, instrument, time point, and treatment context. Sensitivity analysis cannot repair a poorly chosen threshold, an incoherent estimand, or an implausible assumption set. It makes the consequences of the stated choices visible; it does not validate those choices.