Sensitivity of the TOST Decision to the Equivalence Margins

Why Examine Sensitivity to the Margins?

A TOST conclusion depends on both the estimated effect and the equivalence margins. The confidence interval summarizes uncertainty about the effect; the margins specify which effects count as sufficiently close for the scientific and regulatory question. If an interval lies inside one defensible margin pair but crosses a narrower pair, the results are sensitive to the chosen definition of equivalence. That sensitivity is important to show, not a reason to select whichever pair gives the preferred conclusion.

The earlier material on why equivalence margins must be set independently of results establishes the key safeguard: margin choices need a scientific justification independent of the observed treatment estimate and its interval. This tutorial considers what to do when it is useful to recalculate a single endpoint’s TOST decision under more than one scientifically discussable pair. The calculations reveal how the decision changes; they do not make a data-selected margin valid.

What Changes, and What Stays Fixed?

For a difference-scale contrast, the interval may be calculated as \(\hat{\theta}\pm t_{1-\alpha,df}SE\), where \(\hat{\theta}\) is the estimated difference, \(SE\) is its standard error, and \(t_{1-\alpha,df}\) is the relevant \(t\) critical value. The two one-sided tests at level \(\alpha\) conclude equivalence precisely when this interval is strictly inside the margins. For example, \(\alpha=0.05\) corresponds to a 90% two-sided interval. If the analysis instead uses a normal critical value, the choice must be justified by the model and degrees-of-freedom treatment; the margin comparison itself is unchanged.

A sensitivity analysis changes the margin pair, not the estimate or uncertainty calculation. Recomputing a new standard error, changing the confidence level, or choosing a different analysis model at the same time would confound the effect of changing the margins. For each pair, compare both interval endpoints with their corresponding limits. A result can fail because the lower endpoint is too low, the upper endpoint is too high, or both.

The statistical assumptions concern the interval calculation, not the choice of margins: the estimated contrast and its standard error must come from an appropriate analysis for the study design and endpoint; the sampling distribution or model approximation used for the interval must be adequate; and the stated degrees of freedom and critical value must match that analysis. These assumptions are carried unchanged across the margin comparisons. Sensitivity analysis does not repair a misspecified model, an unreliable standard error, or an inappropriate endpoint.

Worked Margin Comparisons

Justification, Diagnostics, and Common Pitfalls

The monotonicity result follows directly from interval containment. If a margin pair is widened without moving either limit inward, every point in the narrower acceptable range remains acceptable under the wider range. Consequently, an interval contained in the narrower range must also be contained in the wider range. But an interval that only fits the wider range can extend beyond a narrower limit. This is a property of set inclusion, not evidence that a wider pair is more appropriate.

A useful diagnostic is a comparison table with one row per scientifically considered pair. Keep the estimate, interval, scale, and confidence level visible alongside the limits and decision. Record whether failure occurs at the lower limit, upper limit, or both. Recheck arithmetic and units, especially when a confidence limit is close to a margin. Use sufficient precision for comparisons, then report rounded values that do not change the decision. Equality with a limit does not satisfy strict interval containment.

Also inspect the analysis that produced the interval. If the same estimate and standard error are held fixed but the interval is recalculated with a different critical value for one margin pair, the comparison no longer isolates margin sensitivity. If different scales are used, transform the interval or limits consistently. If the endpoint, model, or study population changes, explain that separately; the resulting decisions are not a clean comparison of margins alone.

A common mistake is to reason, “The widest limits produce equivalence, so the treatment is equivalent.” The calculation only says that the interval fits those limits. It does not validate the limits or make the result definitive. The correction is to identify the margin pair justified independently of the observed results, state the decision for that pair, and describe other calculations as sensitivity results. Another mistake is to treat failure under a narrow pair as proof of non-equivalence. Failure to establish equivalence means the interval is not wholly contained within that pair; it does not, by itself, establish that the true effect lies outside the acceptable range.

The main alternative presentation is to report only the TOST decision for the governing, justified pair. That is concise and appropriate when no meaningful margin sensitivity question arises, but it hides how close the interval is to either limit. A sensitivity table adds that context. Reporting the two one-sided tests instead of interval containment is mathematically equivalent when the tests use the same model, level, and margins, but it does not remove the need to justify the margins. Neither presentation supports choosing a margin after inspecting which one passes.

Margin sensitivity has limitations. It cannot quantify uncertainty about whether a particular margin is clinically meaningful, and it cannot substitute for prespecification or external scientific judgment. It also addresses one endpoint at a time; it does not resolve how decisions across several endpoints should be combined. The useful conclusion is therefore conditional and explicit: under this specified pair, this interval does or does not satisfy containment.