Deriving the Confidence-Interval Rule for TOST

Why a Confidence Interval Can Summarize TOST

The Intersection-Union Logic of TOST gives the decision structure: equivalence requires rejecting both one-sided null hypotheses. This tutorial derives a useful equivalent description of that decision. Under a common estimate, standard error, and reference distribution, both tests reject exactly when a two-sided confidence interval lies entirely inside the equivalence margins. The result is an algebraic equivalence, not a different criterion.

The confidence interval must use the right critical value. If each one-sided test is conducted at significance level \(\alpha\), the matching two-sided interval has confidence level \(1-2\alpha\), not \(1-\alpha\). For example, two one-sided tests at \(\alpha=0.05\) correspond to a 90% two-sided interval.

Model and Assumptions

Let \(\theta\) be the treatment effect on the scale on which equivalence is defined, and let \(L\) and \(U\) be fixed equivalence margins, with \(L<U\). The estimated effect is \(\hat{\theta}\), its standard error is \(SE\), and the analysis has \(df\) degrees of freedom. The two one-sided tests use the same estimate, standard error, degrees of freedom, and significance level.

The \(t\)-based statement is exact when the stated \(t\) pivot and degrees of freedom are appropriate. In model-based analyses, the reference distribution and standard error must be justified by the fitted model; an approximate degrees-of-freedom method gives an approximate, rather than exact, interval calibration. The margins must be specified on the same scale as \(\theta\), and the estimate and standard error must refer to that same treatment contrast. In crossover analyses, for instance, the contrast may be defined on the log scale; the derivation does not change the scale or the contrast being analyzed.

The one-sided hypotheses are \(H_{0L}:\theta\leq L\) versus \(H_{aL}:\theta>L\), and \(H_{0U}:\theta\geq U\) versus \(H_{aU}:\theta<U\). The lower-bound test rejects for a sufficiently large estimate relative to \(L\); the upper-bound test rejects for a sufficiently small estimate relative to \(U\). The rejection rules follow from the respective boundary values \(L\) and \(U\).

Deriving the Interval Rule

At the least-favorable boundary of the lower null, \(\theta=L\), the lower-test statistic is \((\hat{\theta}-L)/SE\). Its level-\(\alpha\) rejection rule is to exceed the \(1-\alpha\) \(t\) quantile. The corresponding upper-test statistic, evaluated at its boundary \(U\), must fall below the negative of that quantile. Since the \(t\) distribution is symmetric, the same positive critical value is used for both tests.

The rearrangements use \(SE>0\). Both tests reject if and only if both resulting inequalities hold. The left endpoint of the interval must be above \(L\), while the right endpoint must be below \(U\).

The interval has endpoints \(\hat{\theta}-t_{1-\alpha,df}SE\) and \(\hat{\theta}+t_{1-\alpha,df}SE\). Its central coverage is \(1-\alpha-\alpha=1-2\alpha\), because the symmetric \(t\) distribution leaves probability \(\alpha\) in each tail. Thus the two conditions say exactly that the matching two-sided \((1-2\alpha)\) confidence interval lies wholly within \((L,U)\).

The strict inequalities matter. If an interval endpoint equals a margin, the corresponding test statistic is exactly at its critical value, so that test does not reject under the usual rule. “Inside” therefore means that both endpoints are strictly interior, not merely that the interval overlaps the margins or touches them.

Worked Applications

Checks, Alternatives, and Limitations

The interval rule and the pair of one-sided tests are two ways of expressing the same decision only when they use the same estimate, standard error, reference distribution, degrees of freedom, margins, and \(\alpha\). If the two tests use different standard errors or reference distributions, a single symmetric interval of the form derived here need not represent their joint decision. Likewise, an interval computed with a different critical value cannot be substituted without checking the corresponding rejection rules.

The main alternative presentation is to report the two one-sided tests separately, including their test statistics or p-values. That presentation makes the two boundary comparisons explicit; the interval presentation makes the estimate’s precision and its distance from both margins easy to see together. Neither method changes the equivalence hypotheses. A conventional test of a zero treatment difference answers a different question: not finding evidence of a difference does not establish that the effect lies within prespecified equivalence margins.

A practical diagnostic is to recompute the interval endpoints and compare each with its corresponding margin, then verify that the two one-sided inequalities give the same result. A disagreement is a signal to check the confidence level, critical value, scale, contrast, standard error, degrees of freedom, or whether an endpoint merely touches a margin. This algebra does not validate the fitted model, the choice of margins, or the quality of the data. Those require separate scientific and statistical justification.

• Pitfall: “The estimate is between \(L\) and \(U\), so the treatments are equivalent.” The estimate is only a point; it does not show that its uncertainty is contained within the margins. Check both confidence limits.
• Pitfall: Using a 95% two-sided interval because each one-sided test uses \(\alpha=0.05\). The matching interval is 90%, since its two tails each contain 0.05 under the stated symmetric reference distribution.
• Pitfall: Treating an endpoint equal to a margin as a rejection. The usual rejection condition is strict, so equality at the critical boundary does not pass that test.