Why the Confidence Level Must Match the Test
In a two one-sided tests procedure (TOST), equivalence is assessed by testing each equivalence boundary at a chosen one-sided significance level \(\alpha\). The interval used to summarize the same decision must use critical values that match that level. In particular, a 5% TOST uses a central 90% confidence interval, not a central 95% interval.
The reason is the allocation of tail error. A two-sided interval has a lower and an upper endpoint. To match two one-sided tests, each endpoint is based on a tail probability of \(\alpha\). The total probability outside the central interval is therefore \(2\alpha\), so its confidence level is \(1-2\alpha\). The earlier material on the confidence-interval rule for TOST establishes the decision rule; here the focus is on the level and critical value that make the interval correspond to the tests.
This relationship concerns the nominal error level of the procedure. It does not mean there is a \(1-2\alpha\) probability that a fixed parameter lies in the particular interval computed from the data. A confidence level describes the long-run coverage of the interval procedure under its assumptions.
Deriving the Matching Critical Value
Let \(\theta\) be the parameter being assessed, with equivalence limits \(L\) and \(U\), where \(L<U\). Equivalence requires \(\theta\) to be between the limits. Let \(\hat{\theta}\) estimate \(\theta\), let \(SE\) be its standard error, and let \(df\) be the degrees of freedom for the inference. Suppose the standardized estimation error follows a \(t\) distribution with \(df\) degrees of freedom under the model. Write \(t_{q,df}\) for the \(q\)-quantile of that distribution.
The TOST null hypotheses are \(H_{0L}:\theta\leq L\) and \(H_{0U}:\theta\geq U\). At one-sided level \(\alpha\), rejection of the lower-bound null requires the lower one-sided confidence bound to exceed \(L\); rejection of the upper-bound null requires the upper one-sided confidence bound to be below \(U\). Each one-sided bound uses \(t_{1-\alpha,df}\).
The lower-bound test rejects when \(\hat{\theta}-t_{1-\alpha,df}SE>L\). The upper-bound test rejects when \(\hat{\theta}+t_{1-\alpha,df}SE<U\).
TOST concludes equivalence only when both one-sided null hypotheses are rejected. The two inequalities together say that both endpoints of the interval lie strictly between \(L\) and \(U\).
The resulting interval is centered at \(\hat{\theta}\) with half-width \(t_{1-\alpha,df}SE\). Its two tails each have probability \(\alpha\), leaving central coverage \(1-\alpha-\alpha=1-2\alpha\).
For known standard deviation and a normal sampling distribution, the corresponding critical value is \(z_{1-\alpha}\) rather than a \(t\) quantile. The same tail argument gives confidence level \(1-2\alpha\). In crossover analyses, the interval must use the standard error and degrees of freedom appropriate to the fitted model; changing those does not change the level relationship.
Worked Examples
Worked Example: Converting a 5% TOST Level to an Interval
A hypothetical equivalence analysis uses \(\alpha=0.05\) for each one-sided test. The estimate is \(1.80\) units, its standard error is \(0.40\) units, and the standard error is treated as known with normal sampling. The equivalence limits are \(1.00\) and \(2.50\) units. Find the matching interval and assess whether it is inside the limits.
The matching confidence level is \(1-2\alpha=1-2(0.05)=0.90\), or 90%. The normal critical value is \(z_{0.95}=1.645\), because each tail has area 0.05.
The interval lies strictly within \([1.00,2.50]\) units: \(1.142>1.00\) and \(2.458<2.50\). Under the stated normal model and known-standard-error assumption, this interval rule gives an equivalence conclusion at the 5% TOST level.
Worked Example: A 2.5% TOST Level
In a hypothetical analysis, the one-sided significance level is \(\alpha=0.025\). An estimate is \(3.20\) units, \(SE=0.15\) units, and \(df=24\). Determine the matching confidence level and calculate the interval.
The matching level is \(1-2(0.025)=0.95\), or 95%. For \(df=24\), the required critical value is \(t_{0.975,24}=2.064\). This is the correct quantile because each tail has probability 0.025.
The arithmetic gives a 95% central interval, matched to two one-sided tests at 2.5% each. The degrees of freedom and standard error must come from the analysis being used; the level conversion alone does not validate those model inputs.
Worked Example: How a Mismatched 95% Interval Can Change the Decision
Consider hypothetical data with equivalence limits \(-0.10\) and \(0.10\) units, estimate \(\hat{\theta}=0.060\) units, \(SE=0.020\) units, and \(df=19\). The planned TOST level is \(\alpha=0.05\). Compare the matching interval with a 95% interval.
For the 5% TOST, use the 90% interval and \(t_{0.95,19}=1.729\):
Both endpoints are strictly within the equivalence limits, so the matching interval rule indicates equivalence at the planned 5% level. Now use a 95% interval, whose critical value is \(t_{0.975,19}=2.093\):
This interval extends beyond the upper limit. It does not satisfy the interval criterion. That different result is not a contradiction: the 95% interval corresponds to two one-sided tests at 2.5%, not the planned 5%. For a 5% TOST, using the 95% interval applies a more stringent criterion than the one specified.
Assumptions, Diagnostics, and Alternatives
The level conversion is mathematical, but a valid equivalence inference also depends on the interval being valid for the parameter. The estimate, standard error, and degrees of freedom must represent the same prespecified estimand and fitted analysis. The model’s distributional assumptions and variance estimation must support the stated \(t\) or normal reference distribution. In applications using log-scale crossover models, relevant residual diagnostics and model checks help assess whether the chosen analysis is defensible.
Before applying the interval rule, check that the equivalence limits are on the same scale and in the same units as \(\hat{\theta}\), and that \(\alpha\), the interval level, the standard error, and the degrees of freedom were selected consistently. A software output labeled “95% CI” is not automatically the interval matching a 5% TOST. Inspect the actual critical value or confidence level rather than relying on a default.
The two common ways to present the same decision are the direct TOST and its matching confidence interval. The direct approach reports the two one-sided test results; the interval approach reports whether the matching interval fits within the equivalence limits. When they use the same estimate, standard error, reference distribution, degrees of freedom, and \(\alpha\), they give the same decision. A 95% interval is sometimes chosen for other reporting purposes, but it is not the matching interval for a 5% TOST.
The overall type I error of TOST is controlled at no more than \(\alpha\), not \(2\alpha\). The null hypothesis for equivalence is the union of the two non-equivalence regions: \(\theta\leq L\) or \(\theta\geq U\). A false equivalence conclusion requires rejecting the null component that is true; because both tests must reject, the probability of that event is bounded by the level of the individual test. This intersection-union structure is why no extra division of \(\alpha\) between the two tests is needed.
The interval criterion is a decision rule, not a replacement for reporting the estimate and its uncertainty in context. It also does not establish that the equivalence limits are scientifically appropriate; those limits need justification independent of the observed estimate. If the model or interval construction is unsuitable, choosing the nominal level correctly cannot repair the analysis.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A TOST uses one-sided \(\alpha=0.05\). What central confidence level matches it, and what probability is assigned to each tail?
- What one-sided level matches a central 98% interval?
- For a 5% TOST with \(df=14\), which \(t\) quantile is used for the matching interval: \(t_{0.95,14}\) or \(t_{0.975,14}\)? Explain.
- A researcher uses a 95% interval for a TOST specified at 5%. Is that interval the matching one? What TOST level does a central 95% interval match?
Solutions. (1) The matching level is \(1-2(0.05)=0.90\), so it is a 90% interval with 0.05 in each tail. (2) If \(1-2\alpha=0.98\), then \(2\alpha=0.02\) and \(\alpha=0.01\) per one-sided test. (3) Use \(t_{0.95,14}\), because the upper tail of each one-sided bound has probability 0.05; the central interval consequently leaves 0.05 in each tail. (4) No. A central 95% interval has 0.025 in each tail and matches one-sided tests at \(\alpha=0.025\), not 0.05.