Applying TOST to a Model-Provided Contrast

From a Model Output to an Equivalence Decision

A model may report an estimated contrast, its standard error, and degrees of freedom without displaying the two equivalence tests that are needed for a TOST decision. The contrast might compare treatment groups after adjustment for other model terms, or represent a specified combination of model coefficients. The calculation is straightforward once the supplied estimate corresponds to the intended estimand and is on the same scale as the equivalence limits.

The purpose here is to use those model-provided quantities as inputs, not to reconstruct the model fit or recalculate its covariance matrix. The essential checks are whether the contrast has the right direction and scale, and whether its standard error and degrees of freedom belong to that same contrast.

Applying the Two Tests

For a \(t\)-based contrast, the standard error describes uncertainty in \(\hat{\theta}\), and \(df\) determines the reference \(t\) distribution. Each test evaluates the estimate relative to one boundary. The lower-bound test asks whether the contrast is sufficiently above \(L\); the upper-bound test asks whether it is sufficiently below \(U\).

Here \(T_{df}\) denotes a random variable with a \(t\) distribution on \(df\) degrees of freedom. The lower test is right-tailed because evidence against \(\theta\le L\) consists of a sufficiently large estimate. The upper test is left-tailed because evidence against \(\theta\ge U\) consists of a sufficiently small estimate. Both tests must have p-values below \(\alpha\). A small p-value for only one test is not enough.

The reference distribution must be justified by the model analysis that produced the output. In a normal linear model with the relevant variance estimated under its assumptions, a contrast statistic may have an exact \(t\) distribution. In other model settings, software may report a \(t\) approximation and an approximate \(df\). Use the reported inferential framework consistently; do not combine a standard error from one method with degrees of freedom or critical values from another.

The interval rule is a useful check. With \(t_{1-\alpha,df}\) denoting the \(1-\alpha\) quantile of the \(t\) distribution, the corresponding two-sided \(100(1-2\alpha)\%\) confidence interval is

Both one-sided tests reject at level \(\alpha\) exactly when this interval lies within the equivalence limits, subject to the convention used for equality at a boundary. This follows by rearranging the test inequalities: \(p_L<\alpha\) requires \(t_L>t_{1-\alpha,df}\), or \(\hat{\theta}-t_{1-\alpha,df}SE>L\). Likewise, \(p_U<\alpha\) requires \(t_U<-t_{1-\alpha,df}\), or \(\hat{\theta}+t_{1-\alpha,df}SE<U\). Thus both tests pass precisely when both interval endpoints are inside the limits. In practice, avoid basing a conclusion on rounded values when an endpoint is very close to a margin.

Worked Calculations From Model Output

Checks Before Using the Output

A model output is useful only when it represents the intended contrast. Before substituting numbers, verify the contrast definition, the direction of subtraction, the measurement scale, and the matching uncertainty quantities. For example, a treatment-minus-reference contrast and a reference-minus-treatment contrast have opposite signs. Reversing the order also reverses the corresponding limits: the intended interval \([L,U]\) becomes \([-U,-L]\). Changing the estimate’s sign without changing the limits gives a different test.

Confirm that \(SE\) is the standard error of the contrast, not the standard error of one group mean or of an individual coefficient. A contrast can combine several coefficients, and its uncertainty depends on their covariance. The model output should supply the contrast-specific standard error and degrees of freedom, or a documented equivalent. This tutorial takes those outputs as given rather than recomputing them.

The equivalence limits and estimate must also use the same scale and units. If a model reports a log-scale contrast, limits stated on a ratio scale cannot be inserted directly: use the log-scale limits established for that analysis. Likewise, a difference in concentration units cannot be compared directly with percentage-point limits. The limits should be scientifically justified and specified independently of the observed contrast.

A model’s ordinary test of \(H_0:\theta=0\) does not answer the equivalence question. It tests whether the contrast differs from zero, whereas TOST tests whether it is demonstrably above the lower limit and below the upper limit. A statistically nonsignificant zero-effect test may reflect imprecision and does not replace either TOST component.

Assumptions, Diagnostics, and Alternatives

The calculations presume that the contrast estimates the target parameter, the standard error is correctly calculated for it, and the model’s reference distribution and degrees of freedom provide valid inference. The model’s own assumptions must therefore be plausible for the analysis at hand. Check the analysis report for the contrast definition, covariance or variance method, degrees-of-freedom method, and any stated approximation. When model diagnostics indicate serious misspecification, dependence, influential observations, or a poor distributional approximation, the reported \(t\)-based p-values may not have their intended error rates.

A practical diagnostic is internal consistency: calculate both one-sided tests and compare the result with the confidence interval using the matching confidence level and degrees of freedom. If one route says both tests pass while the other interval crosses a limit, check for a wrong tail, wrong margin, mismatched \(df\), or rounding. Also check whether the displayed estimate and \(SE\) were rounded; use unrounded output when available, especially near a boundary.

The direct model-output approach is efficient when the model has already provided a valid contrast estimate, standard error, and degrees of freedom. An alternative is to form the matching confidence interval and compare its endpoints with the limits; this is equivalent under the same \(t\)-based framework and often makes the decision easier to communicate. Re-deriving the contrast variance from model coefficients is appropriate only when the needed model output is unavailable and the analyst has the full fitted model and covariance information. It is not a substitute for checking that the contrast itself matches the scientific question.