Why the Reference Distribution Matters
A TOST calculation needs a critical value to turn an estimate and its standard error into confidence limits. That value determines the interval’s width. Choosing a normal critical value when the supplied model calls for a t critical value can make the interval too narrow and can change the equivalence conclusion. The reverse mistake can make an interval unnecessarily wide.
The choice does not come from TOST itself, the equivalence margins, or a universal sample-size cutoff. It comes from the inferential model for the estimate and its standard error. That model specifies how the standardized estimation error is calibrated: against a t distribution with stated degrees of freedom, or against a standard normal distribution. First identify that calibration; then calculate the interval.
For example, with \(\alpha=0.05\), the corresponding interval is a 90% interval and the upper-tail quantile is the 0.95 quantile: \(t_{0.95,df}\) or \(z_{0.95}\). The normal quantile is approximately 1.645. A t quantile depends on \(df\) and is greater than 1.645 for finite degrees of freedom; it approaches 1.645 as \(df\) grows.
How the Model Determines the Critical Value
In a classical t model, an estimate is normally distributed around its target conditional on the model, and the variance is estimated from the data. When the variance estimate has the appropriate independent chi-square sampling distribution, the standardized error follows a t distribution. The degrees of freedom describe the variance estimate’s sampling information. In a one-sample normal model with an estimated variance, for example, the usual t statistic has \(n-1\) degrees of freedom. Other models can supply different degrees of freedom.
A normal reference is appropriate when the stated inferential model calibrates the standardized error as standard normal. This can occur when the relevant variance is treated as known, or when the model uses a large-sample normal approximation, such as a Wald procedure. These are different justifications. A large sample may motivate a normal approximation, but there is no universal rule that a particular sample size automatically changes a specified t procedure into a z procedure.
In practice, the inferential output or analysis specification may identify the distribution and degrees of freedom directly. If it reports a t-based interval, use its t reference and degrees of freedom. If it specifies a normal-based Wald interval, use the normal quantile. The numerical fact that a standard error has been estimated does not, by itself, tell you which calibration the model uses.
Why the TOST Interval Uses These Quantiles
Let \(\theta\) be the parameter being assessed for equivalence. The equivalence region is \(L<\theta<U\). TOST tests whether the parameter is above the lower limit and below the upper limit. Both one-sided tests must reject their respective null hypotheses. With a common standard error and the stated reference distribution, the two boundary tests use the same critical value \(c\), where \(c=t_{1-\alpha,df}\) or \(c=z_{1-\alpha}\).
The lower-bound test rejects when the estimate is sufficiently above \(L\); the upper-bound test rejects when it is sufficiently below \(U\). Writing these two requirements gives:
Because \(SE>0\), the first inequality is equivalent to \(\hat{\theta}-cSE>L\), and the second is equivalent to \(\hat{\theta}+cSE<U\). Thus both tests reject exactly when the interval \((\hat{\theta}-cSE,\hat{\theta}+cSE)\) lies inside the equivalence region. The quantile used for \(c\) must therefore match the inferential model for the tests.
Each test is one-sided at level \(\alpha\), so the central interval has confidence level \(1-2\alpha\): it leaves probability \(\alpha\) in each tail under the calibrated reference distribution. This is why the quantile is \(1-\alpha\), not \(1-\alpha/2\). For \(\alpha=0.05\), using \(t_{0.975,df}\) or \(z_{0.975}\) would instead produce a central 95% interval, which is wider than the interval corresponding to the two 5% TOST tests.
Worked Example: A Small-Degrees-of-Freedom t Model
Suppose a hypothetical analysis estimates a mean difference of \(\hat{\theta}=1.8\) units, with \(SE=1.8\) units. The prespecified equivalence limits are \(L=-5\) units and \(U=5\) units. The inferential model specifies a t reference distribution with \(df=9\), and \(\alpha=0.05\). The model assumptions are taken to be satisfied: the standardized estimation error follows the specified t calibration, and the reported standard error and degrees of freedom belong to this same model.
The required quantile is \(t_{0.95,9}=1.8331\). The interval is:
The lower limit is inside the equivalence region, but the upper limit, \(5.0996\) units, exceeds \(U=5\) units. The interval is not wholly inside the margins, so this calculation does not establish equivalence.
For comparison, using a normal quantile would give \(z_{0.95}=1.6449\) and:
That narrower interval lies within the margins and would produce an equivalence conclusion. But it does not follow the supplied t model. The changed conclusion illustrates why selecting a reference distribution is not a cosmetic choice.
Worked Calculations Under Different Models
Worked Example: A Model That Specifies a Normal Reference
Consider a hypothetical estimate of \(\hat{\theta}=0.4\) units with \(SE=0.9\) units and equivalence limits \(-2.5\) and \(2.5\) units. The supplied inferential model specifies a normal-based Wald procedure, so the standardized error is calibrated using the standard normal distribution. Assume the model’s conditions for this approximation are adequate and use \(\alpha=0.05\).
Use \(z_{0.95}=1.6449\), not a t quantile. The margin of error is:
Therefore, the interval is:
Both limits lie strictly between \(-2.5\) and \(2.5\) units. Under the stated normal model, the interval supports equivalence at the corresponding TOST level. The choice of normal critical value is justified here by the supplied model, not merely by the presence of a standard error.
Worked Example: How Degrees of Freedom Change the Limits
A hypothetical estimate is \(\hat{\theta}=-0.3\) units with \(SE=0.7\) units. The equivalence limits are \(-1.5\) and \(1.5\) units, and \(\alpha=0.05\). Suppose two candidate analyses are being compared: one specifies a t reference with \(df=4\), and the other specifies a t reference with \(df=24\). Each calculation must use the degrees of freedom from its own inferential model; this comparison does not imply that the models are interchangeable.
For \(df=4\), \(t_{0.95,4}=2.1318\). The interval is:
The lower limit is below \(-1.5\) units, so the interval extends beyond the lower equivalence limit. It does not establish equivalence.
For \(df=24\), \(t_{0.95,24}=1.7109\). The calculation becomes:
Both limits are inside \((-1.5,1.5)\) units, so this second model’s interval supports equivalence. The smaller t critical value at 24 degrees of freedom gives a narrower interval. Near a margin, even a modest difference in the critical value can affect the decision.
Diagnostics, Alternatives, and Limitations
Before calculating, check that the estimate, standard error, reference distribution, and degrees of freedom (if applicable) come from the same inferential model. Confirm the intended \(\alpha\) and whether the requested interval is the TOST interval at confidence level \(1-2\alpha\). Then check the arithmetic and compare both endpoints with the margins on the same scale and in the same units.
A useful model summary is:
| Supplied inferential model | Critical value for TOST at level \(\alpha\) | What to verify |
|---|---|---|
| t-based | \(t_{1-\alpha,df}\) | The stated degrees of freedom apply to this estimate and standard error. |
| Normal-based | \(z_{1-\alpha}\) | The model specifies normal calibration, such as a suitable Wald approximation. |
The main alternative to the specified reference is not a competing choice to select by intuition; it is a different inferential model. A normal approximation can be close to a t calculation when \(df\) is large, but that does not license replacing the model’s stated procedure. Conversely, using a t quantile simply because a standard error was estimated may not match a model that specifies normal-based inference.
If the model documentation does not identify the reference distribution or degrees of freedom, the estimate and standard error alone are insufficient to resolve the choice. Consult the analysis specification or obtain the missing inferential information rather than guessing. Also, the interval method assumes the supplied model is a suitable description of estimation uncertainty. A correct quantile cannot repair a misspecified model, an inappropriate standard error, or an invalid approximation.
Common Mistakes
- Using the normal quantile because the sample seems large. Use the reference distribution stated by the inferential model. A large sample may support a normal approximation, but it does not override a specified t procedure.
- Using t automatically because the standard error is estimated. Estimated standard errors also appear in normal-based Wald procedures. The model’s calibration, not that fact alone, determines the critical value.
- Ignoring degrees of freedom. A t critical value is not a single fixed number. Its value depends on \(df\), so using the wrong degrees of freedom changes the interval.
- Using a 95% interval for two tests at \(\alpha=0.05\). The matching central interval is 90%, with critical quantile at 0.95. A 95% interval uses the wrong tail allocation for this TOST rule.
- Choosing the distribution that gives the preferred result. The model and analysis plan determine the reference distribution; the result must not determine it.
Check Your Understanding
Answer each question using the stated model. Brief solutions follow the questions.
- A TOST model specifies \(df=12\) and \(\alpha=0.05\). Which quantile is required, and what confidence level does the interval have?
- A model specifies a normal-based Wald interval and reports an estimated standard error. Should you automatically switch to a t critical value? Why or why not?
- For \(\hat{\theta}=0.2\) units, \(SE=0.5\) units, and \(L=-1.5\), \(U=1.5\) units, a normal model is specified. Calculate the TOST interval at \(\alpha=0.05\) and state the interval-based conclusion.
- Why is \(t_{0.975,df}\) not the appropriate critical value for a TOST interval when each one-sided test uses \(\alpha=0.05\)?
Solutions. (1) Use \(t_{0.95,12}\); the matching interval is 90%. (2) No. Use the normal quantile because the model specifies normal calibration; an estimated standard error alone does not imply a t reference. (3) Use \(z_{0.95}=1.6449\): \(0.2\pm1.6449(0.5)=0.2\pm0.8225=(-0.6225,1.0225)\) units. Both endpoints lie inside \((-1.5,1.5)\) units, so the interval supports equivalence under the stated model. (4) The two one-sided tests allocate \(\alpha=0.05\) to each relevant tail, so the central interval uses quantile \(1-\alpha=0.95\), not \(1-\alpha/2=0.975\).