Why Rounding Matters Near an Equivalence Boundary
A TOST conclusion is determined by comparing both confidence-interval endpoints with the equivalence limits. When an endpoint is comfortably inside or outside a limit, ordinary display rounding is unlikely to confuse the interpretation. Near a limit, however, a small rounding difference can make an interval appear to touch or cross a boundary when the unrounded result does not—or make a result that narrowly fails appear to pass.
The key distinction is between rounding values for display and rounding values before analysis. Display rounding is often necessary, but the decision should be based on the full-precision values used by the analysis, not on a recomputation from rounded estimates or on a visual comparison of rounded endpoints. This tutorial assumes that the equivalence limits have already been specified for the analysis; it focuses on how precision affects the calculation and its reporting.
For an estimate \(\hat{\theta}\) with standard error \(SE\), a two-sided interval can be written as
where \(c\) is the appropriate critical value for the analysis model and confidence level. For example, a large-sample normal procedure at \(\alpha=0.05\) uses a 90% interval and \(c\approx1.645\). If a \(t\) procedure is appropriate, its critical value depends on the degrees of freedom. The same model and critical value must be used consistently in the calculation and interpretation.
Where Rounding Can Enter the Decision
Rounding can affect the equivalence assessment at several distinct points. An estimate or standard error rounded before computing the interval can change the calculated endpoint. A margin rounded before comparison can move the decision boundary. Finally, rounding an already calculated endpoint can hide which side of a boundary it lies on. These are different problems, so a defensible report should make clear that displayed precision was not used to redefine the analysis.
Suppose the upper limit is \(U\), and the calculated upper confidence endpoint is \(\hat{\theta}+cSE\). Define the distance from that endpoint to the upper boundary as \(g=U-(\hat{\theta}+cSE)\). A positive \(g\) means the endpoint is inside the upper limit; a negative \(g\) means it exceeds the limit. If the estimate, standard error, critical value, and limit are rounded, their combined changes can be larger than this gap. The decision is vulnerable whenever rounding is not small relative to the distance to the boundary.
A Short Justification for Checking the Boundary Gap
The effect of rounding on the upper endpoint can be expressed directly. Let the rounded inputs differ from the analysis inputs by \(\delta_\theta\), \(\delta_{SE}\), and \(\delta_c\), respectively. The rounded-input endpoint is
Subtracting the original endpoint \(\hat{\theta}+cSE\) gives the change
Thus the rounded-input endpoint can move by the combined input changes, including the product term. If the absolute endpoint change is smaller than the original gap to the limit, the endpoint remains on the same side of that limit. If it is as large as, or larger than, the gap, rounding could change the apparent comparison. A rounded limit adds another change to the comparison, so the safe check is whether the change in the endpoint-to-limit difference could reach zero. This is a diagnostic, not a substitute for using the original values.
Worked Example: Rounding Inputs Changes the Recomputed Decision
Suppose a hypothetical TOST assesses a mean difference in a measurement, in units, against limits \(L=-0.04000\) and \(U=0.04000\). A large-sample normal analysis uses \(\alpha=0.05\), so \(c=1.645\). The unrounded analysis estimate is \(\hat{\theta}=0.02004\), and \(SE=0.01213\). Assume the estimator and standard error are valid for the stated normal approximation and that these limits are the limits used in the analysis.
Using the analysis values, the interval is
Therefore, the endpoints are \(0.00008615\) and \(0.03999385\). The lower endpoint exceeds \(-0.04000\), and the upper endpoint is below \(0.04000\). The interval is strictly within both limits, so this calculation supports equivalence.
Now suppose the estimate and standard error are each rounded to two decimal places before someone recomputes the interval: \(\hat{\theta}=0.02\), \(SE=0.01\). The rounded-input upper endpoint becomes
In this particular recomputation the decision still appears equivalent, but the endpoint is no longer the endpoint from the analysis. If instead the rounded standard error is reported to three decimal places, \(SE=0.012\), the recomputed upper endpoint is \(0.02+(1.645)(0.012)=0.03974\), again not the original result. This example shows why rounding inputs and recalculating is not a valid way to reproduce an analysis, even when it happens not to reverse the conclusion. In more boundary-sensitive cases, it can.
Worked Example: A Rounded Endpoint Hides a Narrow Failure
Consider another hypothetical analysis on the same scale, with \(L=-0.04000\), \(U=0.04000\), and \(c=1.645\). Let \(\hat{\theta}=0.02010\) and \(SE=0.01215\). Assume the same normal-model conditions as in the preceding example.
The upper endpoint is
Because \(0.04008675>0.04000\), this interval extends beyond the upper equivalence limit. Equivalence is not established by this interval, regardless of how close the endpoint is to the limit. The lower endpoint is \(0.02010-0.01998675=0.00011325\), which is above \(L\); it is the upper comparison that fails.
If the upper endpoint is displayed to three decimal places, it becomes \(0.040\), as does the limit \(0.04000\) at that display precision. The displayed values look equal, but the unrounded endpoint is outside the limit. Treating the rounded equality as if it were safely inside would reverse the decision. The correct conclusion follows from the full-precision comparison: the upper endpoint exceeds the limit.
Worked Example: Rounding a Limit Can Make a Failing Interval Look Acceptable
Suppose the upper equivalence limit is obtained from a ratio limit of \(1.05\) on a natural-log scale. The corresponding upper limit is \(U=\ln(1.05)\approx0.0487902\). In a hypothetical analysis, the upper confidence endpoint is \(0.04890\), and the lower endpoint is \(-0.02000\). Assume the interval was calculated using the appropriate model and confidence level.
Using the unrounded upper limit,
so the interval exceeds the upper equivalence limit and does not satisfy the interval criterion for equivalence. But if the limit is rounded to three decimal places, it appears as \(0.049\), and the rounded comparison gives \(0.04890<0.049\). That apparent pass results from moving the boundary through rounding; it is not the comparison defined by the analysis. Keep the transformed limit at adequate precision for the decision, and round it only for display.
Diagnostics and a Defensible Reporting Approach
A useful diagnostic is to compare the distance to each boundary with the precision of the numbers being displayed. If the reported endpoint and limit round to the same value, or to adjacent values, the rounded display may be insufficient to show the direction of the comparison. That is a cue to inspect the unrounded analysis output, not a cue to infer a result from the rounded numbers. The lower and upper comparisons must both be checked; a clear result at one boundary does not resolve a close result at the other.
Calculate the confidence interval and compare it with the limits using the unrounded estimate, standard error, critical value, and limits used in the analysis.
Confirm separately that the lower endpoint is above \(L\) and the upper endpoint is below \(U\). If either comparison fails, the interval does not establish equivalence.
Report enough digits to communicate the estimate and interval usefully. When an endpoint is close to a limit, provide additional digits or state the comparison using unrounded values.
State that the decision was made using full-precision values, and avoid recomputing the interval or changing a limit from rounded display values.
The same principle applies if a TOST is reported using its two one-sided tests rather than the corresponding confidence interval. Under the same model, standard error, limits, and significance level, the two tests and the \(100(1-2\alpha)\%\) interval criterion express the same decision. Rounding a test statistic, p-value, or endpoint for presentation must not change the conclusion reached from the analysis values. If rounded outputs appear to disagree, verify the underlying values and that the same model and confidence level were used.
Rounding is not the only reason a boundary comparison can be unreliable. A wrong standard error, critical value, transformation, or confidence level can also produce an incorrect interval. Precision checks cannot repair those analysis errors; they only help diagnose whether a correct calculation has been obscured by its display. Likewise, reporting more decimal places than the data or model supports does not make an estimate more precise. The goal is to avoid a display that hides the relevant comparison, not to imply unwarranted accuracy.
Common Mistakes
- Recomputing from rounded estimates. A displayed estimate and standard error are not necessarily sufficient to reproduce the original interval. Use the analysis values for the decision.
- Calling a rounded tie a pass. If an endpoint and limit both display as \(0.040\), that alone does not show that the endpoint is below the limit. Inspect their unrounded values.
- Rounding a limit before comparison. A rounded limit can move the boundary and create an apparent pass or failure. Compare against the limit used in the analysis.
- Reporting only a binary conclusion. A conclusion without the estimate, interval, limits, or enough precision to interpret a close boundary gives the reader no way to understand the decision.
- Adding digits as a cure for an incorrect analysis. More display precision cannot correct an inappropriate model, standard error, or critical value.
Check Your Understanding
Answer each question using the full-precision decision rule, not only the displayed rounded values.
- A 90% interval is \([-0.01004,\,0.03002]\), and the limits are \(-0.01000\) and \(0.04000\). Does the interval establish equivalence?
- An upper endpoint is \(0.05004\), while the upper limit is \(0.05000\). Both display as \(0.050\) to three decimal places. What is the correct comparison?
- Why can rounding the estimate and standard error before calculating a confidence interval be misleading even if the final decision happens to stay the same?
- Give one defensible way to report a result whose endpoint is very close to an equivalence limit.
Solutions. (1) No. The lower endpoint, \(-0.01004\), is below the lower limit \(-0.01000\), so the interval is not strictly inside both limits. (2) The endpoint exceeds the limit because \(0.05004>0.05000\); the rounded display does not change that comparison. (3) Rounding changes the inputs and can change the calculated endpoints, so the recomputed interval may not match the analysis interval and could even fall on the other side of a boundary. (4) Report the interval and limits with enough digits to show their relative positions, or state explicitly that the comparison used unrounded analysis values and provide the decision.