Reading Equivalence on a Confidence-Interval Plot

Reading the Plot Before Reading the Conclusion

A confidence-interval plot places three things on the same scale: the prespecified equivalence region, the estimated treatment effect, and an interval expressing uncertainty about that effect. The visual question is direct: does the entire confidence interval fit between the equivalence margins? If it does, the interval supports equivalence at the corresponding test level. If it does not, equivalence has not been established. A confidence interval entirely beyond a margin provides stronger information: it supports an effect outside the allowed region.

This is a way to read the result of the TOST procedure, not a replacement for choosing a valid model, defining the relevant effect, or setting clinically justified margins. The estimate, interval, and margins must refer to the same parameter, scale, and units. For example, an interval in assay-response units cannot be compared directly with margins expressed in percentage points.

A plot should show the margins as fixed reference points and the confidence interval as a horizontal segment, usually with a point marking the estimate. The estimate is informative about the interval’s centre, but the decision depends on both endpoints. These schematic number lines use the same margins, \(-10\) and \(10\), to illustrate the three main patterns:

Entirely inside:   −10 ├──────[────●────]──────┤ 10

Overlaps a margin:   −10 ├────[────●──────]────┤ 10

Entirely beyond the upper margin:   −10 ├────────────┤ 10    [────●────]

In each line, the vertical marks at \(-10\) and \(10\) define the equivalence region, the bracketed segment is the confidence interval, and the dot is the point estimate. The drawings are schematic rather than to scale. A real plot should label the numerical axis, parameter, units, confidence level, and margins.

The Assumptions Behind the Visual Rule

Before classifying a plot, check that the interval is suitable for the equivalence decision. It must be an interval for the treatment-effect parameter specified by the analysis, based on a valid model and standard error. In a crossover study, for instance, the estimate should come from the intended treatment contrast and the analysis should account appropriately for the design. The interval’s coverage relies on the model assumptions and the adequacy of its uncertainty estimate; a poorly specified model does not become reliable merely because its interval fits between two lines.

The margins must be set before interpreting the observed result and justified for the scientific or clinical question. They are not confidence limits and should not be moved to make an interval fit. The interval and margins must also be on the same scale. For the plots and numerical examples here, assume a scalar treatment effect measured in assay-response units, with margins \(L=-10\) and \(U=10\) units. Assume the effect estimate and its standard error come from an appropriate analysis, and that the specified confidence interval has its stated coverage.

For a TOST at significance level \(\alpha\), the corresponding two-sided confidence interval has confidence level \(1-2\alpha\). Thus, when \(\alpha=0.05\), a 90% interval is the one used for the usual interval-based TOST decision. A 95% interval is wider and is not the matching interval for that same test level. The confidence level is part of the interpretation, not a decorative label on the plot.

Why the Interval Rule Matches Both Tests

The TOST procedure tests whether the treatment effect is above the lower margin and below the upper margin. For a normal-based illustration, let \(\hat{\theta}\) be the estimate, \(SE\) its standard error, and \(c\) the positive critical value used for the one-sided tests. The confidence interval associated with both one-sided bounds is

The lower-bound test rejects effects at or below \(L\) when the lower one-sided confidence bound exceeds \(L\). The upper-bound test rejects effects at or above \(U\) when the upper one-sided confidence bound is below \(U\). Both rejections are required for equivalence. The interval rule follows by combining those two inequalities:

This is the reason the entire interval must fit inside the margins. If even one endpoint reaches or passes a margin, at least one of the two requirements is unmet. The graphical rule is not based on whether the point estimate alone lies inside the equivalence region.

Three Visual Classifications and Their Meaning

The distinction between “not established” and “shown to be outside” matters. Both a wide interval that straddles a margin and a narrow interval far beyond a margin fail the equivalence criterion, but they tell different stories. The first leaves substantial uncertainty about where the effect lies; the second places the whole interval outside the permitted range.

Diagnostics and Common Misreadings

A plot can reveal a decision pattern, but it can also reveal problems that deserve investigation. First verify the plotted endpoints against the reported interval and confirm that the estimate is centred correctly when a symmetric interval is intended. Check that the two margins are the ones specified in the analysis plan, and that they are plotted on the same axis and in the same units as the interval. Then confirm the confidence level matches the TOST significance level. If any of these labels or values are missing, a visual classification may be impossible or misleading.

An interval unexpectedly wide may reflect a large standard error, limited information, or substantial variability. That is a reason to inspect the analysis and data quality, not a reason to relax the margins after seeing the result. An interval unexpectedly displaced from the estimate may be valid for some methods, but it should prompt a check of the reported method and the plotted values rather than an assumption that the graph is wrong.

• Mistake: “The estimate is inside the margins, so equivalence is established.” Correction: the estimate is only the centre or point summary. Its interval must fit inside both margins. A point estimate inside the region can still have an interval that crosses a margin.
• Mistake: “The interval crosses a margin, so the treatment is non-equivalent.” Correction: an overlapping interval fails to establish equivalence, but it may still include effects inside the region. It is inconclusive unless the entire interval lies outside.
• Mistake: “The interval does not include zero, so equivalence is impossible.” Correction: the equivalence region is determined by \(L\) and \(U\), not by zero alone. An interval can exclude zero and still lie wholly between the equivalence margins.
• Mistake: comparing a 95% interval with a decision rule that calls for a 90% interval. Correction: use the confidence level corresponding to the chosen TOST significance level. A different interval may answer a different inferential question.

How the Plot Relates to Other Summaries

The number-line plot is a visual expression of the confidence-interval rule for TOST. The two one-sided test results give the same decision when they use the corresponding assumptions and significance level: equivalence requires rejection of both one-sided null hypotheses. Reading the interval is often quicker, especially when comparing results across several analyses, but the plot does not replace checking the underlying estimate, uncertainty calculation, and test calibration.

A point estimate with margins but no interval is insufficient for a statistical equivalence conclusion. A conventional test of a zero effect also answers a different question: failing to find a statistically significant difference from zero does not establish that the effect is small enough to be equivalent. Conversely, a confidence interval can exclude zero and remain wholly within the equivalence region. The equivalence margins, rather than zero as a special point, define the decision.

The plot also has limits. It summarizes uncertainty about one parameter and does not communicate every modeling choice, possible bias, or clinical consideration. A visually decisive interval cannot rescue inappropriate margins, an unsuitable analysis, or a mismatch between the parameter estimated and the one relevant to the decision. Use it as a compact reading tool alongside the stated analysis and its assumptions.