Applying Equivalence Limits To Model-Derived Metrics

From a Model-Derived Ratio to an Equivalence Decision

Estimating a test-to-reference exposure ratio and deciding whether it meets equivalence criteria are related but distinct tasks. A model may estimate a ratio close to 1, yet uncertainty around that estimate may be too large for the confidence interval to fit within the prespecified equivalence limits. Conversely, the decision rule does not ask whether the estimated ratio is exactly 1: it asks whether the data support a sufficiently precise ratio within the chosen limits.

The model-derived metric might be AUC or Cmax calculated from fitted concentration-time profiles. The estimand and the method for estimating its geometric mean ratio are established before applying limits. Here the focus is the next step: assessing the ratio and its uncertainty against limits specified for the metric and comparison.

Definition: Let \(R\) be the test-to-reference geometric mean ratio for a model-derived exposure metric, and let \(L\) and \(U\) be its prespecified lower and upper equivalence limits, where \(0<L<U\). The equivalence criterion is that the confidence interval for \(R\) lies entirely within \([L,U]\). On the log scale, write \(\theta=\log(R)\); the corresponding limits are \(\log(L)\) and \(\log(U)\).

Limits must be specified for the particular decision. Limits of 0.80 and 1.25 are common for some conventional average bioequivalence assessments, but they are not automatic for every metric, product, or regulatory setting. The analysis should use the limits and confidence level in the applicable protocol or regulatory requirement, rather than selecting them after seeing the estimate.

Model-Based Interval and Decision Rule

Suppose the model estimates \(\hat{\theta}\), the log test-to-reference exposure ratio, with standard error \(SE\) and degrees of freedom \(df\). Under a model in which the standardized estimation error follows a \(t\) distribution with those degrees of freedom, a two-sided \(100(1-2\alpha)\%\) confidence interval for \(\theta\) is:

\hat{\theta}\ \pm\ t_{1-\alpha,df}\,SE

Exponentiating both endpoints gives the interval for \(R\). Because exponentiation is increasing, the lower endpoint remains the lower endpoint and the upper endpoint remains the upper endpoint. A common choice is \(\alpha=0.05\), giving a 90% interval. That interval is used for the usual two one-sided tests (TOST) equivalence decision at the 5% level.

Decision rule: Conclude equivalence under the specified rule only when the entire confidence interval for \(R\) is within \([L,U]\). Equivalently, the entire confidence interval for \(\theta\) must be within \([\log(L),\log(U)]\). If any part of the interval falls outside the limits, the analysis has not demonstrated equivalence; that result alone does not establish that the formulations are inequivalent.

This rule presumes that the confidence interval is appropriate for the model-derived estimand. In particular, the model and analysis must support the target comparison; the reported standard error must account for the uncertainty in the estimated contrast; and the stated \(t\)-based or normal approximation must be reasonable. If the interval is built from a nonlinear function of fitted model parameters, its standard error must reflect the relevant parameter uncertainty and covariance. A narrow interval is not reassuring if it was obtained by omitting important sources of uncertainty.

Why the Confidence-Interval Rule Matches TOST

The equivalence hypotheses on the log scale are \(H_0:\theta\leq\log(L)\) or \(\theta\geq\log(U)\), against \(H_a:\log(L)<\theta<\log(U)\). The null is a union of two ways equivalence can fail: the ratio may be at or below the lower limit, or at or above the upper limit. Both one-sided null hypotheses must be rejected to declare equivalence.

For the lower limit, the one-sided test rejects at level \(\alpha\) when the estimate is more than \(t_{1-\alpha,df}\) standard errors above \(\log(L)\). For the upper limit, it rejects when the estimate is more than that many standard errors below \(\log(U)\). Together, these conditions are:

\frac{\hat{\theta}-\log(L)}{SE}>t_{1-\alpha,df} \quad\text{and}\quad \frac{\log(U)-\hat{\theta}}{SE}>t_{1-\alpha,df}

Rearranging the first inequality gives \(\hat{\theta}-t_{1-\alpha,df}SE>\log(L)\). Rearranging the second gives \(\hat{\theta}+t_{1-\alpha,df}SE<\log(U)\). These are precisely the requirements that both endpoints of the \(100(1-2\alpha)\%\) interval lie between the log limits. Exponentiating gives the corresponding requirement on the ratio scale. This equivalence of the two procedures holds when the same estimator, standard error, degrees of freedom, and prespecified limits are used in both.

Worked Example: An Interval Fully Within the Limits

Suppose a hypothetical model estimates a test-to-reference AUC ratio of \(\exp(-0.0400)\) for a specified population estimand. The estimated log ratio is \(\hat{\theta}=-0.0400\), with \(SE=0.0350\) and \(df=24\). The prespecified limits are 0.80 and 1.25, and the decision uses a 90% confidence interval. Assume the model-based standard error and \(t\) approximation are appropriate for this analysis.

For \(df=24\), \(t_{0.95,24}\) is approximately 1.711. The margin of error on the log scale is \(1.711(0.0350)=0.0599\). Thus:

-0.0400\pm 0.0599=[-0.0999,\ 0.0199]

Exponentiating the estimate gives a point estimate of \(\exp(-0.0400)=0.9608\). Exponentiating the interval endpoints gives \([\exp(-0.0999),\exp(0.0199)]=[0.9049,1.020]\), to three significant figures. Both endpoints are within 0.80 and 1.25. Under the specified rule, the analysis demonstrates equivalence for this AUC ratio.

The conclusion concerns the interval and the prespecified limits, not whether the point estimate equals 1. A ratio estimate of 0.9608 is compatible with modestly lower test exposure; the equivalence decision is that the interval is sufficiently contained within the allowed range.

Uncertainty in a Contrast of Model Predictions

A model-derived ratio is often formed by subtracting predicted mean log exposures: \(\hat{\theta}=\hat{\eta}_T-\hat{\eta}_R\), where \(\hat{\eta}_T\) and \(\hat{\eta}_R\) are the model-based mean log exposures for test and reference. The standard error of this contrast depends on the uncertainty in both predictions and their covariance. In general, for random estimates \(A\) and \(B\), \(\operatorname{Var}(A-B)=\operatorname{Var}(A)+\operatorname{Var}(B)-2\operatorname{Cov}(A,B)\). Ignoring the covariance can make the interval too wide or too narrow.

Worked Example: Including Covariance in the Interval

In a hypothetical analysis, the estimated mean log AUC is 0.0250 units higher for test than reference, so \(\hat{\theta}=0.0250\) and the estimated ratio is \(\exp(0.0250)=1.025\). Suppose the model-based variances of the two estimated mean log exposures are 0.00250 and 0.00360, and their covariance is 0.00100. Assume the fitted model supports a \(t\)-based interval with \(df=30\), and the applicable hypothetical limits are 0.90 and 1.11.

The variance and standard error of the difference are:

\widehat{\operatorname{Var}}(\hat{\theta}) =0.00250+0.00360-2(0.00100)=0.00410, \qquad SE=\sqrt{0.00410}=0.0640

With \(t_{0.95,30}\approx1.697\), the margin is \(1.697(0.0640)=0.1086\). The log-scale interval is \(0.0250\pm0.1086=[-0.0836,0.1336]\). On the ratio scale this is approximately \([0.920,1.143]\). The point estimate, 1.025, is within the limits, but the upper confidence limit, 1.143, exceeds 1.11. The analysis therefore does not demonstrate equivalence under these limits.

If the covariance were omitted, the calculated variance would be \(0.00250+0.00360=0.00610\), which is larger here and would produce a wider interval. The direction of the error is not always the same: a negative covariance increases the variance of a difference, while a positive covariance decreases it. The covariance appropriate to the contrast should be taken from the fitted model’s uncertainty calculation.

When the Point Estimate Is Not Enough

A point estimate inside the limits is necessary for its confidence interval to be fully inside, but it is not sufficient. The interval also depends on precision. With an estimate near a limit, even a narrow interval may extend outside it. With an estimate nearer the centre, a wider interval may still fit. The decision is therefore about both estimated location and uncertainty.

Worked Example: A Point Estimate Inside, but an Interval Outside

Consider a hypothetical model-derived Cmax ratio estimate of 1.18, with a log-scale standard error of 0.0700 and \(df=20\). Assume 0.80 and 1.25 were prespecified limits and that the interval assumptions are appropriate. The estimate on the log scale is \(\hat{\theta}=\log(1.18)=0.1655\). For a 90% interval, use \(t_{0.95,20}\approx1.725\), giving a margin of \(1.725(0.0700)=0.1208\).

The log interval is \([0.1655-0.1208,\ 0.1655+0.1208]=[0.0447,0.2863]\). Exponentiating gives approximately \([1.046,1.331]\). Although the point estimate 1.18 is inside 0.80 and 1.25, the upper confidence limit is above 1.25. The result does not demonstrate equivalence. It also does not, by itself, prove that the true ratio exceeds 1.25: the interval includes values below that limit as well.

Diagnostics, Alternatives, and Limitations

Before making the decision, check that the interval answers the intended question. The target metric, formulation contrast, population or prediction target, confidence level, and equivalence limits should all match the prespecified analysis. Confirm that the standard error is for the log ratio, not for one formulation’s exposure alone, and that uncertainty in the fitted predictions has been propagated into the contrast. Verify that the endpoints were exponentiated from the log interval rather than constructed by adding and subtracting a symmetric margin directly on the ratio scale.

The interval rule is a convenient presentation of TOST, not a different evidential standard. Directly reporting the two one-sided tests can make clear which boundary is limiting, while the interval makes the estimated range and its relation to both limits easy to inspect. A model-derived contrast can also be calculated from predicted exposures and their covariance, or estimated directly as a model contrast when the parameterization permits it. These are alternative ways to obtain the same target contrast; they should agree when they use the same estimand and uncertainty calculation.

A \(t\)-based Wald interval relies on the fitted model providing a sufficiently reliable estimate, standard error, and reference distribution. Poor fit, weak identification, an unsuitable covariance structure, or a highly nonlinear transformation can undermine that approximation. Inspect model fit and estimation diagnostics relevant to the analysis, and investigate unstable estimates or standard errors rather than treating a software-generated interval as automatically valid. Profile-likelihood or resampling-based intervals may be considered when the Wald approximation is inadequate, provided the method is appropriate and prespecified or otherwise justified.

Finally, equivalence of a model-derived geometric mean ratio is not a claim that every individual has the same exposure under both formulations, nor does it establish equivalence of the full exposure distributions. Those are different targets. The conclusion is limited to the stated exposure metric, estimand, model, limits, and analysis assumptions.

Common Mistakes

Using the point estimate alone. A ratio inside the limits does not establish equivalence if its confidence interval crosses a limit. Apply the rule to both interval endpoints.
Calling a failed equivalence decision proof of inequivalence. An interval that extends beyond a limit means equivalence was not demonstrated under that analysis. It does not necessarily show that the true ratio is outside the limits.
Applying limits on the wrong scale. Compare ratio-scale limits with exponentiated confidence limits, or compare log-scale limits with the log interval. Do not compare one scale’s estimate with another scale’s limits.
Ignoring covariance. The uncertainty of a test-minus-reference contrast includes the covariance between the model predictions. Use the full contrast variance.
Choosing limits after seeing results. Equivalence limits define the decision and should be prespecified for the relevant metric and context.

Key takeaway: A model-based equivalence decision is made by comparing the full confidence interval for the exposure ratio with prespecified limits. Estimation supplies the ratio and its uncertainty; the limits and decision rule determine whether equivalence is demonstrated.

Check Your Understanding

Work through each question, then compare your reasoning with the solutions below.

A hypothetical log-ratio estimate is 0.0200, with \(SE=0.0400\) and \(df=25\). Using \(t_{0.95,25}=1.708\), calculate the 90% interval on both scales and decide whether equivalence is demonstrated for limits 0.80 and 1.25.
A ratio estimate is 0.79, and its confidence interval extends from 0.74 to 0.84. What conclusion follows under limits 0.80 and 1.25? What conclusion does not follow?
A log-scale 90% interval is \([-0.0800,0.1000]\). For limits 0.90 and 1.10, is equivalence demonstrated? Show the relevant transformation.
Why can omitting covariance between predicted test and reference log exposures change the equivalence decision?

Solutions. (1) The margin is \(1.708(0.0400)=0.0683\), so the log interval is \([-0.0483,0.0883]\). Exponentiating gives approximately \([0.953,1.092]\). Both limits are within 0.80 and 1.25, so equivalence is demonstrated under the stated assumptions and rule. (2) The interval crosses the lower limit of 0.80, so equivalence is not demonstrated. Because the interval also includes values above 0.80, this result does not prove that the true ratio is below 0.80 or otherwise establish inequivalence. (3) Exponentiating the endpoints gives \([\exp(-0.0800),\exp(0.1000)]=[0.923,1.105]\). The upper endpoint exceeds 1.10, so equivalence is not demonstrated. (4) The variance of the difference is the variance of the test prediction plus the variance of the reference prediction minus twice their covariance. Omitting that term changes the standard error and hence the interval width; depending on the sign of the covariance, the resulting interval can be too wide or too narrow and can alter whether it fits within the limits.