Multiple Test Formulations Compared With One Reference

Why Several Test Formulations Require a Clear Claim

A study may compare multiple test formulations with the same reference—for example, several candidate formulations assessed against one reference product. The common reference makes the comparisons related, but it does not by itself determine how to handle multiplicity. The correct procedure depends on the claim the study is designed to support: that every test formulation is bioequivalent, or that one or more selected formulations are bioequivalent.

For a conventional crossover analysis, pharmacokinetic endpoints such as AUC or Cmax are generally analyzed on the logarithmic scale. Each treatment contrast then compares a test formulation with the reference, and exponentiating that contrast gives a geometric mean ratio. The analysis should report every prespecified contrast and its interval, then apply the multiplicity rule specified for the intended claims. A procedure chosen after inspecting the results can distort the error rate and is not a substitute for prespecification.

Definition: For test formulation \(T_j\) and reference formulation \(R\), let \(\theta_j\) denote the population mean log-endpoint difference, \(T_j-R\). The geometric mean ratio is \(\exp(\theta_j)\). Under the usual average bioequivalence limits, formulation \(T_j\) meets the interval criterion when its confidence interval for \(\theta_j\) lies entirely within \([\log(0.80),\log(1.25)]\), equivalently when the exponentiated interval lies entirely within \([0.80,1.25]\).

Model and Conditions for the Comparisons

In a crossover study, a suitable mixed model for the log-transformed endpoint includes fixed effects for treatment, period, and sequence, together with a random subject effect nested within sequence. In schematic form, an observation for subject \(i\), in sequence \(s\), period \(p\), and treatment \(k\) can be written as:

Y_{ispk}=\mu+\text{Sequence}_s+\text{Subject}_{i(s)} +\text{Period}_p+\text{Treatment}_k+\varepsilon_{ispk}.

Here \(Y_{ispk}\) is the log endpoint, and the subject-within-sequence term is random. This specification avoids treating subject indicators and sequence indicators as redundant fixed effects. The treatment contrasts are estimated from the fitted model, with standard errors and degrees of freedom derived using its covariance structure and the prespecified estimation method.

The model relies on a suitable log-scale mean structure, an adequately specified covariance structure, and residual behavior compatible with the chosen inference procedure. The treatment and period effects must be estimable under the study design; there must be no unaddressed data-integrity problem or protocol feature that makes the planned comparison inappropriate. In a crossover, each treatment contrast must be supported by the observed treatment-period pattern and the design’s within-subject comparisons. Data and model diagnostics should be reviewed before relying on the estimates, especially for influential subjects, unusual exposure profiles, missing observations, or patterns suggesting carryover. A concern does not automatically dictate exclusion or a new model, but it may require investigation and prespecified sensitivity analysis.

The model does not make an invalid design valid: for example, it cannot recover a treatment contrast that the design does not identify. Nor does a well-fitting model establish bioequivalence on its own. Bioequivalence is assessed by comparing the treatment contrast and its uncertainty with the prespecified limits.

Worked Example: One Test Formulation Versus the Reference

Suppose a hypothetical two-treatment crossover analysis of log AUC estimates the contrast \(T_1-R\) as \(0.0400\), with \(SE=0.0300\) and \(df=24\). Assume diagnostics give no material concern with the fitted model, and use a 90% confidence interval for the ordinary, unadjusted TOST assessment. The two-sided 90% \(t\) critical value is approximately \(1.711\).

The interval on the log scale is:

0.0400\pm1.711(0.0300) =0.0400\pm0.05133 =[-0.01133,\ 0.09133].

Exponentiating the endpoints gives the geometric mean ratio interval:

[\exp(-0.01133),\ \exp(0.09133)] \approx[0.9887,\ 1.0956].

The interval lies within \([0.80,1.25]\), so this comparison meets the usual 5% TOST criterion for average bioequivalence. The point estimate corresponds to a ratio of \(\exp(0.0400)\approx1.041\), or about 104.1%. This conclusion concerns this contrast and this analysis; it does not decide how a family of multiple formulation claims should be handled.

From Two One-Sided Tests to an Interval

For a single test formulation, let \(L=\log(0.80)\) and \(U=\log(1.25)\). The bioequivalence alternative is \(L<\theta_j<U\). The null is the union of the two possibilities that the true contrast is at or below the lower limit, or at or above the upper limit. Two one-sided tests (TOST) reject both boundary null hypotheses at level \(\alpha\) only when the contrast is sufficiently far from each limit.

With a \(t\)-based standard error and \(df\), the lower test rejects when the estimate minus its one-sided critical margin exceeds \(L\), and the upper test rejects when the estimate plus that margin is below \(U\). Thus both tests reject exactly when the corresponding two-sided interval is contained in the limits:

\widehat{\theta}_j-t_{1-\alpha,df}SE_j>L \quad\text{and}\quad \widehat{\theta}_j+t_{1-\alpha,df}SE_j<U.

At \(\alpha=0.05\), \(t_{1-\alpha,df}=t_{0.95,df}\); the interval between these two bounds is a 90% two-sided confidence interval. This is why the ordinary TOST decision uses a 90%, rather than a 95%, interval. The result concerns the specified equivalence limits and error rate; it is not a statement that the parameter has a particular probability of lying in the computed interval.

Multiplicity Depends on the Claim

There are two common claim structures. If the only intended conclusion is that all test formulations are bioequivalent, the global alternative requires every formulation to meet its TOST criterion. This is an intersection-union test: the global null is that at least one formulation is not equivalent, and the global alternative is that all are equivalent. Testing each formulation’s TOST at level \(\alpha\) and declaring the global claim only if every TOST rejects controls the global type I error at no more than \(\alpha\). If any one formulation is truly outside the equivalence region, a false global declaration requires falsely rejecting that formulation’s null, an event with probability no greater than \(\alpha\).

That reasoning does not justify unadjusted testing when the study may claim that whichever formulation happens to pass is bioequivalent. Such a procedure permits selection among several comparisons and can inflate the probability of at least one false positive claim. A prespecified family-wise procedure, such as Bonferroni adjustment, can address that claim structure. For \(m\) comparisons and family-wise level \(\alpha\), Bonferroni tests each comparison’s TOST at level \(\alpha/m\). The matching two-sided TOST interval uses critical value \(t_{1-\alpha/m,df}\), or confidence level \(1-2\alpha/m\). These are adjusted decision intervals for the TOST claims; they should not be mislabeled as ordinary 90% intervals.

Derivation: If each of \(m\) comparisons is tested using level \(\alpha/m\), the probability of at least one false rejection is no greater than the sum of the \(m\) individual false-rejection probabilities. Therefore \(P(\text{at least one false rejection})\leq m(\alpha/m)=\alpha\), regardless of dependence between contrasts. For a TOST at level \(\alpha/m\), each one-sided bound uses \(t_{1-\alpha/m,df}\). Containment of the resulting interval in \([L,U]\) is equivalent to rejecting both one-sided null hypotheses for that comparison.

Because all tests share a reference, the estimated contrasts are generally correlated. Bonferroni remains valid without requiring independence, though it may be conservative. A Dunnett-type simultaneous procedure can use the common-reference covariance to gain precision when its assumptions and implementation are appropriate. A stepwise procedure such as Holm’s method is another possible family-wise approach, but interval reporting must match the procedure used for the decisions. The governing rule is to use the prespecified method and explain exactly which family of claims it protects.

Worked Example: Three Comparisons With Bonferroni Adjustment

Consider a hypothetical crossover study comparing three test formulations with one reference on log AUC. Suppose the model diagnostics support the planned analysis, the comparisons have \(df=24\), and the prespecified objective permits claims about whichever formulations pass. The family-wise level is \(\alpha=0.05\), so each TOST is conducted at \(0.05/3=0.01667\). The relevant two-sided interval has confidence level \(1-2(0.01667)=0.9667\), and the critical value \(t_{0.98333,24}\) is approximately \(2.258\).

The following model estimates and standard errors are hypothetical. Each interval is calculated as \(\widehat{\theta}_j\pm2.258SE_j\), and then exponentiated.

Contrast	Estimate	SE	Adjusted log-scale interval	Ratio interval	Conclusion
\(T_1-R\)	-0.0150	0.0400	[-0.1053, 0.0753]	[0.9000, 1.0782]	Meets limits
\(T_2-R\)	0.0350	0.0450	[-0.0666, 0.1366]	[0.9356, 1.1464]	Meets limits
\(T_3-R\)	0.1200	0.0600	[-0.0155, 0.2555]	[0.9846, 1.2911]	Does not meet upper limit

For instance, the first interval is obtained by:

-0.0150\pm2.258(0.0400) =-0.0150\pm0.09032 =[-0.1053,\ 0.0753],

and exponentiation gives \([0.9000,1.0782]\). The third interval is:

0.1200\pm2.258(0.0600) =0.1200\pm0.13548 =[-0.0155,\ 0.2555],

which exponentiates to approximately \([0.9846,1.2911]\). Its upper endpoint exceeds 1.25. Thus \(T_1\) and \(T_2\) meet the prespecified Bonferroni-adjusted TOST criterion, while \(T_3\) does not. The correct report retains all three results; it does not treat the two passing formulations as evidence that all three are bioequivalent.

Worked Example: An “All Formulations” Claim

Suppose instead that the sole prespecified claim is that all three test formulations are bioequivalent. Each of the three ordinary TOSTs is performed at \(\alpha=0.05\), using its 90% interval. For a hypothetical fit with \(df=24\), the critical value is \(t_{0.95,24}\approx1.711\). Let the estimated log contrasts and standard errors be \(0.0100\) and \(0.0300\) for \(T_1-R\), \(-0.0200\) and \(0.0350\) for \(T_2-R\), and \(0.0150\) and \(0.0400\) for \(T_3-R\). Assume the crossover model and diagnostics are acceptable.

The intervals are:

T_1-R:\quad 0.0100\pm1.711(0.0300) =[-0.04133,\ 0.06133],

T_2-R:\quad -0.0200\pm1.711(0.0350) =[-0.07989,\ 0.03989],

T_3-R:\quad 0.0150\pm1.711(0.0400) =[-0.05344,\ 0.08344].

All three log-scale intervals lie within \([\log(0.80),\log(1.25)]\), approximately \([-0.2231,0.2231]\). Their ratio intervals are approximately \([0.9595,1.0632]\), \([0.9232,1.0407]\), and \([0.9480,1.0870]\), respectively, also within \([0.80,1.25]\). The global conjunctive claim is therefore supported at the prespecified level. This conclusion follows from requiring every comparison to pass; it does not license a later, unplanned search for whichever individual result appears most favorable.

Reporting, Diagnostics, and Common Mistakes

A useful results table identifies the endpoint and analysis scale, the contrast direction, each estimate and standard error, the degrees of freedom, the interval and its confidence level, the exponentiated ratio interval, and the decision under the prespecified procedure. State whether the multiplicity family contains all test-versus-reference comparisons for an endpoint, and clarify whether the intended conclusion is conjunctive or allows selected claims. When analyses cover several endpoints or additional claim families, their scope should also be described rather than silently treated as part of the same adjustment.

Choosing a correction after seeing the results. This makes the advertised error control unreliable. Prespecify the family of claims and the procedure, then report any deviations transparently.
Calling every interval “90%” without checking the test level. A TOST at \(\alpha/m\) uses a \(1-2\alpha/m\) interval, not the ordinary \(1-2\alpha\) interval. Report the actual level and method.
Applying no adjustment to selective claims. Requiring every comparison to pass supports a conjunctive global claim at level \(\alpha\); selecting any passing comparison is a different claim structure and can require family-wise control.
Using an inappropriate fixed-effects parameterization. If subject indicators are fixed, sequence indicators can be redundant because subjects belong to sequences. Use a valid mixed model with subject(sequence) random, or omit redundant sequence effects in a fixed-subject formulation.
Reporting only ratios or only decisions. Ratios aid interpretation, but the log-scale contrasts and intervals show the model-scale uncertainty. Include both and state units for the endpoint, such as AUC or Cmax.

Multiplicity adjustment cannot remedy poor data quality, a misspecified model, or inadequate information. Conversely, a wide interval that crosses an equivalence limit is not proof that the formulations differ; it means the study has not established that the interval lies wholly inside the limits. Interpretation should follow the interval and the prespecified decision rule, not the point estimate alone.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

Two test formulations are compared with one reference. The only planned claim is that both are bioequivalent. Must a Bonferroni adjustment automatically be used for this conjunctive claim? Explain.
There are four comparisons, and the plan allows a bioequivalence claim for any formulation that passes. At family-wise \(\alpha=0.05\), what per-comparison TOST level and two-sided interval level does Bonferroni use?
A Bonferroni-adjusted ratio interval is \([0.82,1.24]\). Does it meet the usual limits? What if the upper endpoint is 1.27?
Why is it incorrect to describe a 90% interval as the Bonferroni interval when three TOST comparisons are each tested at \(0.05/3\)?

Solutions. (1) No. If the only claim requires all comparisons to pass, conducting every TOST at \(\alpha=0.05\) and requiring all to reject controls the global type I error at no more than 0.05. (2) Each TOST uses \(0.05/4=0.0125\); its corresponding two-sided interval has level \(1-2(0.0125)=0.975\), or 97.5%. (3) The interval \([0.82,1.24]\) lies wholly within \([0.80,1.25]\), so it meets the interval criterion. An upper endpoint of 1.27 exceeds the limit, so it does not. (4) With three comparisons, each TOST level is \(0.01667\), giving a corresponding two-sided interval level of \(1-2(0.01667)=0.9667\), or about 96.67%, not 90%.