Why Variance Specification Matters
In reference-scaled bioequivalence, the estimated within-subject variability of the reference product helps determine the scaled criterion. A variance model therefore affects more than a standard error: it can change the estimated scale itself. Two defensible models fitted to the same replicate data may produce different reference-variance estimates, different uncertainty, and different conclusions under the scaled rule.
The purpose of a sensitivity analysis is not to find the model that gives the preferred outcome. It is to ask whether reasonable changes to the variance assumptions materially change the estimated reference variability or the resulting inference. The reference variance is a within-subject variance on the log scale; it is not the total variance among all observed concentrations or exposures. The earlier material on estimating reference variability in replicate designs distinguishes the designs that identify this component.
All quantities in this expression are on the log scale. In particular, \(\sigma_{wR}\) is a standard deviation, while \(\sigma_{wR}^2\) is a variance. Confusing the two changes the scale term and can reverse an interpretation. Also, the point-estimate constraint is separate: a favorable scaled criterion does not remove the requirement that the estimated test/reference geometric mean ratio meet the applicable constraint.
What Changes Across Variance Models?
A mixed model for replicate crossover observations commonly includes fixed effects for treatment, period, and sequence, together with subject effects appropriate to the design. The variance specification determines how residual variability is represented. Under a common-residual model, the residual variance is assumed equal across treatments. Under a treatment-specific model, residual variances can differ by treatment, allowing the reference variance to be estimated without imposing equality with test-treatment variability.
These specifications encode different assumptions. A common residual variance can be more stable when the data support homogeneity, but it may pool test and reference variation in a way that does not represent reference variability well. A treatment-specific model is more flexible, but estimating separate variances can be imprecise in a small study. Other reasonable specifications may address covariance or design features, such as period-related heterogeneity, when justified by the design and prespecified analysis. Adding complexity does not automatically improve the model: each extra component needs to be identifiable and scientifically interpretable.
A sensitivity comparison should keep the data, endpoint, fixed-effect definition, scaled rule, and analysis population consistent. Change the variance specification deliberately, then examine the resulting \(\hat{\sigma}_{wR}^2\), its uncertainty and degrees of freedom, the treatment contrast, and the decision statistic. If the fixed-effect estimate changes as well, report that change rather than attributing the whole difference to the reference variance.
Worked Example: Comparing Two Plug-In Criteria
Suppose two plausible models fitted to a hypothetical replicate study give the same estimated log treatment difference, \(\hat{\delta}=0.090\). A common-residual fit gives \(\hat{\sigma}_{wR}^2=0.0155\), while a treatment-specific fit gives \(\hat{\sigma}_{wR}^2=0.0120\). Compare their plug-in criteria using \(\theta=0.7966\).
For the common-residual fit:
For the treatment-specific fit:
Both plug-in criteria are negative, but they differ by about \(0.002788\) on the squared-log scale. The difference arises here from the reference-variance estimates, because the fitted treatment contrast was held fixed. These calculations do not establish bioequivalence: they omit uncertainty in both estimates and do not calculate the required upper confidence bound.
Justifying a Model-Based Sensitivity Calculation
The criterion \(Q\) depends jointly on the treatment contrast and reference variance. A first-order approximation explains why uncertainty can change differently from the plug-in value. Let \(\hat{\delta}\) and \(\hat v\) estimate \(\delta\) and \(v=\sigma_{wR}^2\), and suppose their joint sampling distribution is approximately normal with estimated variances \(s_\delta^2\), \(s_v^2\), and covariance \(s_{\delta v}\). For \(g(\delta,v)=\delta^2-\theta v\), the gradient is \((2\delta,-\theta)\). Applying the first-order variance formula gives:
The final term matters when the fitted treatment contrast and variance estimate are correlated. Omitting it is justified only if the covariance is negligible or the approximation explicitly assumes zero covariance. Even when the approximation is useful for comparing model sensitivity, it is not a replacement for the regulator-prescribed upper-bound method, which accounts for the relevant estimation procedure and degrees of freedom.
Worked Example: How Uncertainty Can Change the Comparison
Continue with \(\hat{\delta}=0.090\). For illustration, suppose the two hypothetical model fits have \(s_\delta=0.018\), zero estimated covariance \(s_{\delta v}=0\), and standard errors for the variance estimates of \(0.0030\) and \(0.0040\), respectively. Calculate a one-sided 95% Wald upper bound for \(\hat Q\) using \(\hat Q+1.645\,SE_{\hat Q}\). This is an asymptotic sensitivity illustration, not the FDA decision procedure.
For the common-residual fit, \(s_v=0.0030\), so:
Using \(\hat Q_C=-0.0042473\) from the previous example:
For the treatment-specific fit, \(s_v=0.0040\):
In this illustrative calculation, both upper bounds are positive. The treatment-specific model has a less negative plug-in criterion and a larger estimated uncertainty, so its approximate upper bound is also larger. This demonstrates why comparing only point estimates of reference variance, or only the sign of \(\hat Q\), is inadequate. The Wald bounds here should not be reported as regulatory bounds.
Reading Sensitivity and Diagnosing Problems
Variance-model sensitivity is most consequential when reasonable specifications lead to materially different estimates or different conclusions under the prescribed decision rule. A larger reference variance makes the term subtracted in \(Q\) larger, which can make the plug-in criterion more favorable. But that does not guarantee a more favorable upper confidence bound: a less stable estimate, altered degrees of freedom, or covariance with the treatment contrast can increase uncertainty.
Before interpreting a model comparison, check whether each specification is supported by the design and data. Examine residual diagnostics by treatment and period, convergence messages, variance estimates near a boundary, and whether the estimated component is identifiable. A model that converges can still fit poorly or produce a variance estimate that is highly sensitive to one subject. The earlier material on residual diagnostics and influential observations provides useful checks; here the question is whether those issues make the variance specification comparison unreliable.
A particularly useful check in a full replicate design is whether the observed within-subject reference contrasts are broadly compatible with the model’s estimated reference variability. Large discrepancies across subjects, or a pattern tied to period, may suggest that a single common reference variance is an inadequate summary. In a partial replicate design, available repeated reference observations still identify reference variability, but the information and precision differ from a full replicate design. The design must support the component being estimated.
Worked Example: A Direct Check Using Reference Replicates
In a hypothetical full replicate study, six subjects each have two reference observations on the log scale. Their within-subject differences \(d_i=Y_{i,R1}-Y_{i,R2}\) are \(-0.20,\ 0.10,\ 0.30,\ -0.10,\ 0.20,\ -0.30\). Under independent, normally distributed, equal-variance reference residuals, each difference has variance \(2\sigma_{wR}^2\). Use the sample variance of the differences as a simple diagnostic estimate.
The differences sum to zero, so their sample mean is zero. Their squared deviations sum to:
Thus the sample variance is \(s_d^2=0.28/(6-1)=0.0560\). Since \(\operatorname{Var}(d_i)=2\sigma_{wR}^2\), the diagnostic estimate is:
Suppose a common-residual mixed model instead gives \(\hat{\sigma}_{wR}^2=0.0160\), and both models estimate \(\hat{\delta}=0.100\). Their plug-in criteria are:
The difference is substantial, but six subjects provide little precision, and the simple difference-based calculation is a diagnostic rather than a substitute for the design-specific mixed-model analysis. The finding calls for checking the model assumptions and uncertainty, not choosing the larger variance because it produces a more favorable criterion.
Comparing Approaches and Avoiding Overinterpretation
| Approach | Strength | Main risk or limitation |
|---|---|---|
| Common residual variance | Uses a parsimonious variance structure and can be more stable. | May impose equality across treatments when reference variability differs. |
| Treatment-specific residual variances | Allows the reference variance to be estimated separately. | Can be imprecise or unstable in small samples. |
| Alternative design-justified covariance structure | Can represent heterogeneity or dependence not captured by simpler models. | May be weakly identified, overly complex, or sensitive to implementation. |
The regulatory analysis should follow the prespecified, applicable procedure. Sensitivity fits are supporting analyses: they show how assumptions affect the result, but they do not license switching models after seeing which one passes. A model that is not supported by the design, diagnostics, or analysis plan is not made acceptable by a favorable conclusion. Conversely, a meaningful sensitivity finding should not be hidden simply because the primary model was prespecified.
Sensitivity has limits. With few subjects, variance estimates can be noisy even when the model is correctly specified. Different variance models may also change degrees of freedom and covariance estimates, making a single-number comparison incomplete. The objective is not to eliminate all model dependence; it is to establish whether the conclusion is robust to plausible, scientifically defensible specifications and to make important dependence transparent.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Why is a reference within-subject variance not interchangeable with the total variance of all observed log exposures?
- If two models have the same \(\hat{\delta}\), but one estimates a larger \(\hat{\sigma}_{wR}^2\), what happens to the plug-in \(Q\)? Does that alone establish bioequivalence?
- A model comparison changes both \(\hat{\delta}\) and \(\hat{\sigma}_{wR}^2\). What should be reported to interpret the sensitivity correctly?
- Why should the model with the more favorable scaled result not be selected after inspecting the results?
Solutions. (1) Total variance includes between-subject variation and other sources; the scaling criterion uses within-subject reference variability on the log scale. (2) Since \(\theta>0\), increasing \(\hat{\sigma}_{wR}^2\) decreases \(\hat Q=\hat{\delta}^2-\theta\hat{\sigma}_{wR}^2\). It does not establish bioequivalence because uncertainty, the required upper confidence bound, and the point-estimate constraint remain. (3) Report both estimates and their uncertainty, identify the changed variance assumptions, and compare the complete decision statistic under the applicable procedure. (4) Outcome-based model selection can misrepresent uncertainty and violates the principle that the analysis specification should be justified independently of which result it produces.