Instability of Reference Variance Estimates in Small Samples

Why Few Reference Replicates Matter

Reference-scaled bioequivalence uses within-subject reference variability to determine how much the acceptance criterion may widen, or to evaluate a variance-dependent criterion. That makes the reference variance more than a descriptive statistic: estimation uncertainty can affect the decision itself. When few repeated reference observations are available, the estimated variability may change substantially if one subject’s replicate difference changes. The resulting uncertainty is especially important when the estimate lies near a scaling threshold or when the FDA upper confidence bound is close to zero.

This tutorial focuses on that instability, not on diagnosing an individual influential observation or selecting among variance models. The key distinction is between the true within-subject reference standard deviation, \(\sigma_{WR}\), and its estimate, \(s_{WR}\). The first is a population quantity; the second is a random quantity that can vary from sample to sample.

The normal-contrast model is a useful basis for understanding sampling instability. In an actual analysis, the contrasts or variance component are obtained from the study design and fitted model; the degrees of freedom and standard error should reflect that analysis. The simple model assumes independent contrasts, a common within-subject variance, normally distributed log-scale contrasts, and no unaccounted structure that changes their variance. It is not a substitute for the design-specific estimation described in the earlier material on estimating reference variability and degrees of freedom.

Sampling Uncertainty in a Variance Estimate

For independent normal observations, the scaled sample variance follows a chi-square distribution. Since \(s_{WR}^2=s_D^2/2\), the same result applies to the reference variance:

This relation gives a direct measure of how much the estimate can fluctuate. Its relative standard error is approximately \(\sqrt{2/(m-1)}\), so it decreases only as the number of available contrasts grows. With four degrees of freedom it is about 70.7%; with nineteen it is about 32.4%. These are not confidence levels, but relative standard deviations of the variance estimate under the stated model.

An exact \(100(1-\alpha)\%\) confidence interval for \(\sigma_{WR}^2\) is obtained by inverting the chi-square probability statement. If \(\chi^2_{q,\nu}\) denotes the \(q\)-quantile of a chi-square distribution with \(\nu=m-1\) degrees of freedom, then:

Taking square roots gives the interval for \(\sigma_{WR}\). Small \(\nu\) makes the chi-square distribution asymmetric and broad, so the interval can be much wider above the estimate than below it. A narrow-looking point estimate alone therefore does not establish that the underlying reference variability is known precisely.

Consequences for Expanded Limits

In the EMA reference-scaled approach for \(C_{\max}\), the expanded limits depend on the estimated reference standard deviation once the scaling threshold is exceeded. The untruncated limits have the form \(\exp(\pm k s_{WR})\), where the applicable constant is \(k=0.760\); the maximum permitted expansion caps the expanded acceptance limits at approximately \(0.6984\) and \(1.4319\). The separate \(0.80\)–\(1.25\) point-estimate constraint also applies. These rules mean uncertainty in \(s_{WR}\) can affect whether limits are expanded and how wide the calculated limits are.

A useful way to communicate this result is to report the estimate and its uncertainty, not only the expanded limits calculated from the estimate. The interval is a diagnostic for how weakly the data determine variability; it is not an alternative set of regulatory acceptance limits. Nor does crossing the threshold within an uncertainty interval itself establish that expansion is permitted under the applicable procedure.

Consequences for the FDA Upper Confidence Bound

The FDA linearized reference-scaled criterion can be expressed in terms of the population quantity \(Q=(\mu_T-\mu_R)^2-\theta\sigma_{WR}^2\), with \(\theta=[\ln(1.25)/0.25]^2\). The population-side condition is \(Q\leq0\). The FDA decision is based not merely on substituting estimates into \(Q\), but on whether the prescribed upper confidence bound for the criterion is nonpositive. The bound accounts for uncertainty in the estimated treatment difference and reference variability under the specified analysis.

Because the variance contribution is subtracted, a lower plausible value of \(\sigma_{WR}^2\) makes \(Q\) less negative, or more positive. Sparse replicate information can therefore weaken the evidence for satisfying the criterion: the estimate may suggest a favorable negative value, while uncertainty makes the upper confidence bound remain above zero. This is why a plug-in value of \(Q\) is not interchangeable with the FDA upper confidence bound.

Diagnostics, Comparisons, and Limits

The following checks help determine whether a variance estimate is too fragile to interpret without qualification:

• Inspect the information behind the estimate. Report the number of replicate contrasts and the relevant degrees of freedom, rather than presenting \(s_{WR}\) alone.
• Quantify uncertainty. A confidence interval or standard error can reveal that a point estimate near a scaling boundary is imprecise.
• Check the model assumptions. Strong departures from normality, unequal variances across relevant conditions, dependence, or a poorly supported variance-component model can make the simple chi-square calculation inappropriate. Use the fitted model and its design-specific diagnostics.
• Look for sensitivity to individual data contributions. Large changes after removing or perturbing a contrast may signal fragility. Such investigation should be planned and reported transparently, not used to discard inconvenient observations.

A full replicate design may provide more repeated reference information than a partial replicate design, but design labels alone do not determine precision: the amount of usable data, missing periods, and fitted-model degrees of freedom matter. A model-based REML estimate can use the design structure, yet it does not make a small information base precise. Likewise, a confidence interval is useful for describing uncertainty but does not replace the regulatory decision procedure.

The main alternative to emphasizing a single estimate is to present uncertainty alongside the regulatory calculation. For EMA limits, this means distinguishing the limit computed from \(s_{WR}\) from uncertainty about \(\sigma_{WR}\). For the FDA approach, it means distinguishing the plug-in linearized criterion from the prescribed upper confidence bound. Neither approach licenses substituting a confidence limit for the required estimate or changing the decision rule after seeing the result.