Validating Model-Based Bioequivalence Decisions By Simulation

Why Validate a Model-Based Decision?

A model-based bioequivalence analysis turns observed concentration or exposure data into estimates, uncertainty intervals, and ultimately a decision. The model may be complex, but a central validation question is direct: when the true formulation effect is known, does repeatedly applying this model and decision rule behave as intended? Simulation addresses that question by generating many hypothetical datasets under specified conditions, fitting the proposed model to each one, and summarizing the results.

This tutorial focuses on three complementary checks: bias of the estimated formulation effect, coverage of its uncertainty interval, and the false-equivalence rate when the true effect is outside the prespecified equivalence limits. These are repeated-sampling properties of the full analysis procedure, not claims about any one observed study. A successful simulation provides evidence about performance under the data-generating assumptions used; it does not establish that those assumptions describe every real study.

Set the Truth and the Decision Rule First

For clarity, let \(\theta\) denote the true mean test-minus-reference effect on the log exposure scale. The usual log-scale equivalence limits are \(L=\log(0.80)\) and \(U=\log(1.25)\), approximately \(-0.2231\) and \(0.2231\). An effect is inside these limits when \(L<\theta<U\). Exponentiating \(\theta\) expresses the corresponding test-to-reference ratio.

A simulation needs an explicit decision rule. For the examples below, suppose the proposed analysis is Bayesian and declares bioequivalence when the posterior probability that \(\theta\) lies between \(L\) and \(U\) is at least 0.95. The rule is therefore \(\Pr(L<\theta<U\mid\text{data})\geq 0.95\). The rule, prior, model, and any data-processing steps should be fixed before the simulation results are inspected. Otherwise, selecting them in response to the results can make the validation misleading.

For each scenario, record the true \(\theta\), the model estimate \(\hat{\theta}_r\) from replicate \(r\), an interval for \(\theta\), and whether the decision rule declares bioequivalence. For \(M\) replicates, useful summaries are:

Here \(C_r\) is the interval from replicate \(r\), and \(I(\cdot)\) equals 1 if its statement is true and 0 otherwise. Coverage is assessed against the known true effect. The false-equivalence rate, denoted \(\widehat{\mathrm{FE}}\) here, is reported for scenarios in which \(\theta\) is outside the equivalence limits. When the true effect is inside the limits, the corresponding decision rate is not a false-equivalence rate.

A Simple Model for Checking Calculations

A scalar model can make the logic transparent before applying the same checks to a richer concentration model. Suppose the simulated subject-level log exposure contrasts \(D_i\) are independent and normally distributed, \(D_i\sim N(\theta,\sigma^2)\), with known \(\sigma\). Use a normal prior \(\theta\sim N(0,\tau^2)\). This deliberately simple setup is not a substitute for a full crossover or concentration-time model; it provides an explicit example of generating data, fitting a model, and checking its operating behavior.

For \(n\) contrasts with sample mean \(\bar{D}\), the posterior is normal. Its variance \(V\) and mean \(m\) are:

These expressions follow by multiplying the normal likelihood by the normal prior. The likelihood contributes a term proportional to \(\exp[-n(\theta-\bar{D})^2/(2\sigma^2)]\), and the prior contributes \(\exp[-\theta^2/(2\tau^2)]\). Collecting the coefficients of \(\theta^2\) gives posterior precision \(1/V=1/\tau^2+n/\sigma^2\); collecting the linear terms gives posterior mean \(m=Vn\bar{D}/\sigma^2\). Thus each simulated dataset can be analyzed with an exact posterior in this example.

The same simulation checks apply when the proposed model estimates several effects or uses random effects and a more complicated residual structure. In that setting, generate the data from specified parameter values, fit the complete proposed model to each replicate, and retain the estimate, interval, and decision. The model and simulation assumptions must be stated precisely enough that the data-generating procedure is reproducible.

Bias, Coverage, and Monte Carlo Uncertainty

Bias measures systematic displacement of the estimates from the known truth. If the average estimate across replicates is close to \(\theta\), the empirical bias is small. A small estimate of bias is not by itself proof of good performance: estimates can be centered correctly but too variable, and model intervals can still have inadequate coverage.

For intervals advertised as 95% intervals, the target repeated-sampling coverage is 0.95: over repeated datasets generated under a fixed scenario, approximately 95% of the intervals should contain the true parameter. When the intervals are Bayesian credible intervals, their posterior probability statement is conditional on the observed data and prior. Their empirical coverage across simulated datasets is a separate repeated-sampling property. One must not interpret a 95% credible interval as guaranteeing 95% simulation coverage.

Simulation summaries have Monte Carlo error because they are calculated from a finite number of replicates. If \(K\) of \(M\) replicates cover the truth, the estimated coverage is \(\hat{p}=K/M\), and its approximate Monte Carlo standard error is \(\sqrt{\hat{p}(1-\hat{p})/M}\). The same binomial calculation applies to a decision rate. For estimated bias, the Monte Carlo standard error is the sample standard deviation of the \(M\) replicate estimates divided by \(\sqrt{M}\). These errors describe simulation precision, not uncertainty in the true parameter of a real study.

False-Equivalence Rates Need Boundary Scenarios

A model can produce estimates with little bias and intervals with nominal coverage yet still have an undesirable decision rate near an equivalence boundary. Therefore, evaluate the decision rule at several known effects: one clearly inside the limits, effects near each boundary, and effects outside the limits. A false-equivalence rate is specifically the fraction of replicates declared equivalent when the simulated true effect is outside \([L,U]\). It is not the probability that a particular real study’s conclusion is wrong.

For a Bayesian decision rule, this rate is not automatically controlled at 5% merely because the rule requires posterior probability at least 0.95. The posterior threshold is a decision criterion, while the false-equivalence rate is a repeated-sampling property under a particular scenario. It depends on the sample size, variability, prior, model, true effect, and all parts of the analysis.

Practical Checks and Limits

Before interpreting results, check whether the simulation and repeated fits actually represent the intended analysis. Confirm that generated values follow the stated model, that parameter values are in the intended units and scale, and that the fitting procedure uses the same model, prior, interval construction, and decision threshold in every replicate. For complex fitted models, record convergence failures and boundary estimates. Silently discarding difficult replicates can make performance appear better than it is; report how many failures occurred and how they were handled.

Also distinguish a statistical failure from inadequate Monte Carlo precision. If an estimated rate is near its target but its Monte Carlo error is large, increase \(M\) or use a precision-based stopping rule set in advance. If coverage or false-equivalence rates vary sharply across scenarios, report the pattern rather than averaging away the problematic conditions. A single favorable scenario cannot validate a general claim about model performance.

Analytic calculations can provide checks when the model is simple enough to yield exact sampling results, while simulation can evaluate more complex fitted procedures where such calculations are impractical. The two approaches complement each other: agreement in a tractable case can catch implementation errors, whereas simulation provides empirical operating characteristics for the specified scenarios. Simulation remains conditional on its data-generating assumptions. Evaluating performance under deliberately different assumptions is a separate robustness question.