Comparing Candidate Schedules by Parameter Precision

Why Compare Schedules by Simulation?

A sampling schedule is useful only in relation to the parameters the study needs to estimate. Moving a sample earlier may help characterize absorption, while adding participants may improve information about population parameters. But the size of either benefit depends on the model, the variability between participants, residual error, and the rest of the design. Simulation lets us compare proposed schedules under a stated set of assumptions by repeatedly creating and analyzing hypothetical studies.

The comparison is about precision: how much an estimate would vary across repeated studies conducted under the same design and model assumptions. Precision is not the same as accuracy. Estimates can vary little but be systematically displaced from the true value. A useful comparison therefore reports both variability and bias, rather than treating a small standard deviation as sufficient evidence that a schedule performs well.

Set Up a Fair Comparison

Let \(\theta_j\) denote a population parameter of interest, such as typical clearance, and let \(\hat{\theta}_{jr}\) be its estimate in simulated data set \(r\), for \(r=1,\ldots,R\). The simulation begins with a fully specified model: the structural pharmacokinetic model, the population parameter values, the distribution of between-participant differences, the residual-error model, and any relevant covariates. It must also specify the dose regimen, participant count, and exact sample times for each candidate schedule.

The study model used to fit the simulated data should represent the analysis that is actually planned. For a focused comparison, keep the generating model, fitting model, participant count, dosing conditions, and estimation procedure the same across schedules unless one of these is the feature being varied. Otherwise, a difference in precision cannot be attributed clearly to the schedule. The fitted model can still be misspecified; simulation results describe performance under the assumptions used to generate and analyze the data, not under every possible real-world model.

For each schedule, generate independent hypothetical studies and fit the model to each one. Record convergence and other fitting outcomes. Do not silently discard problematic runs: report how often they occur and consider what happens if failed runs are counted as design failures. A schedule that gives precise estimates only after excluding many difficult fits may not be a dependable choice.

For a parameter with units, such as clearance in L/h, \(SD_j\) has the same units. Relative empirical SD and relative bias are percentages and therefore have no units. When the parameter is close to zero, a relative summary can become unstable; report the absolute-scale result instead. For a positive parameter, also check whether estimates near zero or extreme values make the distribution strongly skewed, because a mean and standard deviation alone may not describe it well.

Worked Comparisons

What the Comparison Can and Cannot Establish

A complete design comparison names the target parameters and reports their summaries by schedule. It is often helpful to include the empirical SD in natural units, relative empirical SD, bias, and RMSE. If the scientific goal is to estimate several parameters, a schedule may improve precision for one while leaving another unchanged or making it worse. There is no single “best” schedule until the priorities are stated.

A model-based standard error from one fitted data set is an alternative summary. It describes estimated uncertainty conditional on that data set and the fitted model. Repeating the simulation instead measures how estimates vary across generated data sets and can reveal bias, skewness, and fitting failures. A local information-matrix approximation can be faster for screening candidate designs, but it relies on local approximation and model assumptions; simulation is generally more direct about the behavior of the planned estimator under the specified scenario. Neither approach protects against an incorrect generating model.

Simulation results themselves have Monte Carlo uncertainty because they use a finite number of repetitions. For example, if estimates are approximately normally distributed, the relative Monte Carlo standard error of an empirical SD is roughly \(1/\sqrt{2(R-1)}\). With \(R=500\), this is approximately \(1/\sqrt{998}=0.0317\), or 3.17%. Small differences between schedules may therefore reflect simulation noise. Increase \(R\), especially when the comparison is close or parameter estimates are unstable. Using matched random-number streams can also reduce noise in pairwise comparisons when the same simulated participant features can sensibly be shared across schedules.

Diagnostics should include more than the final precision table. Check convergence rates, implausible estimates, boundary estimates, and whether failures cluster under one schedule. Inspect the distribution of estimates for outliers or skewness. Confirm that apparent gains persist when the simulation is repeated with more replicates and, where appropriate, under plausible alternative model assumptions. A design comparison is fragile if its ranking depends on a few extreme runs or on excluding difficult fits.

A common mistake is to select the schedule with the smallest empirical SD and call it the most accurate. For example, a schedule could produce tightly clustered estimates that are all systematically too low. The correction is to examine bias and RMSE alongside precision, and to state explicitly that the conclusion is conditional on the simulation model. Another mistake is to compare schedules that differ in both sample timing and participant count, then attribute the result solely to timing. Keep other design features fixed, or use a structured set of scenarios that separates their effects.