Bootstrap Uncertainty in Target Attainment

What Does Uncertainty in Target Attainment Mean?

A target-attainment analysis asks what fraction of a specified patient population is expected to achieve an exposure goal under a regimen. For example, the goal might be an area under the concentration–time curve above a threshold, or a trough concentration within a prespecified range. The answer depends on the pharmacokinetic model, its estimated parameters, the variability among patients, the covariate distribution of the target population, and the definition of the exposure target. A single simulated percentage can conceal the fact that the model parameters themselves are uncertain.

Bootstrap uncertainty analysis makes that parameter uncertainty visible. It repeatedly creates a bootstrap dataset using a resampling scheme appropriate to the study, refits the model, and simulates patients under the same regimen and target-population definition. Each refit produces a target-attainment estimate and an exposure distribution. Variation among these estimates describes how much the conclusions could change with the sampling variability represented by the bootstrap. It is distinct from the variation in exposure between patients generated under any one fitted model.

The target set, regimen, exposure metric, and target population must be held consistent across replicates if the goal is to quantify uncertainty in the same estimand. The target-population covariates may be fixed at a prespecified distribution or sampled according to a defined design, but that choice must be stated. If the scientific question concerns a different population or a different regimen, it is a different target-attainment estimand.

Constructing the Bootstrap Distribution

For bootstrap replicate \(b\), obtain a dataset according to the chosen bootstrap design and refit the model to produce parameter estimate \(\hat{\psi}_b\). Then simulate \(M\) patients under the regimen of interest. Each simulated patient should receive covariates and individual random effects according to the target population and fitted model. Include residual error only when the exposure metric and target definition require it: a target on a model-predicted individual exposure may not include observation error, whereas a target defined on a noisy measured concentration may.

Let \(G_{bm}\) be the simulated exposure metric for patient \(m\) in replicate \(b\), and let \(I(\cdot)\) equal 1 when its statement is true and 0 otherwise. The simulated attainment estimate is

The collection \(\hat{q}_1,\ldots,\hat{q}_B\) estimates the bootstrap sampling distribution of the target-attainment probability. Its mean or median summarizes the center, its standard deviation estimates bootstrap standard error, and its percentile quantiles can provide a bootstrap interval. The interval describes uncertainty in the population probability under the bootstrap procedure and model; it is not an interval containing a specified fraction of individual patients.

The same replicate-wise approach applies to exposure distributions. For example, calculate a population median exposure, a specified exposure quantile, or the fraction above an exposure cutoff separately within each refit. The distribution of that summary across bootstrap replicates quantifies its parameter uncertainty. Alternatively, compare estimated cumulative distribution functions \(F_b(x)\) across refits to assess uncertainty in the population exposure distribution at particular values \(x\). Pooling all simulated patients from all bootstrap refits into one list does not by itself provide a confidence interval for a population quantile: it mixes variation among patients with variation among fitted models.

Why the Two Sources of Variation Must Be Kept Distinct

Within a bootstrap replicate, simulated patient-to-patient variability determines how much exposure differs among patients conditional on \(\hat{\psi}_b\). Across replicates, the fitted parameters vary because the observed study sample could have been different. The target-attainment probability \(q(\psi)\) is a population quantity, not the binary outcome for one simulated patient. Thus, patient variability is already used to define and calculate \(q(\psi)\); it should not be added to the uncertainty in \(q\) as though a patient-level Bernoulli outcome were the estimand.

A short variance decomposition clarifies the distinction. Imagine drawing a bootstrap-supported parameter value \(\Psi\), then drawing one patient’s target indicator \(Z\) conditional on that value. Since \(Z\) is Bernoulli with conditional success probability \(q(\Psi)\), the law of total variance gives

The first term is variation in the population attainment probability across parameter values. The second is the variability of an individual patient’s attainment indicator conditional on the parameter value. If the estimand is the population probability, bootstrap uncertainty concerns the first term. The second term is relevant to individual outcomes, not an extra component to add to a confidence interval for the population probability.

Finite simulation adds a further, computational source of noise. Conditional on replicate \(b\), if the \(M\) simulated patients are independent and each has the same attainment probability \(q_b\), then

This Monte Carlo error can be made small by increasing \(M\), but it is not eliminated by increasing the number of bootstrap refits \(B\). A practical analysis uses enough simulated patients that Monte Carlo error is small relative to the between-refit spread, or reports and accounts for both sources. If simulated patients are not independent—for example, because simulation preserves clusters—the simple binomial formula does not apply without adjustment.

Worked Exposure-Distribution Summaries

A target-attainment percentage is one feature of the exposure distribution. A decision may also depend on a median, a tail probability, or a quantile. Calculate the chosen summary for each refit rather than relying only on the full-data simulated distribution. For quantiles, use the same quantile definition and simulation or numerical calculation in every replicate. The resulting bootstrap distribution describes uncertainty in that population summary.

Target-Population Composition and Bootstrap Replicates

The target population’s composition can materially affect attainment. If covariates are fixed by design or specified from an external population, use the same covariate weights in every replicate to isolate model-parameter uncertainty. If uncertainty in the population composition is also part of the intended inference, it must be represented by an appropriate sampling procedure; otherwise the interval does not include that source of uncertainty.

Assumptions, Diagnostics, and Alternatives

A bootstrap interval is only as relevant as the resampling and simulation procedure used to construct it. The analysis assumes the resampling unit represents the independent sampling unit or that dependence is otherwise respected; the model is refit comparably across replicates; the fitted model adequately represents the exposure-generating process for the target population; and the target, regimen, and covariate weighting are defined consistently. A nonparametric bootstrap represents sampling variability by resampling observed units, while a parametric bootstrap generates bootstrap data from a fitted model. The choice should reflect the study design and the inferential goal.

Useful diagnostics include examining the distribution of replicate attainment estimates, checking convergence and parameter estimates, confirming that simulation error is small, and verifying that every replicate uses the same target definition and population weights. Compare results with a larger \(M\) for selected replicates: substantial changes indicate that patient-simulation noise is masking the bootstrap pattern. Check whether replicate exposure distributions are plausible and whether the target threshold lies in a region with enough simulated patients to estimate the tail reliably.

A fixed-parameter simulation is simpler and useful for describing patient variability conditional on one fitted model. It does not, by itself, quantify uncertainty from estimating that model. An analytic approximation such as a delta-method calculation may be computationally efficient when the target probability is a smooth function of well-behaved parameter estimates, but can be less convenient for nonlinear simulation-based targets or complicated mixture populations. Bootstrap simulation is flexible, but it inherits the model’s limitations, can be computationally intensive, and does not automatically capture uncertainty in a target population or model structure that was held fixed.

A common mistake is to pool all patient simulations across all bootstrap refits and treat the resulting spread as a confidence interval for target attainment. This mixes differences among patients under a model with changes in the estimated population probability across models. The correction is to calculate one population summary per replicate, then summarize the replicate-level estimates. A related mistake is to report the simulated patient-level fraction as though it were the certainty of the population rate: a large number of simulated patients reduces Monte Carlo error, but it cannot replace uncertainty from limited study data.