Interpreting Bootstrap and Shrinkage Evidence in Context

What These Summaries Can—and Cannot—Tell You

A population pharmacometric model produces more than parameter estimates. A report may also include a bootstrap distribution for a population parameter and empirical Bayes estimates (EBEs) of individual random effects. These summaries address different questions. A bootstrap describes how an estimate varies when the analysis is repeated on resampled data under a specified model and sampling scheme. Shrinkage describes how much individual estimates are drawn toward the population distribution when the individual data provide limited information.

Neither summary validates the model or removes uncertainty. A narrow bootstrap distribution does not establish that the structural or error model is correct. A shrunken individual estimate is not a direct measurement of that individual's true parameter. The useful reporting question is therefore not simply “Is the interval narrow?” or “Is shrinkage high?” It is: what evidence does this result provide, under which assumptions, and what conclusion would go beyond that evidence?

Interpreting a Bootstrap Distribution

For a population-level estimate, a nonparametric bootstrap often resamples the independent sampling units with replacement. In a typical population PK analysis, those units are subjects: a resampled subject brings along that subject's observations and relevant covariates. The model is refitted to each resampled data set, yielding estimates \(\hat{\theta}^{*(1)},\ldots,\hat{\theta}^{*(B)}\) for a parameter \(\theta\). The empirical distribution of these \(B\) estimates is the bootstrap distribution.

This procedure has assumptions. The resampling unit must reflect how the data were sampled and which observations are independent. The model and estimation procedure must be suitable for the question, and refits must succeed often enough to characterize the distribution. The bootstrap uses the observed data as an approximation to the population sampling process; it cannot repair a biased sample, a misspecified model, or a parameter that is poorly identified. If sampling is clustered or stratified, the resampling scheme must preserve that design rather than treating all records as exchangeable.

The bootstrap standard error is the sample standard deviation of the replicate estimates. If \(\bar{\theta}^{*}\) is their mean, then

The denominator is \(B-1\) because the replicate deviations are measured from their estimated mean. A percentile interval uses selected quantiles of the bootstrap estimates; for example, the 2.5th and 97.5th percentiles form a 95% percentile interval. Under suitable regularity and sampling conditions, this interval is intended to have approximately 95% repeated-sampling coverage. It does not mean there is a 95% probability that the fixed parameter lies in the particular computed interval.

A bootstrap distribution can also reveal features hidden by one standard error. Strong skewness, multiple modes, extreme estimates, or estimates piling up at parameter bounds may indicate weak identification, unstable optimization, or a parameterization problem. These patterns should be investigated and described, not compressed into a single symmetric interval without comment.

Why Individual Estimates Shrink

An EBE combines individual data with information from the fitted population distribution. A simple normal model makes the mechanism explicit. Suppose an individual's random effect is \(\eta\sim N(0,\tau^2)\), and the individual data provide a noisy summary \(y\) satisfying \(y\mid\eta\sim N(\eta,s^2)\). Here \(\tau\) is the population standard deviation of the random effect, while \(s\) represents the uncertainty in the individual's data-based summary. These are assumptions for the illustrative model, not a claim that every nonlinear pharmacometric model has an exactly normal conditional distribution.

Because the prior density for \(\eta\) and the likelihood for \(y\) are both normal, their product is a normal posterior density. Collecting the terms involving \(\eta\) in the exponent gives posterior precision \(1/\tau^2+1/s^2\), and posterior mean equal to that precision's inverse times \(y/s^2\). Thus,

The multiplier \(\tau^2/(\tau^2+s^2)\) lies between zero and one. When individual information is weak (\(s^2\) large), the multiplier is small and the estimate is pulled toward zero, the population mean. When individual information is strong (\(s^2\) small), the multiplier approaches one and the estimate is closer to the individual's data-based value. In fitted population models, the population distribution parameters are estimated from the data, and EBEs are typically conditional estimates such as conditional modes; the simple formula illustrates the information-sharing principle rather than reproducing every estimation algorithm.

Reporting a Shrinkage Diagnostic

One common diagnostic for interindividual random effects is the standard-deviation-based eta-shrinkage measure. If \(\omega\) is the population standard deviation of an eta and \(SD(\hat{\eta})\) is the across-subject standard deviation of the corresponding EBEs, the measure is

The formula is convention-dependent: reports should state the exact calculation, the random effect, and the subjects included. It summarizes the spread of the EBEs relative to the fitted population spread. It is not an estimate of the fraction of each individual's true effect that is known, nor is it a direct measure of parameter bias. When shrinkage is high, EBEs may show too little between-subject variation to support detailed interpretation or some individual-level diagnostic plots.

Diagnostics, Alternatives, and Reporting Limits

Before interpreting either type of evidence, check how it was produced. For a bootstrap, report the resampling unit and scheme, number of attempted and successful refits, interval method, and any relevant failures. Examine the replicate distribution for skewness, boundary accumulation, outliers, or multiple modes. A large failure rate is itself important evidence: reporting only the successful estimates can select a nonrepresentative subset and make uncertainty appear smaller or more regular than it is. Do not silently discard failures; investigate their causes and state how they affect interpretation.

For shrinkage, state whether it refers to eta or another quantity, give the formula and population scale used, and note the population and data included. Examine the amount and timing of individual observations, the balance of sampling across subjects, and whether estimates are concentrated near the population mean. Sparse or uninformative individual data can produce high shrinkage even when population parameters are estimated adequately. Conversely, low shrinkage does not prove that the model is unbiased or that individual estimates are precise in every relevant respect.

The main alternatives answer related but different questions. An asymptotic covariance-based standard error uses local curvature of the estimation objective around the fitted estimate; it is often computationally efficient but may be unreliable with weak identification or irregular parameter boundaries. Profile likelihood evaluates the objective over fixed values of a parameter and can show asymmetry, though it requires additional fits and remains conditional on the model. These do not make bootstrap results interchangeable with one another. Likewise, an EBE is a conditional point estimate, not a measure of its own precision; conditional uncertainty summaries address uncertainty about an individual effect more directly.

A common mistake is to write, “The bootstrap 95% interval gives a 95% probability that the true clearance is between these limits, and low shrinkage confirms accurate individual predictions.” The first clause assigns a probability to a fixed parameter after the interval has been computed; the second treats one spread diagnostic as proof of individual accuracy. Instead, report the interval as a repeated-sampling uncertainty summary under the specified procedure, and report shrinkage as evidence about the relative spread of conditional estimates. Neither statement alone establishes model adequacy, individual truth, or causal interpretation.