Parameter Uncertainty From a Nonparametric Bootstrap

Why Use Bootstrap Parameter Draws?

A simulation at the fitted population parameter estimates treats those estimates as fixed. But estimates vary across samples, and that uncertainty can matter for predicted concentrations, exposure measures, or other model-derived quantities. A nonparametric bootstrap offers a way to represent this uncertainty without assuming that the parameter estimates follow a multivariate normal distribution.

The central idea is to resample the original sampling units, refit the same NLME model to each resampled data set, and retain the resulting parameter estimates. The collection of successful refits forms an empirical distribution. In simulation, draw a complete parameter vector from that collection and use it in the model. Drawing complete vectors matters: it preserves the empirical shape of each parameter’s distribution and the observed dependence among parameters.

This distribution approximates the sampling variability of the estimator under the observed data and the chosen resampling scheme. It is not a posterior distribution, and it does not automatically include residual error or between-subject variability in a simulated outcome. Those sources of variation are handled separately.

Definition: Let $\hat{\boldsymbol{\theta}}^{*(b)}$ be the vector of fitted NLME parameters from the $b$th successful bootstrap refit, for $b=1,\ldots,B$. The empirical bootstrap distribution assigns probability $1/B$ to each retained vector: $$ \widehat{G}_{B}=\frac{1}{B}\sum_{b=1}^{B}\delta_{\hat{\boldsymbol{\theta}}^{*(b)}}, $$ where $\delta_{\hat{\boldsymbol{\theta}}^{*(b)}}$ places all its mass at that parameter vector. A draw from $\widehat{G}_{B}$ is made by selecting one retained vector uniformly at random.

Constructing the Empirical Distribution

A bootstrap refit is meaningful only in relation to a stated sampling design. For independent subjects with repeated measurements, a usual nonparametric scheme resamples subjects and carries each selected subject’s full record into the bootstrap data set. Resampling individual concentration records as if they were independent would break the within-subject structure. If the design has higher-level sampling units, such as sites that contribute groups of subjects, the resampling scheme must respect that structure.

For each resampled data set, fit the same model specification and retain all parameters needed for the planned simulation: fixed effects, variance components, random-effect correlations, and any other model quantities that define the simulation. Keep each set of estimates together as one vector. If a parameter is fitted on a transformed scale, apply a consistent scale convention when storing and using it; a parameter must satisfy the model’s required constraints when used for simulation.

A refit that fails to converge or produces an invalid parameter vector is not a successful draw. Excluding failures from the empirical distribution is necessary if the estimates cannot be used, but it is not innocuous: if failure is associated with particular parts of the bootstrap sampling distribution, the retained estimates may be systematically unrepresentative. Record the number and causes of failures, investigate them, and do not describe a distribution based on a small, selectively retained subset as if it represented all bootstrap refits.

Choose the resampling unit.
Use the independent sampling unit implied by the study design, keeping linked observations together.

Resample and refit.
Create bootstrap data sets, refit the specified NLME model, and record convergence and parameter validity.

Retain complete vectors.
Store the full parameter vector from each valid refit, with a consistent scale and parameter ordering.

Draw and simulate.
Select a retained vector, then calculate the desired model output using that vector. Add other simulation components only if they are part of the question.

What the Empirical Distribution Preserves

The empirical distribution does not impose a bell-shaped marginal distribution. If bootstrap estimates of clearance have a long upper tail, that tail remains in the set of possible draws. Nor does the method require that parameter pairs be independent. Selecting a row of the bootstrap parameter table samples the observed joint distribution; selecting each parameter separately instead constructs a different distribution.

For any scalar output $g(\boldsymbol{\theta})$, the bootstrap-based simulation can evaluate $g$ at each retained vector. For example, the empirical mean and variance of those outputs are

\overline{g}^{*}=\frac{1}{B}\sum_{b=1}^{B}g\!\left(\hat{\boldsymbol{\theta}}^{*(b)}\right), \qquad s_{g}^{2}=\frac{1}{B-1}\sum_{b=1}^{B}\left[g\!\left(\hat{\boldsymbol{\theta}}^{*(b)}\right)-\overline{g}^{*}\right]^2.

These calculations summarize variation across bootstrap parameter draws in the chosen output. They do not, by themselves, add random effects or residual error to individual simulated observations. In particular, an output distribution from parameter draws alone answers a different question from a predictive distribution that also includes variability among subjects or observations.

Worked Example: Seeing a Skewed Marginal Distribution

Hypothetical data. Suppose the fitted population clearance estimate is $5.0$ L/h. Ten successful bootstrap refits give clearance estimates, in L/h, of $4.0,\ 4.1,\ 4.2,\ 4.4,\ 4.5,\ 4.7,\ 5.0,\ 5.4,\ 6.2,\ 9.0$. The small number of refits is for hand calculation only; it would not usually be enough to characterize tail behavior reliably.

The empirical distribution puts probability $1/10=0.10$ on each value. Its mean is

\overline{CL}^{*}=\frac{4.0+4.1+4.2+4.4+4.5+4.7+5.0+5.4+6.2+9.0}{10} =\frac{51.5}{10}=5.15\text{ L/h}.

The value $9.0$ creates a visibly longer upper tail than lower tail. For a concrete percentile calculation, use the linear-interpolation convention $h=1+(B-1)p$, where $p$ is the desired percentile as a proportion, and interpolate between ordered values at the positions surrounding $h$. For $p=0.025$, $h=1+9(0.025)=1.225$, so the percentile is $4.0+0.225(4.1-4.0)=4.0225$ L/h. For $p=0.975$, $h=1+9(0.975)=9.775$, giving $6.2+0.775(9.0-6.2)=8.370$ L/h.

These interpolated percentiles describe the empirical spread in this illustrative set, not a precise estimate of a population tail. In a simulation that draws a clearance value from these ten estimates, the high value remains possible rather than being removed to make the distribution symmetric.

Why Draw Whole Parameter Vectors?

Let $\hat{\boldsymbol{\theta}}^{*(b)}$ include, for example, clearance and volume estimates from one refit. If a particular resampled data set produces higher clearance estimates together with lower volume estimates, that pairing is part of the bootstrap information. Drawing clearance from one row and volume independently from another can create combinations that were uncommon or absent in the refits.

A covariance matrix can summarize linear dependence, but it does not fully describe a joint distribution. Replacing bootstrap vectors by draws from a multivariate normal distribution with the same estimated mean and covariance would impose a different distributional shape and may allow invalid parameter values. Joint empirical resampling avoids those particular assumptions: it draws only parameter combinations actually present among the retained refits. It cannot, however, represent combinations that the finite collection failed to capture.

Worked Example: Preserving an Observed Parameter Pairing

Hypothetical data. Consider four successful refits of clearance $CL$ in L/h and volume $V$ in L. The retained pairs are $(4,20), (5,18), (6,16), (7,14)$. This deliberately small set makes the dependence easy to see; it is not a realistic claim that fitted parameters are perfectly correlated.

The means are $\overline{CL}^{*}=(4+5+6+7)/4=5.5$ L/h and $\overline{V}^{*}=(20+18+16+14)/4=17$ L. The deviations from those means are $(-1.5,3), (-0.5,1), (0.5,-1), (1.5,-3)$. Their cross-products are $-4.5,-0.5,-0.5,-4.5$, so the sample covariance is

s_{CL,V}=\frac{-4.5-0.5-0.5-4.5}{4-1}=-3.333.

The sample variances are $s_{CL}^{2}=5/3$ and $s_{V}^{2}=20/3$, so the sample correlation is $-3.333/\sqrt{(5/3)(20/3)}=-1.000$. The perfect negative correlation reflects this constructed example. Joint resampling selects one of the four observed pairs with probability $1/4$; it never selects $(4,14)$, for example. Independent marginal draws could select that pair, even though it is not among the refits.

In practice, a finite bootstrap set may not include every plausible combination, and an observed correlation can itself be imprecise. The point is not that joint empirical draws solve every dependence problem; it is that they retain the dependence information present in the bootstrap estimates instead of discarding it by design.

Propagating Parameter Uncertainty Through a Model

For a nonlinear model, the effect of uncertain parameters on a derived quantity need not be symmetric or equal to the effect evaluated at average parameter estimates. A direct calculation at each bootstrap vector carries the empirical shape and dependence through the model’s nonlinear function. This is one reason to simulate from the vectors rather than to summarize each parameter separately and then plug in marginal averages.

Worked Example: Half-Life From Joint Bootstrap Draws

Hypothetical data. Use the four $CL,V$ pairs above in a one-compartment calculation. For each pair, calculate the elimination half-life $t_{1/2}=\ln(2)V/CL$, in hours, using $\ln(2)\approx0.6931$.

For $(CL,V)=(4,20)$, $t_{1/2}=0.6931(20/4)=3.466$ h. For $(5,18)$, $t_{1/2}=0.6931(18/5)=2.495$ h. For $(6,16)$, $t_{1/2}=0.6931(16/6)=1.848$ h. For $(7,14)$, $t_{1/2}=0.6931(14/7)=1.386$ h.

The empirical mean half-life is $(3.466+2.495+1.848+1.386)/4=2.299$ h. In contrast, evaluating the formula at the separate parameter means gives $0.6931(17/5.5)=2.141$ h. These differ because the ratio $V/CL$ is nonlinear and the bootstrap parameter estimates are dependent. For uncertainty simulation, calculating the output for each complete bootstrap vector directly is the relevant operation; plugging in average parameter values does not reproduce that empirical output distribution.

Monte Carlo Error, Diagnostics, and Limitations

There are two finite-sample issues. First, the original data determine which bootstrap refits are obtained, so a limited number of resamples may poorly represent the sampling distribution, especially its tails. Second, a simulation that repeatedly samples from a fixed set of $B$ refits has Monte Carlo error. Increasing the number of draws from that fixed set reduces the latter but cannot create information missing from the bootstrap refits.

Check the resampling scheme, convergence rate, parameter constraints, marginal distributions, and pairwise relationships before using the draws. Look for extreme estimates, boundary estimates, multimodality, and strong asymmetry. Compare distributions across independent batches of bootstrap refits if feasible. If a few successful refits have extreme values, determine whether they reflect plausible sampling behavior, numerical instability, or an unsuitable model fit; do not remove them solely because they make a plot inconvenient.

The bootstrap also has limits. It approximates uncertainty under the observed sampling design and fitted analysis, not uncertainty about model misspecification, an unrepresented population, or a future covariate distribution. A nonparametric bootstrap may behave poorly when the data are sparse, the model is weakly identified, or many resampled data sets cannot support the model. The resampling unit and fitting procedure are part of the method, not implementation details that can be chosen arbitrarily.

Parameter draws alone represent uncertainty in fitted parameters. To simulate a distribution of individual observations, one may additionally draw random effects and residual error according to the model, as appropriate to the simulation question. Keeping these components conceptually separate makes it clear whether a wide simulated distribution comes from uncertain estimates, subject-to-subject variation, or observation-level variability.

Common Mistake

Common mistake: “I have bootstrap estimates for clearance and volume, so I will sample each parameter independently from its list and combine the draws.” This preserves the two marginal distributions but destroys their bootstrap dependence and can create parameter combinations unlike those produced by any refit. Correction: sample a complete row of the retained parameter table, then use that joint vector in the simulation. Also report and investigate failed refits, because the distribution of successful estimates can be distorted if failures are systematic.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

For repeated measurements on independent subjects, what should ordinarily be resampled in a nonparametric bootstrap, and why?
Why is drawing a complete bootstrap parameter vector different from drawing each parameter independently from its marginal list?
A bootstrap run has many failed refits. Why might using only the successful estimates distort the empirical distribution?
Suppose the simulated outcome is $g(\boldsymbol{\theta})$, a nonlinear function of the parameter vector. Why is evaluating $g$ at every bootstrap vector preferable to evaluating it at the mean vector?
Does repeatedly sampling from a fixed collection of bootstrap estimates remove uncertainty due to having only a limited number of bootstrap refits? Explain.

Solutions. (1) Resample whole subjects with all their records, so the within-subject structure is not broken by treating correlated records as independent. (2) Whole-vector draws retain the joint combinations and dependence observed across refits; independent draws can manufacture combinations not represented by the refits. (3) If failures are related to particular resampled data sets or parameter regions, the successful subset may overrepresent other regions. (4) Nonlinear functions of a distribution generally cannot be recovered by applying the function to the mean, and joint parameter dependence also affects the output. (5) No. More draws from the fixed collection reduce simulation noise conditional on that collection, but they do not add information to the empirical distribution or fix inadequate bootstrap coverage.