Why Bootstrap the Complete Fitting Procedure?
A sequential PK/PD analysis estimates PK parameters first, then uses those estimates or derived exposures in a PD fit. A joint analysis estimates PK and PD parameters together. In either case, uncertainty belongs to the estimation procedure as a whole, not just to its final optimization step. If a sequential analysis treats the estimated PK parameters as fixed, its PD uncertainty can omit variability introduced by the PK stage.
A subject-level bootstrap makes this distinction operational. It repeatedly constructs datasets by sampling subjects, then applies the same analysis procedure to every dataset. The crucial implementation rule is to repeat each fitting step that would be repeated if a new study sample had been collected. The earlier material on simulation-based comparison of estimation strategies considers operating characteristics across simulated studies; here the focus is estimating uncertainty from resampled subject records.
The resampling unit should match the independent sampling unit. When subjects are independent but their repeated measurements are not, resample subjects—not individual concentration or response records. If the design has independent sampling strata, such as treatment groups whose observed sizes are to be held fixed, resampling within strata may be appropriate. That choice targets uncertainty conditional on the stratum sizes; it should reflect the design and estimand rather than being adopted automatically.
What Must Be Repeated?
Let \(D\) denote the observed subject-level dataset and let \(T(D)\) represent the entire estimator, including data preparation and all fitting steps. The bootstrap replicate is \(T(D^{*(b)})\), where \(D^{*(b)}\) is a resampled dataset. For a sequential analysis, \(T\) includes the PK fitting stage and every downstream step that uses its outputs. For a joint analysis, \(T\) is the complete joint fitting procedure.
Draw \(n\) subject records with replacement. Keep each selected subject’s PK and PD observations together, including their time alignment and covariates. Treat repeated selections of the same subject as separate bootstrap copies.
Refit the PK model for the selected subject copies. Recalculate any PK-derived quantities, then refit the PD model and recompute the final sequential estimand using the same rules as in the original analysis.
Fit the PK and PD components jointly to the resampled records, re-estimating all parameters that were estimated in the original joint analysis, including relevant variability and residual parameters.
Record the estimate of the same parameter or estimand from each completed replicate. Repeat the procedure \(B\) times, assess failures, and summarize the resulting \(B\) estimates.
A duplicate subject must not be collapsed into one record: it represents two draws from the empirical subject distribution and therefore contributes twice to the replicate. In software, assign duplicate copies distinct bootstrap identifiers so that the fitting code does not mistakenly merge their records. Use the original fitting specifications, parameter constraints, data transformations, and estimand definition in each replicate. If the original method includes a data-dependent step, such as selecting a model component, decide in advance whether that step is part of the estimator and repeat it consistently.
Why the Bootstrap Spread Estimates Uncertainty
Write the observed subjects as \(D_1,\ldots,D_n\). Their empirical distribution assigns probability \(1/n\) to each observed subject record. A subject bootstrap samples \(n\) records from this empirical distribution, just as a new sample would draw from the population distribution if that unknown distribution were replaced by the observed one. The procedure therefore approximates the sampling variability of the complete estimator under the empirical distribution.
The bootstrap standard error is the sample standard deviation of the replicate estimates. Define their average as \(\bar{\theta}^{*}\). Then:
The denominator is \(B-1\) because this is the sample standard deviation of the bootstrap estimates. This is an estimate of the standard error of the original estimator, not the standard deviation of individual subject measurements. The bootstrap bias estimate, when useful, is \(\bar{\theta}^{*}-\hat{\theta}\), where \(\hat{\theta}=T(D)\) is the original-data estimate.
Worked Example: Repeating a Two-Stage Fit for One Resample
Suppose a hypothetical study has six independent subjects, A through F. Each subject has paired PK concentration and PD response records. The prespecified two-stage estimator first fits PK for each subject, uses that subject’s PK estimate in the PD fit, and reports the mean of the six individual PD parameter estimates. The original-data PD estimates, in arbitrary response units, are A: 0.7, B: 0.8, C: 0.9, D: 1.0, E: 1.2, and F: 1.4. The original two-stage estimate is
Consider one bootstrap draw: B, B, D, E, F, F. The multiplicities are \(m_A=0\), \(m_B=2\), \(m_C=0\), \(m_D=1\), \(m_E=1\), and \(m_F=2\); they sum to six. The PK and PD records for each selected copy remain paired. The two copies of B are separate bootstrap subjects, and likewise for F.
In this simplified illustration, refitting the same individual models to an exact duplicate gives the same individual estimate for each copy. The replicate two-stage estimate is therefore
In an actual implementation, the analyst must still repeat the PK fits and then the PD fits for this bootstrap dataset; simply resampling the six original PD estimates would not reproduce an estimator whose PD stage depends on refitted PK quantities. A joint analysis of the same resampled records would instead refit its full joint likelihood and store its joint estimate for this replicate.
Summarizing Bootstrap Uncertainty
The bootstrap standard error describes the estimated spread of the estimator. A percentile interval instead uses quantiles of the bootstrap estimates. For a two-sided \(100(1-\alpha)\%\) percentile interval, take the \(\alpha/2\) and \(1-\alpha/2\) quantiles of the replicate estimates. The interval’s coverage is an approximate repeated-sampling property; it is not a probability statement that the fixed parameter lies in this particular interval.
Worked Example: Bootstrap Standard Error and Estimated Bias
Suppose five illustrative bootstrap replicates of a PK/PD parameter measured in mg/L give \(0.10,\ 0.20,\ 0.30,\ 0.40,\ 0.50\) mg/L. These five replicates are deliberately small to make the arithmetic transparent; they would not normally be enough for a reliable uncertainty analysis. Their mean is
The squared deviations from \(0.30\) mg/L sum to \(0.10\;(\text{mg/L})^2\):
Thus the bootstrap standard error is
If the original-data estimate is \(0.28\) mg/L, the estimated bootstrap bias is \(0.30-0.28=0.020\) mg/L. Neither this bias estimate nor the standard error is precise with only five replicates; substantially more replicates are needed for stable summaries.
Worked Example: A Percentile Interval From Bootstrap Fits
For a hypothetical joint PK/PD analysis, suppose \(B=1000\) successful subject-level bootstrap fits estimate a parameter in mg/L. Sort the estimates from smallest to largest. Using the nearest-rank convention, the empirical \(q\) quantile is the value at rank \(\lceil qB\rceil\). Suppose the estimate at rank 25 is 0.74 mg/L and the estimate at rank 975 is 1.21 mg/L. Since \(\lceil 0.025(1000)\rceil=25\) and \(\lceil 0.975(1000)\rceil=975\), the 95% percentile interval is
This interval is based on the empirical 2.5th and 97.5th percentiles of the bootstrap estimates. Under suitable regularity and sampling conditions, the procedure is intended to give approximately 95% coverage across repeated samples. It does not mean that there is a 95% probability that the fixed parameter is in this realized interval. The interval can also perform poorly when the bootstrap distribution is strongly skewed, near a parameter boundary, or otherwise poorly represented by ordinary subject resampling.
Diagnostics, Alternatives, and Limitations
Bootstrap output should be inspected, not summarized automatically. Review the distribution of replicate estimates, the fraction of failed fits, and whether failures cluster in particular resamples or reflect a systematic problem. Examine whether parameters repeatedly hit bounds, whether estimates become implausible, and whether the replicate distribution has extreme tails or multiple clusters. These patterns can signal weak information, model instability, or an optimization problem. Report the number of attempted and successful replicates and the reasons for failures.
Do not silently discard failed replicates and present the remaining spread as if it were an unconditional summary. If failures are rare and clearly due to a correctable numerical issue, resolve the issue and rerun with the same prespecified fitting method. If failures are frequent or data-dependent, they are part of the procedure’s instability and should be reported; the nominal bootstrap summaries may not be trustworthy. A small \(B\) makes quantiles especially unstable, so percentile intervals generally require many more replicates than a rough standard-error calculation.
| Approach | What it estimates or represents | Key caution |
|---|---|---|
| Subject-level nonparametric bootstrap | Sampling variability under resampling of observed subject records, with the full estimator reapplied. | Can be unreliable with few subjects, sparse covariate groups, or unstable fits. |
| Covariance-based standard errors or the delta method | Approximate uncertainty from local model curvature or propagation of estimated covariance. | May rely on regularity and local approximation; a sequential calculation can omit upstream uncertainty if PK estimates are held fixed. |
| Parametric bootstrap | Sampling variability under data generated from a specified fitted model, followed by refitting. | Can be more informative for within-subject model-based variability, but depends on the adequacy of the assumed generating model. |
These approaches answer related but not identical questions. A subject bootstrap preserves the observed records within each resampled subject and targets sampling variation across subjects. A parametric bootstrap generates new records under a model and can represent within-subject sampling variation according to that model. Covariance-based methods can be computationally simpler, but are not a substitute for repeating omitted fitting stages. Compare methods only after stating which sources of uncertainty each includes.
A subject bootstrap cannot create information absent from the observed sample. With few subjects, resampling may repeatedly reproduce a narrow set of subject patterns, while rare covariate combinations may disappear from many replicates. The resulting distribution can be a poor guide to population uncertainty. In such settings, interpret bootstrap summaries cautiously and consider whether model-based simulation or another inferential approach better matches the target and assumptions.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A subject has repeated PK and PD observations. What should be resampled in a subject-level bootstrap, and why?
- In a sequential fit, which operations must be repeated if the target is uncertainty for the complete estimator?
- Six bootstrap estimates are \(1,2,3,4,5,6\) response units. Calculate their mean and bootstrap standard error.
- Why can dropping all failed bootstrap fits without comment make an uncertainty summary misleading?
Solutions. (1) Resample whole subject records, retaining each subject’s repeated PK and PD observations together, because measurements within a subject are not independent sampling units. (2) Refit the PK stage, recompute its derived inputs, and refit the PD stage, along with any other data-dependent steps in the defined estimator. (3) The mean is \((1+2+3+4+5+6)/6=3.5\) response units. The squared deviations sum to \(17.5\), so \(SE_{\mathrm{boot}}=\sqrt{17.5/(6-1)}=\sqrt{3.5}=1.871\) response units. (4) Failures may occur more often for particular resamples or parameter regions. Removing them can condition the reported distribution on successful fitting and understate or otherwise distort uncertainty; failure frequency and causes must be assessed and reported.