Two Intervals, Two Questions
A simulated interval is meaningful only when its target is clear. Is the interval describing uncertainty about a population quantity, such as the population mean exposure under a dosing regimen? Or does it describe the range of outcomes that could occur for a future individual or record? These questions require different simulation layers. Parameter uncertainty affects both; between-subject variability and residual error usually affect predictions for individuals, but are not themselves uncertainty in a population parameter.
For either interval, define the quantity before simulating. A population summary might be the population mean of AUC under fixed dosing and specified covariates. A prediction target might instead be the next concentration measurement for a new subject at a specified time. If the target changes—for example, from a new subject to a repeat observation on a known subject—the relevant random effects and errors may change too.
Which Simulation Layers Belong?
Let \(\theta\) denote the population-model parameters, including fixed effects and any variance or covariance parameters. Let \(b\) denote a subject's random effects, \(x\) the covariates, and \(\epsilon\) residual error. A model-based outcome can be written generally as \(Y=f(\theta,b,x,\epsilon)\). A population estimand \(\psi\), by contrast, is a specified summary of the model-implied population, such as a mean or quantile under a defined covariate distribution and dosing schedule.
To form an interval for \(\psi\), simulate plausible parameter values and calculate the same population summary for each draw. In doing so, integrate over the random-effect distribution or otherwise calculate the intended population summary; do not mistake the spread of individual outcomes for uncertainty in that summary. To form a prediction interval, simulate the future outcome itself, including whichever covariate, subject, occasion, and residual layers the prediction target calls for. Parameter uncertainty may also be included if the interval is intended to reflect uncertainty from estimating the model.
For \(B\) simulated parameter draws \(\theta^{(1)},\ldots,\theta^{(B)}\), calculate \(\psi^{(b)}\), the target population summary under draw \(b\). A common simulation-based interval uses the empirical quantiles of these \(B\) values. For a nominal 95% percentile interval, the limits are the 2.5th and 97.5th percentiles. The interval's coverage interpretation depends on how the parameter draws were obtained and on the validity of the method: a bootstrap interval is intended as a frequentist confidence interval, while posterior-draw quantiles give a Bayesian credible interval.
For prediction, each draw must instead produce a future outcome \(Y_{\mathrm{new}}^{(b)}\), with the relevant sources of variability sampled according to the prediction target. The corresponding empirical quantiles describe a predictive distribution under the model. If parameter draws are included, this distribution reflects both uncertainty in the fitted parameters and variation in future outcomes. If parameters are held fixed, it reflects only the outcome variation represented by the simulated subject, occasion, covariate, and residual layers.
Worked Examples
Worked Example: A Confidence Interval for a Population Exposure Summary
Suppose a hypothetical nonparametric bootstrap produced 4,000 valid parameter sets for a fitted NLME model. For each set, the analyst calculated the population geometric mean AUC under one fixed dosing schedule and a specified covariate profile. The 2.5th and 97.5th percentiles of the resulting log summaries were 1.788 and 2.8171, where each summary is \(\ln(G/(1\ \mathrm{mg\cdot h/L}))\), and \(G\) is the population geometric mean AUC.
This is a population-summary target. Each bootstrap replicate varies the fitted parameters; the analyst calculates the population summary rather than treating one newly drawn subject's AUC as the target. Exponentiating the endpoints returns the interval to the AUC scale:
Under the assumptions supporting the bootstrap procedure, this is an approximate 95% confidence interval for the specified population geometric mean AUC. It is not an interval containing 95% of individual subjects' AUC values. The exponentiation is appropriate because the reported target was defined on the log scale and then transformed back; the endpoints remain in the original AUC units.
Worked Example: Predicting a Future Concentration
Consider a hypothetical model at one fixed time and covariate profile. With parameters held fixed, suppose the typical concentration is 10 mg/L and a future subject's log concentration follows \(\ln(C/(1\ \mathrm{mg/L}))=\ln(10)+\eta+\epsilon\). Assume independent random effects \(\eta\sim N(0,0.25^2)\) and residual errors \(\epsilon\sim N(0,0.20^2)\). These assumptions make the log concentration normally distributed, with variance equal to the sum of the two variances.
The model-based 95% prediction interval for a single future observed concentration, conditional on the fixed parameters, is calculated on the log scale and then exponentiated:
Here \(1.96(0.3202)=0.6276\), so the log-scale limits are approximately \(1.675\) and \(2.930\). Exponentiating gives approximately 5.34 and 18.73 mg/L. This is a prediction interval for an observed concentration because it includes both between-subject variability and residual error. It is not an interval for the typical concentration or for uncertainty in the population parameters. The calculation assumes the stated normal distributions, independence, and fixed covariates and parameters.
Worked Example: Adding Parameter Uncertainty to a Prediction
Suppose, for illustration, a fitted population mean is 10 mg/L, its estimated standard error is 1 mg/L, and a future individual outcome has a normal subject-level and residual standard deviation of 3 mg/L. Assume the estimator's error is independent of the future individual's deviation from the population mean. A normal-approximation 95% confidence interval for the population mean is
For prediction of one future individual outcome, the prediction error includes both uncertainty in the estimated mean and variation around the true mean. By independence, their variances add:
The first interval concerns the population mean; the second concerns an individual future outcome under the stated approximation. The prediction interval is wider because it includes the assumed individual-level variation as well as uncertainty in the estimated mean. These calculations rely on normal approximations, a valid standard error, and independence; an actual NLME analysis may need simulation to represent nonlinear transformations, parameter dependence, or non-normal outcome distributions.
Why the Distinction Matters
A population summary may itself be defined by averaging over subject random effects. That does not mean the confidence interval should be widened by adding the raw spread of individual outcomes. For each parameter draw, the summary should represent the population quantity specified by the estimand. Random effects may be integrated over to calculate that quantity, but their individual-to-individual dispersion is not the same thing as uncertainty about the quantity.
Conversely, a future observed concentration is not fully represented by a confidence interval for the typical concentration. A prediction interval for a new subject generally needs a new subject-level random-effect draw and, for a measured record, residual error. If prediction concerns the latent concentration trajectory rather than a noisy observation, residual error may be excluded. If the target is a repeat measurement from a subject whose random effect is already known or conditioned upon, the subject-effect layer should be treated accordingly. State the target precisely before choosing the layers.
| Target | Typical simulation layers | Interpretation |
|---|---|---|
| Population summary, such as mean AUC | Parameter uncertainty; calculate the population summary for each draw | Uncertainty about a population quantity |
| Future latent outcome for a new subject | Parameter uncertainty if required; new subject random effects | Range of future underlying subject outcomes |
| Future observed record for a new subject | Parameter uncertainty if required; new subject effects and residual error | Range of future measured outcomes |
The intervals need not be symmetric, particularly for nonlinear models or positive quantities such as exposure. Quantiles of simulated values preserve asymmetry on the scale where they are calculated. If the simulation is performed on a transformed scale, back-transforming the interval endpoints gives limits on the original scale, but it does not make the interval symmetric there.
Diagnostics, Alternatives, and Limitations
Before interpreting an interval, audit the simulation against the target. Check that every replicate represents the same dosing schedule, time point, covariate condition, and summary definition. For a population-summary interval, verify that each replicate computes the population quantity rather than selecting one simulated individual. For a prediction interval, verify that the chosen subject, occasion, covariate, and residual layers match the future outcome being predicted. Examine simulated distributions for implausible values, skewness, multimodality, or failed model draws; such features can make a simple normal approximation unsuitable or expose a model or implementation problem.
A common alternative is a standard-error-based normal interval for a population parameter, such as an estimate plus or minus 1.96 standard errors. It can be concise and useful when the estimator is approximately normal and the standard error is reliable. Simulation-based quantiles can better represent skewness and nonlinear transformations. Prediction can likewise use an analytic distribution when its assumptions permit, as in the lognormal example, or direct simulation when the model is more complex. Neither simulation nor a formula repairs a misspecified model or an incorrectly defined target.
A frequent mistake is to take a confidence interval for a typical concentration and call it a prediction interval because it came from many simulations. Another is to include random-effect and residual draws in every replicate used for a population confidence interval, then interpret the resulting spread as parameter uncertainty. The correction is to label each simulated quantity: parameter draw, population summary, new-subject draw, occasion draw, and residual draw. Retain only the layers required by the stated target. Also distinguish a confidence interval's repeated-sampling coverage interpretation from a posterior credible interval's conditional probability interpretation.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A bootstrap produces estimates of the population geometric mean AUC. Should a new subject random effect be added to the target for each bootstrap estimate? Explain.
- A future measured concentration is required for a new subject. Which outcome-variation layers are ordinarily relevant?
- In the lognormal example, what would change if the target were the latent concentration rather than a measured concentration?
- Why is a prediction interval for an individual outcome usually wider than a confidence interval for the population mean?
Solutions. (1) No. Calculate the population geometric mean for each parameter draw; do not substitute one individual's random effect for the population summary. (2) Ordinarily draw a new subject's random effects and residual error, and include parameter uncertainty if the predictive target is intended to reflect uncertainty in the fitted parameters. (3) Omit residual error if the target is the latent concentration; retain the random-effect variation appropriate to the new subject. (4) The prediction interval includes individual outcome variation in addition to uncertainty in estimating the population mean, so its variance is generally larger under the assumptions in the example.