Sources Of Parameter Uncertainty In Simulation

Why the Source of Uncertainty Matters

A simulation can vary for several different reasons. The population parameter estimates may be uncertain because the data are finite; individuals may differ from one another under the population model; and observed measurements may depart from their model-predicted values because of residual error. These sources are not interchangeable. Including the wrong one can make a simulation answer a different question from the one intended.

For example, a prediction for a typical individual at the fitted population parameters is not the same target as a prediction for a new individual, nor is either the same as a prediction that also reflects uncertainty in the fitted parameters. Before drawing any values, specify the target: what quantity is to be simulated, for whom, and which sources of variation or uncertainty it is intended to represent?

Three Sources, Three Interpretations

Let \(\theta\) denote the population parameters, such as typical clearance or the population mean of a covariate effect. The fitted estimate is \(\hat{\theta}\). A measure such as \(\widehat{\operatorname{Cov}}(\hat{\theta})\) describes uncertainty in the estimated parameters, conditional on the model and estimation procedure. It is not the same object as \(\Omega\), the covariance matrix of individual random effects \(\eta\). Residual error is represented by a separate model for \(\epsilon\), whose form and scale depend on the specified observation model.

In a common hierarchical representation, an individual parameter depends on both the population parameters and that individual’s random effect. An observation then depends on the individual parameter and may also include residual error. For example, a log-normal clearance model can be written as \(CL_i=CL_{\mathrm{pop}}\exp(\eta_i)\), where \(CL_{\mathrm{pop}}\) is typical clearance and \(\eta_i\) is an individual random effect. An observation model might then add residual error to the predicted concentration. The random effect describes differences among individuals; it is not a draw from uncertainty in the estimated \(CL_{\mathrm{pop}}\).

The distinction is clearest when the target is stated explicitly. A typical-individual prediction at \(\hat{\theta}\) holds the parameters fixed and sets individual effects to their typical values. A new-individual prediction allows individual effects to vary. A simulated future observation may also include residual error. A prediction that propagates estimation uncertainty additionally represents uncertainty about \(\theta\), rather than treating \(\hat{\theta}\) as known.

These sources can coexist, but they answer different questions. Individual variability and residual error are generally included by the model for the simulated individual or observation; parameter uncertainty is included only when uncertainty in the fitted population parameters belongs to the target. The targets introduced in conditional and unconditional simulation are useful here: a simulation conditional on fixed parameters does not, by itself, propagate uncertainty in those parameter estimates.

A Justification for Combining Uncertainty Sources

A simple decomposition shows why the sources should be identified before combining them. Suppose a scalar outcome is \(Y=\theta+\eta+\epsilon\), where \(\theta\) is uncertain given the fitted data \(D\), and \(\eta\) and \(\epsilon\) have conditional means zero. Assume, for this illustration, that the two error terms are independent of each other and of \(\theta\), with variances \(\sigma_\eta^2\) and \(\sigma_\epsilon^2\). The law of total variance gives

Given \(\theta\), the conditional variance is \(\sigma_\eta^2+\sigma_\epsilon^2\), and the conditional mean is \(\theta\). Substitution therefore yields

The three terms have different interpretations: variability among individuals, residual variability in observations, and uncertainty in the population parameter. This sum depends on the stated assumptions and the additive example. In nonlinear models, transformations of random effects, heteroscedastic residual error, or dependence among sources can change the calculation. The general lesson remains: define the target and model first; do not add variance components mechanically.

Assumptions, Diagnostics, and Failure Modes

A simulation that represents these sources is only as meaningful as its assumptions. Parameter uncertainty is conditional on the fitted model, available data, and estimation procedure; it does not automatically capture uncertainty about model structure, omitted covariates, data quality, or whether the target population is appropriate. Likewise, \(\Omega\) describes the individual variability encoded in the model, and the residual error model describes deviations conditional on that model. Neither should be interpreted as a general allowance for every unmodeled difference.

Before interpreting results, check that the uncertainty measure refers to the parameters actually varied, that units are consistent, and that the simulated quantity matches the target. A standard error for clearance in L/h cannot be used as a residual standard deviation in mg/L. If the target is a typical-individual prediction, adding \(\eta\) or \(\epsilon\) changes the target; if it is a future observation, omitting both may understate the variation the target is meant to include. A useful diagnostic is to label every source in the simulation specification and verify that each simulated quantity has a clear interpretation.

For nonlinear predictions, a first-order uncertainty approximation can be inaccurate when parameter uncertainty is large, the prediction is strongly curved, or a parameter is near a constraint. Compare the approximate calculation with direct evaluations over a plausible range of parameter values. If estimates are strongly dependent, treating each parameter’s uncertainty as unrelated can also misrepresent prediction uncertainty. The uncertainty representation should preserve relevant dependence; the mechanics of generating correlated parameter draws are a separate topic.

A point prediction at fitted parameters is the simplest alternative. It is appropriate when the target is explicitly conditional on those estimates, but it does not express parameter uncertainty. A simulation with individual effects addresses between-individual variability; adding residual error addresses future observed values. Methods based on resampling fitted data or Bayesian posterior draws can provide other representations of estimation uncertainty, but they have their own assumptions and are not interchangeable without regard to the target or estimation framework.

Common Mistake: Calling Every Spread “Parameter Uncertainty”

A frequent error is to see a wide simulated distribution and describe it as uncertainty in the estimated population parameters. That conclusion is not justified if the spread came from drawing individual random effects or residual error while keeping the fitted parameters fixed. Conversely, a narrow distribution from repeated predictions at one fixed estimate does not establish that the parameter estimates are precise; the simulation may simply have omitted parameter uncertainty.

The correction is to state what was held fixed and what was varied. For example: “This distribution varies individual random effects and residual error at the fitted population estimates; it does not propagate estimation uncertainty in those estimates.” That wording identifies what the simulation can support without overstating its interpretation.