Reviewing Propagation of Parameter Uncertainty

Why Review Parameter-Uncertainty Propagation?

A simulated prediction can vary for several reasons. The model parameters are estimated rather than known exactly; individuals may differ through random effects; and observations may vary around the model prediction because of residual error. These sources are not interchangeable. In particular, a simulation that fixes every parameter at its estimate can produce predictions that look precise while omitting uncertainty in the estimated model.

This review focuses on whether uncertainty in the estimated parameters is carried into derived predictions as intended. The key checks are whether the draws use the correct parameter covariance information, whether they are made on the correct parameter scale, and whether the reported interval corresponds to the quantity being predicted. The earlier material on parameter precision and covariance results provides the estimation-output context; here the emphasis is on tracing that uncertainty into predictions.

A common approximation for an unconstrained parameter vector is \(\boldsymbol{\theta}^{(b)} \sim N(\hat{\boldsymbol{\theta}},\widehat{\mathbf{V}}_{\theta})\), where \(\hat{\boldsymbol{\theta}}\) is the estimate and \(\widehat{\mathbf{V}}_{\theta}\) is its estimated covariance matrix. This approximation is appropriate only if the distribution is reasonably represented by a multivariate normal on the stated parameter scale and the covariance matrix corresponds to the same parameterization and ordering as the draw routine. It is not automatically suitable for parameters constrained to be positive or for estimates near a boundary.

What a Covariance-Aware Draw Must Preserve

The diagonal entries of \(\widehat{\mathbf{V}}_{\theta}\) are estimated variances; the off-diagonal entries are estimated covariances. A draw procedure that uses only the diagonal entries imposes zero correlation among parameters, even when the estimation output indicates otherwise. This can change the uncertainty of a derived prediction substantially: when a prediction depends on a difference between two parameters, positive covariance can reduce its variance, while negative covariance can increase it.

For an unconstrained multivariate normal draw, one implementation is to find a matrix \(\mathbf{L}\) satisfying \(\widehat{\mathbf{V}}_{\theta}=\mathbf{L}\mathbf{L}^{T}\), generate a vector \(\mathbf{z}\) of independent standard normal values, and compute \(\boldsymbol{\theta}^{(b)}=\hat{\boldsymbol{\theta}}+\mathbf{L}\mathbf{z}\). The resulting draw has the target covariance because \(\operatorname{Var}(\mathbf{L}\mathbf{z})=\mathbf{L}\operatorname{Var}(\mathbf{z})\mathbf{L}^{T}=\mathbf{L}\mathbf{L}^{T}=\widehat{\mathbf{V}}_{\theta}\). This identity also explains why generating each component independently from its marginal standard error does not reproduce a covariance matrix with nonzero off-diagonal entries.

Before drawing, verify the parameter order, units, and scale. For example, a covariance matrix for log-clearance and log-volume cannot be applied as though it were a covariance matrix for clearance and volume on their original scales. If estimation was performed on a transformed scale, draw on that scale and transform back as required. The transformed values will generally not have a normal distribution on the original scale.

Propagating Covariance to a Derived Prediction

A prediction is often a nonlinear function \(g(\boldsymbol{\theta},x)\), where \(x\) denotes a specified design point, such as a time after dose. A first-order approximation, called the delta method, translates parameter covariance into approximate prediction variance. Let \(\boldsymbol{\theta}\) vary under the parameter-draw approximation, with mean \(\hat{\boldsymbol{\theta}}\) and covariance \(\widehat{\mathbf{V}}_{\theta}\). Define the row gradient \(\mathbf{J}=\left.\partial g(\boldsymbol{\theta},x)/\partial\boldsymbol{\theta}^{T}\right|_{\boldsymbol{\theta}=\hat{\boldsymbol{\theta}}}\). Then \(\operatorname{Var}\{g(\boldsymbol{\theta},x)\}\approx\mathbf{J}\widehat{\mathbf{V}}_{\theta}\mathbf{J}^{T}\).

This variance is not the variance of \(g(\hat{\boldsymbol{\theta}},x)\), which is a fixed number when the estimate and design point are held fixed. It describes variation in the prediction as the parameter vector varies under the stated approximation. The delta method is useful for a quick check or a nearly linear prediction; simulation from the full covariance matrix is more direct for nonlinear functions or asymmetric prediction distributions.

Do the Prediction Intervals Include the Right Variation?

A parameter-only interval and a prediction interval for a future observation answer different questions. The former describes uncertainty in the model’s predicted mean at a specified design point. A future observation can also vary around that mean because of residual error, and a prediction for a randomly selected individual may additionally include relevant random-effects variability. The target must therefore be stated before judging whether an interval is too narrow or too wide.

If parameter variation in the mean prediction and residual error are independent, their variances add for a future observation. If they are dependent, the covariance must also be included. In general, \(\operatorname{Var}(A+B)=\operatorname{Var}(A)+\operatorname{Var}(B)+2\operatorname{Cov}(A,B)\). Do not add a residual or random-effects component merely because it is available: include only sources relevant to the quantity the interval is intended to describe.

Practical Review: Diagnostics, Alternatives, and Limits

An independent review should follow the uncertainty from its source through to the reported interval. In practice, compare the estimation output with the simulation configuration and inspect the resulting draws and predictions. Useful checks include:

• Matrix and ordering: Confirm that the covariance matrix is finite, symmetric within numerical tolerance, and in exactly the same parameter order as the draw code. Check that its diagonal entries are variances, not standard errors.
• Admissibility: A covariance matrix must be positive semidefinite. A Cholesky factorization is a useful check for a positive-definite matrix; a singular but valid covariance may require a method that handles positive semidefiniteness. A failed factorization can indicate rounding, a mismatched matrix, or an invalid input.
• Draw summaries: Across a sufficiently large set of draws, compare empirical means, variances, and correlations with the intended values. Small differences are expected from Monte Carlo variability, but a persistent zero correlation when a nonzero covariance was specified is a warning sign.
• Scale and constraints: Check that positive parameters are generated through an appropriate transformation or constrained method. A normal draw directly on the original scale can produce impossible negative values; simply discarding such draws can also alter the intended distribution.
• Prediction behavior: Compare intervals with and without parameter variation and check that the reported interval targets a mean prediction or a future observation as stated. Look for impossible predictions, strong skewness, or a few extreme draws that dominate the interval.

The main practical alternatives are a first-order delta-method calculation, direct simulation from an approximate covariance distribution, and resampling-based approaches such as a bootstrap. The delta method is fast and transparent but relies on local linearity. Covariance-based simulation retains the joint uncertainty and can show nonlinear or asymmetric effects, but depends on the validity of the distributional approximation and the quality of the covariance estimate. A bootstrap can represent more of the estimation procedure’s behavior when correctly designed, but can be computationally demanding and may fail or produce unstable estimates in some replicates. These approaches do not remove the need to define the target prediction and the uncertainty sources it should include.

A frequent mistake is to report a narrow interval from predictions evaluated only at \(\hat{\boldsymbol{\theta}}\), then call it an interval that accounts for parameter uncertainty. The correction is to vary the parameter vector according to its estimated joint uncertainty, propagate each draw through the prediction, and summarize the resulting distribution. Another mistake is to draw parameters independently because the marginal standard errors are available. That is only consistent with the covariance output when the off-diagonal covariances are zero or are intentionally set to zero under a justified approximation.