Why Parameter Dependence Matters
An estimated pharmacometric model usually has several parameters whose estimates are uncertain together. For example, estimates of clearance and volume may move in related ways across plausible fits, even when each parameter’s marginal uncertainty is summarized correctly on its own. A simulation that samples each parameter independently can retain the right mean and standard deviation for every parameter while creating combinations that do not represent the fitted model’s joint uncertainty.
This matters because predictions are functions of several parameters. Their uncertainty depends not only on how uncertain each parameter is, but also on how parameter deviations occur together. If deviations reinforce one another, independent sampling can understate predictive uncertainty; if they tend to offset, it can overstate it. For nonlinear predictions, dependence can also change the average prediction, not just its spread.
The earlier material on multivariate normal sampling from parameter covariance establishes the idea of generating joint parameter draws. Here the emphasis is on why the dependence must be retained, what independent sampling changes, and how to check the consequences for a prediction. The examples use hypothetical parameter uncertainty only; they do not represent a real study.
Covariance and the Joint Draw
A joint parameter draw is one vector of values generated together, such as \((\theta_1^{(b)},\theta_2^{(b)})\) for draw \(b\). Preserving dependence means that the pairing between the values is retained according to the intended joint uncertainty distribution. Independent sampling instead combines draws from the separate marginal distributions as if \(\operatorname{Cov}(\theta_1,\theta_2)=0\). It is justified only when independence is an appropriate assumption for the uncertainty representation being used.
For a covariance matrix \(\Sigma\), its diagonal entries are the parameter variances and its off-diagonal entries are covariances. A covariance matrix must be symmetric and positive semidefinite. The correlation matrix expresses the same dependence on a scale that is easier to compare across parameters. Neither matrix alone guarantees that a chosen distribution describes all important features of uncertainty; for example, a covariance matrix does not encode skewness or multiple modes.
How Dependence Enters Prediction Uncertainty
For a linear prediction \(g=a\theta_1+b\theta_2\), expanding the variance gives a direct justification for retaining covariance:
The cross term follows by expanding the squared deviation: \((a(\theta_1-E[\theta_1])+b(\theta_2-E[\theta_2]))^2\). The two cross-products have the same expectation, giving \(2ab\operatorname{Cov}(\theta_1,\theta_2)\). Setting covariance to zero removes this term. That is not a neutral computational simplification: it changes the prediction variance unless the covariance is zero or the coefficient product \(ab\) is zero.
For a nonlinear prediction \(f(\boldsymbol{\theta})\), a first-order approximation around the parameter mean uses the gradient \(\nabla f\). It gives \(\operatorname{Var}(f(\boldsymbol{\theta}))\approx \nabla f^\mathsf{T}\Sigma\nabla f\), with the gradient evaluated at that mean. This approximation can help explain the direction of an effect, but it may be inaccurate for strongly nonlinear functions or broad uncertainty. Joint simulation evaluates the prediction at each joint draw and summarizes those simulated predictions instead.
Worked Example: Covariance Changes a Linear Prediction’s Variance
Suppose hypothetical parameter estimates have standard deviations \(0.20\) and \(0.40\), with correlation \(0.80\). Consider a prediction \(g=\theta_1-2\theta_2\). The parameter units are arbitrary model units, and the prediction has the corresponding units of \(g\).
The covariance is \(\operatorname{Cov}(\theta_1,\theta_2)=0.80(0.20)(0.40)=0.0640\). The marginal variances are \(0.20^2=0.0400\) and \(0.40^2=0.160\). Using \(a=1\) and \(b=-2\),
Thus the standard deviation is \(\sqrt{0.424}=0.651\) prediction units. If the same marginal distributions were sampled independently, the covariance term would be zero, giving variance \(0.0400+4(0.160)=0.680\) and standard deviation \(\sqrt{0.680}=0.825\). In this contrast, positive parameter covariance reduces prediction variance because the parameters enter with opposite-signed coefficients. Independent sampling would overstate this variance relative to the specified joint uncertainty.
Worked Nonlinear Prediction: Same Margins, Different Dependence
A nonlinear transformation makes the consequences especially clear. The following small discrete distribution is hypothetical and chosen so that its probabilities and calculations can be checked exactly. It is an illustration of dependence, not an assertion that parameter estimates generally follow a four-point distribution.
Worked Example: Correlated Draws for an Exponential Prediction
Let \(\theta_1\) and \(\theta_2\) be deviations on a log scale, each with marginal values \(-0.20\) and \(0.20\) for \(\theta_1\), and \(-0.40\) and \(0.40\) for \(\theta_2\), each with probability \(1/2\). In the joint distribution, the two same-sign outcomes each have probability \(0.45\), and the two opposite-sign outcomes each have probability \(0.05\).
| \(\theta_1\) | \(\theta_2\) | Probability | \(\theta_1+\theta_2\) | \(R=\exp(\theta_1+\theta_2)\) |
|---|---|---|---|---|
| \(-0.20\) | \(-0.40\) | 0.45 | \(-0.60\) | 0.5488 |
| \(0.20\) | \(0.40\) | 0.45 | \(0.60\) | 1.822 |
| \(-0.20\) | \(0.40\) | 0.05 | \(0.20\) | 1.221 |
| \(0.20\) | \(-0.40\) | 0.05 | \(-0.20\) | 0.8187 |
Both marginal means are zero. Their variances are \(0.0400\) and \(0.160\). The covariance is \(0.45(0.0800)+0.45(0.0800)+0.05(-0.0800)+0.05(-0.0800)=0.0640\), so the correlation is \(0.0640/(0.20\times0.40)=0.80\). The prediction \(R\) is a dimensionless multiplicative factor.
Under the correlated distribution, its expected value is \(0.45(e^{-0.60}+e^{0.60})+0.05(e^{0.20}+e^{-0.20})\) \(=0.45(0.5488+1.822)+0.05(1.221+0.8187)=1.169\), to three significant figures.
Now sample the two marginal distributions independently. Each of the four possible combinations then has probability \(0.25\), and the expected prediction becomes \((0.5488+1.822+1.221+0.8187)/4=1.103\). Thus independent sampling changes the mean prediction even though it retains both marginal distributions. The difference arises because the joint distribution assigns different probabilities to reinforcing and offsetting parameter deviations.
The prediction variance also changes. For the correlated distribution, \(E[R^2]=0.45(e^{-1.20}+e^{1.20})+0.05(e^{-0.40}+e^{0.40})=1.738\), so \(\operatorname{Var}(R)=1.738-(1.169)^2\approx0.3713\). For independent sampling, \(E[R^2]=[e^{-1.20}+e^{1.20}+e^{-0.40}+e^{0.40}]/4=1.446\), giving \(\operatorname{Var}(R)=1.446-(1.103)^2\approx0.2298\). The correlated draws therefore produce greater predictive variance in this example. The direction and size of the change depend on the prediction function and the dependence structure; they should not be assumed in advance.
Keeping Paired Draws Intact
When uncertainty is represented by a set of joint parameter vectors, the row or draw index is part of the information. Separating parameter columns and re-sampling them independently preserves each column’s marginal values but can destroy their estimated association. The next example shows that this changes a nonlinear prediction even for a small, fully inspectable set.
Worked Example: Independent Pairing Alters a Prediction
Suppose four hypothetical equally weighted parameter-deviation draws are \((\theta_1,\theta_2)=(-0.20,-0.40),(-0.10,-0.20),(0.10,0.20),(0.20,0.40)\). Both marginal means are zero. Using the sample covariance convention with denominator \(n-1\), the covariance is \((0.0800+0.0200+0.0200+0.0800)/3=0.0667\); the sample standard deviations are \(0.1826\) and \(0.3651\), respectively. These paired deviations are perfectly positively correlated.
For \(R=\exp(\theta_1+\theta_2)\), the four sums are \(-0.60,-0.30,0.30,0.60\). The prediction values are \(0.5488,0.7408,1.350,1.822\), so their equally weighted mean is \((0.5488+0.7408+1.350+1.822)/4=1.115\).
Re-pair the same four marginal values as \((-0.20,-0.20),(-0.10,0.40),(0.10,-0.40),(0.20,0.20)\). The marginal values and marginal sample standard deviations are unchanged, but the sum of the four cross-products is \(0.0400-0.0400-0.0400+0.0400=0\); the re-paired sample covariance is zero. The new sums are \(-0.40,0.30,-0.30,0.40\), and the predictions are \(0.6703,1.350,0.7408,1.492\). Their mean is \((0.6703+1.350+0.7408+1.492)/4=1.063\). The altered prediction summary is a consequence of changing the pairings, not of changing either parameter’s marginal sample.
Assumptions, Diagnostics, and Alternatives
A covariance-preserving simulation is meaningful only relative to a defined uncertainty target. State which parameter estimates are uncertain, the scale on which they are represented, and which fitted scenario or model is being evaluated. Then use a joint distribution or set of joint draws that represents the uncertainty of those estimates. If a multivariate normal approximation is used, it assumes the parameter vector is adequately represented by a multivariate normal distribution on the scale sampled; that assumption can be poor near boundaries or for skewed estimates. Transformations used to respect parameter constraints can also alter dependence, so inspect dependence on the scale actually used for simulation.
Useful checks include comparing the simulated marginal means and standard deviations with their targets, comparing the simulated covariance or correlation matrix with the intended one, and checking that probabilities and simulated values are finite. For prediction-focused checks, compare distributions of key outputs under joint draws and under independent sampling; large differences are a reason to investigate the dependence structure and its scientific plausibility. Also check that parameter columns were not accidentally reordered or sampled with different draw indices.
The main approaches differ in what they preserve. Independent marginal sampling is simple but imposes zero dependence. A joint covariance-based distribution preserves a specified covariance structure, subject to its distributional assumptions. A collection of paired joint draws preserves the dependence represented by those draws, although its accuracy depends on how the draws were obtained and how well they cover the uncertainty. For any approach, a covariance matrix summarizes second moments but cannot by itself capture every feature of a non-normal joint distribution. No covariance-preserving method repairs a misspecified model, an inaccurate covariance estimate, or an uncertainty distribution that omits important parameter regions.
The practical limitation is that estimates can be highly correlated or poorly identified, making the joint uncertainty sensitive to modeling and estimation choices. Strong correlation is not, by itself, a reason to remove dependence. Instead, investigate the fit, parameter scale, covariance estimation, and prediction sensitivity. If estimates have transformed constraints, check that the covariance and draws refer to consistent scales. Report the uncertainty representation and the prediction summaries it produces so another analyst can reproduce the simulation.
Check Your Understanding
Answer each question, then compare with the solutions.
- If \(\operatorname{SD}(\theta_1)=0.30\), \(\operatorname{SD}(\theta_2)=0.50\), and their correlation is \(-0.40\), what is their covariance?
- For \(g=\theta_1+\theta_2\), why does positive covariance increase variance relative to independent sampling?
- In the exponential prediction example, why can the expected prediction differ under independent and correlated draws even though the marginal means and variances match?
- A simulation reproduces each parameter’s marginal histogram but its correlation matrix is close to zero. What should be checked before interpreting the prediction intervals?
Solutions. 1. The covariance is \((-0.40)(0.30)(0.50)=-0.0600\), in the product of the parameter units. 2. The variance is \(\operatorname{Var}(\theta_1)+\operatorname{Var}(\theta_2)+2\operatorname{Cov}(\theta_1,\theta_2)\); positive covariance makes the cross term positive. 3. The exponential function is nonlinear, and dependence changes the probabilities of the joint combinations of parameter values; those combinations determine the transformed prediction distribution. 4. Check whether independent sampling was inadvertently applied, whether paired draw indices or parameter ordering were lost, and whether the target covariance was specified on the same parameter scale used for simulation. Then compare key prediction summaries using the intended joint draws.