Why Draw Parameter Vectors Jointly?
A fitted pharmacometric model produces an estimate of its parameters, but simulation results can also depend on uncertainty in that estimate. One way to represent this uncertainty is to draw complete parameter vectors from a multivariate normal approximation centered at the estimate. Each draw is a possible joint value of the model parameters under that approximation; it is not a new individual’s random effect, nor is it automatically a Bayesian posterior draw.
The joint aspect matters. A parameter vector contains several estimates whose uncertainty may be related. A multivariate draw uses the estimated covariance matrix to represent both the uncertainty of each parameter and the estimated linear dependence among parameter estimates. Drawing each parameter independently instead would impose zero covariance, which may not be supported by the fitted model.
How to Generate a Draw
To generate one draw, take a vector \(\mathbf Z\) of \(k\) independent standard normal values and a matrix \(\mathbf A\) satisfying \(\mathbf A\mathbf A^\mathsf{T}=\widehat{\mathbf V}\). Then calculate \(\boldsymbol{\Theta}=\hat{\boldsymbol{\theta}}+\mathbf A\mathbf Z\). Repeat with a fresh independent \(\mathbf Z\) for each draw. The matrix \(\mathbf A\) is a square-root factor of the covariance matrix. The particular factorization method is a numerical implementation choice; the required property is that its product with its transpose gives the covariance matrix being used.
This construction assumes that the covariance estimate is symmetric and positive semidefinite, so that a real covariance square root exists. If it is positive definite, the resulting normal distribution has a density throughout the full \(k\)-dimensional parameter space. If it is singular but positive semidefinite, the distribution is instead restricted to the lower-dimensional affine subspace \(\hat{\boldsymbol{\theta}}+\operatorname{Range}(\widehat{\mathbf V})\). In either case, the covariance describes uncertainty on the same parameter scale and in the same order as the estimate.
The approximation is most defensible when the estimator’s sampling distribution is reasonably close to normal near the fitted value and the covariance estimate adequately represents its uncertainty. These conditions are often motivated by a well-behaved, approximately quadratic likelihood near an interior estimate. They are assumptions to assess, not guarantees supplied by the act of drawing normal values.
Why the Construction Has the Intended Covariance
The construction follows directly from the mean and covariance of an affine transformation. Since the components of \(\mathbf Z\) have mean zero and identity covariance, adding \(\hat{\boldsymbol{\theta}}\) shifts the mean but does not alter covariance. Multiplying by \(\mathbf A\) transforms the covariance on both sides.
Thus, if \(\widehat{\mathbf V}\) is positive definite, the probability density of a draw at \(\boldsymbol{\theta}\) is
The quadratic expression measures distance from the estimate while accounting for covariance. This density formula requires an invertible covariance; for a singular covariance, the distribution has no density over the full \(k\)-dimensional space, although the affine construction still defines the draw.
Worked Example: One Two-Parameter Draw
Suppose a hypothetical model estimates clearance as \(5.0\) L/h and volume as \(20.0\) L. For this example, use the covariance matrix below, whose diagonal entries have units \((\text{L/h})^2\) and \(\text{L}^2\), and whose off-diagonal entries have units \((\text{L/h})(\text{L})\).
A suitable factor for this matrix is \(\mathbf A=\begin{pmatrix}0.20&0\\0.06&0.08\end{pmatrix}\), because \(\mathbf A\mathbf A^\mathsf{T}= \begin{pmatrix}0.0400&0.0120\\0.0120&0.0100\end{pmatrix}\). Take the hypothetical standard normal vector \(\mathbf z=(1.0,-0.5)^\mathsf{T}\). The generated vector is
This is one draw, not a summary of the distribution. Across many independent draws, the target means are \(5.0\) L/h and \(20.0\) L, and the target variances and covariance are the entries of \(\widehat{\mathbf V}\). The off-diagonal entry is positive, so the approximation represents positive linear covariance between the two estimated parameters.
Worked Example: Three Independent Parameter Components
Consider a hypothetical estimate \(\hat{\boldsymbol{\theta}}=(5.0\ \text{L/h},20.0\ \text{L},0.250\ \text{h}^{-1})^\mathsf{T}\), with diagonal covariance corresponding to standard errors \(0.20\) L/h, \(1.0\) L, and \(0.020\) h\(^{-1}\). This example deliberately assumes zero covariance among the three estimated components.
For \(\mathbf z=(-1.0,0.5,1.5)^\mathsf{T}\), the standardized deviations become \(\mathbf A\mathbf z=(-0.20\ \text{L/h},0.50\ \text{L},0.030\ \text{h}^{-1})^\mathsf{T}\). Therefore,
The arithmetic is straightforward, but the zero off-diagonal entries are a substantive assumption in this example. They assert that the normal approximation has no pairwise covariance among these estimate components. A diagonal matrix should not be substituted merely for convenience if the fitted model provides meaningful off-diagonal covariance.
Worked Example: A Singular Covariance and Its Support
Suppose, for illustration, that a two-component estimate has mean \((0,1)^\mathsf{T}\) and covariance \(\begin{pmatrix}1&0\\0&0\end{pmatrix}\). A valid factor is the same diagonal matrix. For a standard normal vector \((Z_1,Z_2)^\mathsf{T}\), the draw is
The second component is always 1, while the first varies normally around 0. The support is the affine line \(y=1\), which is the mean vector plus the range of the covariance matrix. It is not a linear subspace through the origin. In practice, a zero eigenvalue can signal an exact linear dependency, an unidentifiable direction, or numerical degeneracy; it should not automatically be interpreted as evidence that uncertainty is truly zero.
Checks Before and After Drawing
A reliable workflow checks both the inputs and the generated sample. Small numerical discrepancies can occur, but they should be distinguished from a covariance matrix that is invalid or inconsistent with the fitted model.
Confirm that row and column order in \(\widehat{\mathbf V}\) matches the parameter order in \(\hat{\boldsymbol{\theta}}\), and that every entry uses the corresponding parameter scales and units.
Check symmetry and positive semidefiniteness. A materially negative eigenvalue means the supplied matrix cannot be used as a covariance matrix as written. Small negative values near numerical precision require investigation, not automatic acceptance.
For each replicate, form \(\hat{\boldsymbol{\theta}}+\mathbf A\mathbf Z\), using a factor that reconstructs the covariance being used.
With enough draws, compare sample means and covariances with \(\hat{\boldsymbol{\theta}}\) and \(\widehat{\mathbf V}\). Sampling variation means they will not match exactly; large or systematic discrepancies suggest a coding, ordering, or factorization error.
Draws far from the estimate are not by themselves proof of a software error: a normal distribution has unbounded support. However, a substantial fraction of implausible or invalid parameter values is a warning that the approximation may be unsuitable on the chosen scale. The normal model does not enforce scientific bounds or parameter constraints. Assessing or changing parameter scales is a separate modeling decision; do not silently discard or alter draws without documenting the effect on the implied distribution.
Interpretation, Alternatives, and Limits
A normal draw represents parameter-estimation uncertainty under the specified approximation and covariance estimate. In a frequentist analysis, it can approximate the sampling distribution of an estimator; it does not mean there is a stated probability that a fixed true parameter lies in a particular interval. Nor does one draw establish that a parameter value is plausible under every source of uncertainty. The approximation is conditional on the fitted model, its data, the covariance estimate, and the assumptions used to obtain them.
| Approach | What it represents | Main limitation or distinction |
|---|---|---|
| Use only the point estimate | A single fixed parameter vector for simulation | Does not propagate parameter-estimation uncertainty. |
| Draw each parameter independently | Separate marginal normal variation around each estimate | Imposes zero covariance and may misrepresent joint uncertainty. |
| Multivariate normal draws | Joint variation described by an estimate and covariance matrix | Relies on the adequacy of the normal approximation and covariance estimate. |
| Bootstrap or Bayesian draws | Uncertainty draws obtained through a different inferential procedure | They have different constructions and interpretations; they are not interchangeable by name alone. |
This method is especially sensitive to poor covariance estimates, weakly identified parameters, boundary estimates, and likelihood shapes that are strongly asymmetric or otherwise poorly approximated by a quadratic form. In those settings, the generated normal vectors may be mathematically correct for the supplied matrix but still be a poor representation of uncertainty. The method also includes only the uncertainty encoded in \(\widehat{\mathbf V}\); it does not automatically add individual variability, residual error, covariate uncertainty, or uncertainty about model structure.
Common Mistakes
- Using only the diagonal variances. This discards estimated covariances. Use the full matrix when the goal is to represent the reported joint covariance.
- Mixing parameter order or scales. A correctly factored matrix paired with the wrong parameter order still produces incorrect draws. Check labels and units before sampling.
- Treating draws as guaranteed valid parameter values. A normal distribution can generate values outside a parameter’s scientifically admissible region. That limitation is part of the approximation, not evidence that the draw algorithm has enforced constraints.
- Calling these posterior draws. A multivariate normal approximation centered on an estimate and based on its covariance is not, without a Bayesian derivation, a posterior distribution.
- Calling a singular support a linear subspace. The support is the mean plus the covariance range, an affine subspace that need not pass through the origin.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- If a covariance matrix has a diagonal entry of \(0.09\), what is the corresponding standard error, and what does that entry measure?
- Why can independent marginal normal draws fail to represent a fitted joint parameter covariance?
- For mean \((2,3)^\mathsf{T}\) and covariance \(\operatorname{diag}(4,0)\), describe the support of the distribution.
- What does it suggest if a large simulated sample has a covariance far from the input covariance?
- Does drawing from \(N_k(\hat{\boldsymbol{\theta}},\widehat{\mathbf V})\) automatically account for individual variability? Explain.
Solutions. (1) The standard error is \(\sqrt{0.09}=0.300\), in the units of that parameter. The diagonal entry is its estimated variance. (2) Independent draws impose zero off-diagonal covariance, so they omit any nonzero estimated covariance and can misrepresent joint uncertainty. (3) The first component varies with variance 4 and the second is fixed at 3; the support is the affine line \(y=3\), equal to the mean plus the covariance range. (4) It suggests checking the factorization, parameter ordering, covariance reconstruction, and random-number implementation; finite-sample discrepancies are expected, but a large systematic mismatch is not. (5) No. These draws represent only the parameter uncertainty encoded by the estimate and covariance. Individual variability requires a separate model and simulation step.