Why Covariance-Matrix Draws Need Special Care
In an NLME model, the random effects for a subject are often represented as a vector \(\eta_i\) with distribution \(\eta_i \sim N(0,\Omega)\), where \(\Omega\) is the population random-effect covariance matrix. When \(\Omega\) is treated as uncertain, a simulation may draw candidate covariance matrices to represent uncertainty in the fitted model. Those draws must still be mathematically valid covariance matrices. Drawing each variance and covariance independently, or using an unconstrained multivariate normal approximation on the raw matrix elements, does not guarantee validity.
Two kinds of randomness must remain distinct. A covariance-matrix draw represents uncertainty about the population distribution of random effects. A random-effect draw is a draw of \(\eta_i\) conditional on a particular covariance matrix. This tutorial concerns the first task and the validity of the matrices it produces.
A positive-semidefinite matrix can be singular, for example if a random-effect variance is exactly zero or if two random effects have a perfectly determined linear relationship. Such boundary cases may have a model-specific interpretation, but are not ordinary nondegenerate covariance draws. The procedures below target positive-definite draws.
Validity Conditions in Two Dimensions
For two random effects, write the covariance matrix in terms of variances and a covariance:
Symmetry requires the two off-diagonal entries to be equal. For positive definiteness, both variances must be positive and the determinant must be positive. Equivalently, if \(\rho_{12}=\omega_{12}/(\sigma_1\sigma_2)\), then the correlation must satisfy \(-1 < \rho_{12} < 1\).
The condition is stricter than “all variances are positive.” A covariance that is too large in magnitude relative to the variances makes the determinant negative. In higher dimensions, checking diagonal entries alone is also insufficient; the full matrix must be positive definite.
Worked Example: Detecting an Invalid Covariance Draw
Suppose a hypothetical two-random-effect model uses effects measured on the log scale. A draw from an unconstrained approximation gives variances \(0.0400\) and \(0.0225\), and covariance \(0.0320\). Check whether the resulting matrix is valid.
The proposed matrix is
Both variances are positive and the matrix is symmetric, but positive definiteness also requires a positive determinant:
The determinant is negative, so this is not a valid covariance matrix. The implied correlation would be \(0.0320/(\sqrt{0.0400}\sqrt{0.0225})=0.0320/(0.200\times0.150)=1.067\), which is outside the allowable range. A multivariate normal draw with this matrix cannot be generated as specified.
Constructing Valid Draws From Standard Deviations and Correlations
For two random effects, one direct strategy is to draw standard deviations on a scale that guarantees positivity and draw the correlation on a scale that maps to the interval \((-1,1)\). For example, let \(a_1=\log(\sigma_1)\), \(a_2=\log(\sigma_2)\), and \(z=\operatorname{atanh}(\rho_{12})\). After drawing values for these transformed quantities, map them back using \(\sigma_j=\exp(a_j)\) and \(\rho_{12}=\tanh(z)\), then set \(\omega_{12}=\rho_{12}\sigma_1\sigma_2\).
The uncertainty distribution for the transformed quantities must reflect the uncertainty source being used, such as a suitable approximation, bootstrap draws, or posterior draws. Transforming parameters guarantees validity of the resulting matrix; it does not, by itself, ensure that the uncertainty distribution is well calibrated.
Worked Example: Mapping a Transformed Draw to a Valid Matrix
For a hypothetical model, consider one draw with \(\log(\sigma_1)=\log(0.200)\), \(\log(\sigma_2)=\log(0.150)\), and \(z=0.350\), where \(z=\operatorname{atanh}(\rho_{12})\). Convert the draw to a covariance matrix.
The standard deviations are \(\sigma_1=\exp(\log(0.200))=0.200\) and \(\sigma_2=\exp(\log(0.150))=0.150\). The correlation is \(\rho_{12}=\tanh(0.350)\approx0.3364\). Thus the covariance is \(\omega_{12}=0.3364(0.200)(0.150)\approx0.01009\), and the variances are \(0.200^2=0.0400\) and \(0.150^2=0.02250\).
Its determinant is approximately \((0.0400)(0.02250)-(0.01009)^2=0.000900-0.0001018=0.0007982\), which is positive. The matrix is symmetric, has positive variances, and is positive definite. The transformed draw therefore produces an admissible covariance matrix.
Cholesky Parameterization for Multiple Random Effects
For a covariance matrix with more than two random effects, a general construction uses a lower-triangular matrix \(L\) with strictly positive diagonal entries and defines \(\Omega=LL^\mathsf{T}\). The diagonal entries of \(L\) can be represented as exponentials of unconstrained values; its off-diagonal entries can be unconstrained. This construction guarantees symmetry and positive definiteness.
because a triangular \(L\) with nonzero diagonal is nonsingular, so \(L^\mathsf{T}x\ne0\) whenever \(x\ne0\). This is the key justification: any draw of the unconstrained entries that preserves positive diagonal entries yields a positive-definite matrix. The transformation does not prescribe the right uncertainty distribution for those entries; that distribution must still be chosen to represent the uncertainty in the fitted covariance parameters.
Worked Example: Building a Three-Effect Covariance Matrix
Suppose a hypothetical draw uses the lower-triangular factor
The diagonal entries are positive. Multiplying \(L\) by its transpose gives the covariance matrix:
For example, the second variance is \(0.030^2+0.150^2=0.000900+0.02250=0.02340\); the covariance between the second and third effects is \((0.030)(-0.020)+(0.150)(0.040)=-0.000600+0.00600=0.00540\). The product construction is symmetric by definition, and the positive diagonal of \(L\) makes it nonsingular. Therefore the resulting \(\Omega\) is positive definite without needing to separately constrain each covariance.
How to Check Draws and Recognize Failure
Even when a parameterization is intended to preserve validity, numerical implementation should be checked. For each draw, verify that entries are finite, the matrix is symmetric within a stated numerical tolerance, and its diagonal entries are positive. A Cholesky factorization is a practical positive-definiteness check: successful factorization indicates positive definiteness in exact arithmetic, while failure signals a problem or a near-boundary matrix affected by numerical precision. Eigenvalues can provide another check; a positive minimum eigenvalue supports positive definiteness.
For a two-effect matrix, inspect the implied correlation as well as the determinant. For larger matrices, inspect eigenvalues or use Cholesky rather than relying only on pairwise correlations: individually admissible pairwise correlations do not guarantee that the full correlation matrix is positive definite.
Use a distribution for covariance parameters that corresponds to the intended uncertainty source, on a suitable constrained or unconstrained parameterization.
Map it to standard deviations and correlations or to a Cholesky factor, then construct the full matrix in the random-effect order used by the model.
Check finite entries, symmetry, positive variances, and positive definiteness. Record failed checks rather than treating them as ordinary valid draws.
A frequent warning sign is a failed Cholesky factorization after sampling raw covariance elements. The cause may be an invalid draw, a scale mismatch, an incorrect ordering of random effects, or rounding near the positive-definite boundary. Diagnosing the cause is preferable to silently replacing the matrix.
Choosing a Parameterization and Understanding Its Limits
| Approach | Strength | Main caution |
|---|---|---|
| Draw raw variances and covariances independently | Simple to implement | Does not enforce positive variances or positive definiteness |
| Draw standard deviations and correlations on transformed scales | Directly interpretable; readily enforces positive standard deviations and admissible two-effect correlations | In dimensions above two, independently admissible pairwise correlations need not form a positive-definite matrix |
| Draw a Cholesky factor and set \(\Omega=LL^\mathsf{T}\) | Guarantees a positive-definite matrix when the diagonal of \(L\) is positive | The uncertainty distribution is on factor entries, which need not have an intuitive interpretation as marginal variances and correlations |
| Retain only valid draws or adjust failed matrices | May allow an existing raw-scale procedure to continue | Rejection changes the sampled distribution when invalid draws have nonzero probability; adjustment also changes values and can distort uncertainty |
No parameterization makes a poor uncertainty model correct. For example, a normal approximation for transformed parameters can be convenient, but it may be unreliable when estimates are near a boundary or the likelihood is strongly asymmetric. Bootstrap or posterior draws can carry different information, but their covariance matrices should still be checked and consistently ordered. If a model supports a zero variance or singular covariance, that boundary case should be handled deliberately rather than created accidentally by numerical repair.
The practical rule is to preserve the distinction between uncertainty in \(\Omega\) and the random effects generated conditional on \(\Omega\), use a parameterization compatible with the intended uncertainty representation, and validate the resulting matrices. A valid matrix is necessary for downstream simulation, but it is not evidence that the fitted model or the uncertainty distribution is otherwise adequate.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A symmetric two-effect matrix has variances \(0.09\) and \(0.04\), with covariance \(0.050\). Is it positive definite? Show the determinant calculation.
- Why does drawing positive standard deviations and a correlation strictly between \(-1\) and \(1\) guarantee a valid two-effect covariance matrix?
- For a three-effect covariance draw, all three diagonal entries are positive and every pairwise correlation is between \(-1\) and \(1\). Is that sufficient to establish positive definiteness?
- What does a failed Cholesky factorization indicate, and why should failed matrices not automatically be replaced by nearby matrices without recording the change?
Solutions. (1) The determinant is \((0.09)(0.04)-(0.050)^2=0.0036-0.0025=0.0011>0\). With positive variances and symmetry, the matrix is positive definite. (2) Its determinant is \(\sigma_1^2\sigma_2^2(1-\rho_{12}^2)\), which is positive when both standard deviations are positive and \(|\rho_{12}|<1\); symmetry and positive diagonal entries also hold by construction. (3) No. Pairwise correlations in range do not ensure the full matrix is positive definite; check the full matrix, for example by Cholesky factorization or its eigenvalues. (4) It indicates that the matrix is not positive definite in the numerical calculation, or is sufficiently close to a boundary that numerical precision may matter. Replacing it changes the sampled value and potentially the uncertainty distribution, so the failure and any adjustment must be recorded and assessed.