Why Positive Definiteness Matters
In a population PK model, a subject’s random effects describe deviations from typical parameter values. Their covariance matrix, denoted by \(\Omega\), records the variances of those effects and the covariances between them. Its entries cannot be chosen independently: together they must describe a mathematically possible joint distribution.
For example, random effects on clearance and volume may each have a plausible variance, while a proposed covariance between them is too large to be possible. Positive definiteness is the joint constraint that prevents such incompatible combinations. It also matters computationally: the usual nonsingular multivariate normal model for random effects requires a positive-definite covariance matrix.
To see why a covariance matrix cannot fail the semidefinite condition, let \(\eta\) be a vector of centered random effects with covariance matrix \(\Omega\). For any fixed vector \(a\), the variance of the linear combination \(a^\mathsf{T}\eta\) is
A negative value would claim that a random quantity has negative variance, which is impossible. Strict positivity for all nonzero \(a\) additionally rules out a linear combination of effects having exactly zero variance. Thus positive definiteness is stronger than the basic requirement for a covariance matrix: it excludes exact linear dependence among the effects.
Constraints for Two Random Effects
For two centered random effects, such as \(\eta_{\mathrm{CL}}\) and \(\eta_{\mathrm{V}}\), write their covariance matrix as
The diagonal entries are variances; the off-diagonal entry is a covariance. If both variances are positive, the correlation is \(\rho_{\mathrm{CL,V}}=\omega_{\mathrm{CL,V}}/(\omega_{\mathrm{CL}}\omega_{\mathrm{V}})\), where the standard deviations \(\omega_{\mathrm{CL}}\) and \(\omega_{\mathrm{V}}\) are positive. For this matrix to be positive definite, its first diagonal entry and determinant must be positive:
Dividing the determinant inequality by the positive quantity \(\omega_{\mathrm{CL}}^2\omega_{\mathrm{V}}^2\) gives \(1-\rho_{\mathrm{CL,V}}^2>0\), or \(-1<\rho_{\mathrm{CL,V}}<1\). Equivalently, the covariance must obey \[ |\omega_{\mathrm{CL,V}}|<\omega_{\mathrm{CL}}\omega_{\mathrm{V}}. \] The weak inequality holds for a positive-semidefinite covariance matrix, with equality representing a singular boundary.
Worked Example: A Covariance That Is Too Large
Suppose, hypothetically, a model assigns variance \(0.04\) to the clearance random effect and variance \(0.09\) to the volume random effect. A proposed covariance is \(0.07\). Check whether the proposed entries form a positive-definite covariance matrix.
The standard deviations are \(\sqrt{0.04}=0.20\) and \(\sqrt{0.09}=0.30\), so the largest allowable absolute covariance for a positive-semidefinite matrix is their product: \(0.20\times 0.30=0.06\). The proposed covariance \(0.07\) exceeds this bound. The corresponding correlation would be \[ \rho_{\mathrm{CL,V}}=\frac{0.07}{0.20\times 0.30}=1.167, \] which is outside the allowable range.
The determinant confirms the failure: \[ (0.04)(0.09)-(0.07)^2=0.0036-0.0049=-0.0013. \] Because the determinant is negative, this matrix is not positive semidefinite and cannot be a covariance matrix. The problem is not that the covariance is “very strong”; it is that these variances and covariance are mathematically incompatible.
Boundary Correlations and Singular Matrices
If the covariance reaches either bound, then \(\rho=1\) or \(\rho=-1\), and the determinant is zero. For two effects with positive variances, the boundary has a direct interpretation. At \(\rho=1\), the centered effects have an exact positive linear relationship; at \(\rho=-1\), they have an exact negative linear relationship. In either case, one effect is a fixed multiple of the other with probability one. There is no random variation in the corresponding linear combination.
This boundary is allowed by the weak semidefinite condition, but it is not positive definite. The joint distribution is confined to a line rather than occupying the full two-dimensional random-effect space. Consequently, a nonsingular multivariate normal density does not apply at the boundary. A model may sometimes be considered in a degenerate limiting sense, but it is different from the usual full-rank random-effect model.
Worked Example: A Correlation Near the Negative Boundary
Consider a hypothetical two-effect covariance matrix with variances \(0.04\) and \(0.09\), and correlation \(-0.98\). The standard deviations are \(0.20\) and \(0.30\), so the covariance is \[ (-0.98)(0.20)(0.30)=-0.0588. \]
The resulting matrix and determinant are \[ \Omega= \begin{pmatrix} 0.04 & -0.0588\\ -0.0588 & 0.09 \end{pmatrix}, \qquad \det(\Omega)=(0.04)(0.09)-(-0.0588)^2 =0.0036-0.00345744=0.00014256. \] The diagonal entries and determinant are positive, so this matrix is positive definite, but its determinant is small. The centered clearance and volume effects are strongly negatively associated in this hypothetical model: subjects with higher-than-typical clearance effects tend to have lower-than-typical volume effects. The correlation is not exactly \(-1\), so the effects are not an exact linear function of one another.
A value near the boundary can nevertheless be difficult to estimate reliably. Small changes in the fitted covariance may produce large changes in the implied correlation, and numerical calculations involving the matrix can become sensitive. The sign and magnitude should therefore be interpreted alongside uncertainty and model diagnostics, not as proof of an exact biological relationship.
The same principle extends beyond two effects. Checking that every pairwise correlation lies between \(-1\) and \(1\) is necessary, but not sufficient, to guarantee that the full matrix is positive definite. The entries must satisfy joint constraints as well.
Worked Example: Pairwise-Valid Correlations That Are Jointly Impossible
Suppose a hypothetical model has three standardized random effects, with correlation matrix \[ R= \begin{pmatrix} 1 & 0.9 & 0.9\\ 0.9 & 1 & -0.9\\ 0.9 & -0.9 & 1 \end{pmatrix}. \] Each individual correlation is between \(-1\) and \(1\), so each pair alone could have a valid two-effect correlation matrix. But the determinant of the full matrix is \[ \det(R)=1+2(0.9)(0.9)(-0.9) -\left(0.9^2+0.9^2+(-0.9)^2\right) =1-1.458-2.43=-2.888. \]
A negative determinant means the matrix is not positive semidefinite, and therefore cannot be a correlation matrix for three random effects. The three proposed pairwise relationships conflict when considered together. This demonstrates why validating correlations one pair at a time does not establish that a full covariance structure is valid. Rescaling these effects by positive standard deviations would not fix the problem: such rescaling preserves whether the matrix is positive definite.
Checking a Covariance Matrix in Practice
For an estimated \(\Omega\), positive definiteness can be checked by examining its eigenvalues: all must be positive. Equivalently, a symmetric matrix is positive definite if its leading principal minors are positive. For small matrices, determinants and direct algebra can make the constraint transparent; for larger matrices, numerical linear algebra is more practical. A Cholesky factorization is another common check and parameterization: a positive-definite matrix has a factorization \(\Omega=LL^\mathsf{T}\) with a nonsingular lower-triangular \(L\).
A reliable check should use the full matrix, not just its diagonal entries or its pairwise correlations. Also check that the matrix is symmetric and that the reported covariances use the same random-effect scale as the reported variances. Covariances have product units—for example, clearance-effect units times volume-effect units—whereas correlations are dimensionless.
Confirm that diagonal entries are variances, are nonnegative, and that the covariance matrix is symmetric.
Use an eigenvalue, principal-minor, or Cholesky check on the complete matrix. Do not infer validity from pairwise correlation bounds alone.
Look for a very small minimum eigenvalue, a near-zero determinant, or a correlation close to \(1\) or \(-1\). These indicate near-singularity, even if the matrix is technically positive definite.
Ask whether the estimated dependence is supported by the data and whether the model is stable, rather than treating an extreme correlation as automatically meaningful.
Near-boundary estimates can arise because the data contain limited information about separate random effects, because the model is over-parameterized for the available design, or because the fitted structure is a poor description of the data. Symptoms can include failed matrix factorizations, unstable optimization, large changes across starting values, or estimates that repeatedly approach the correlation bounds. These are prompts to investigate, not a diagnosis on their own.
Modeling Choices and Their Limits
A full covariance structure allows correlations among all modeled effects, but it also requires the complete matrix to obey the positive-definiteness constraint. A diagonal structure sets off-diagonal covariances to zero; a block-diagonal structure permits dependence only within specified groups. These restrictions can reduce the number of estimated associations and avoid trying to estimate unsupported covariance terms, but they impose assumptions. They do not establish that the omitted correlations are truly zero.
The Cholesky parameterization provides a way to construct a positive-definite covariance matrix during estimation; it does not guarantee that the resulting covariance estimates will be far from the boundary or precisely estimated. Nor does mathematical validity establish scientific plausibility. A valid matrix may still encode a dependence pattern that is poorly supported or inappropriate for the PK model.
A common mistake is to see a correlation of \(1.00\) or \(-1.00\) and conclude that the data show an exceptionally strong but otherwise ordinary association. The correction is to recognize that an exact boundary value implies a singular covariance matrix and exact linear dependence for the two effects. A value merely close to a boundary is not exact dependence, but it can still signal weak information or numerical instability. Reported rounding can also conceal the distinction, so inspect unrounded estimates and the matrix diagnostics when available.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A two-effect covariance matrix has variances \(0.16\) and \(0.25\), and covariance \(0.18\). Is it positive definite?
- For two effects with positive variances, what does a correlation of exactly \(1\) imply about the covariance matrix and the centered effects?
- A three-effect correlation matrix has all three pairwise correlations within \([-1,1]\). Is that enough to establish positive definiteness? Explain.
- Why should an estimated correlation of \(-0.99\) be investigated even if the covariance matrix is positive definite?
Solutions. (1) The standard deviations are \(0.4\) and \(0.5\), so the maximum allowable absolute covariance is \(0.4\times0.5=0.20\). The determinant is \((0.16)(0.25)-(0.18)^2=0.04-0.0324=0.0076>0\); with positive diagonal entries, the matrix is positive definite. (2) The determinant is zero, so the matrix is singular and only positive semidefinite. The centered effects are exact positive multiples of one another. (3) No. Pairwise bounds are necessary but do not ensure joint compatibility; the determinant or eigenvalues of the full matrix must also be checked. (4) It is close to the singular boundary, so estimates and numerical calculations may be sensitive, and the extreme dependence may reflect limited information or model problems. It is not itself proof of exact linear dependence.