Eigenvectors And Weakly Estimated Parameter Combinations

Why Parameter Combinations Matter

In a population PK model, estimates of two parameters can be strongly related. The likelihood may constrain each parameter to a narrow range when the other is held fixed, yet allow a long, shallow ridge in which both change together. In that case, separate standard errors do not fully describe the model’s precision: one particular combination of parameters may be much less well determined than the individual summaries suggest.

Eigenvectors of an information matrix provide a way to identify such directions. Each eigenvector describes a direction in parameter space, while its eigenvalue measures the local information in that direction. A small eigenvalue signals that movement along the associated combination changes the likelihood relatively little. This is a local precision diagnostic, not by itself proof that the model is structurally nonidentifiable.

Information in Different Directions

Let \(\boldsymbol{\theta}\) denote the vector of estimated parameters, and let \(\hat{\boldsymbol{\theta}}\) be an interior maximum-likelihood estimate. The observed information matrix is the negative Hessian of the log-likelihood evaluated at that estimate. The expected Fisher information is the expectation of the corresponding negative Hessian under the model. Both describe local curvature, though they need not be numerically identical in a particular data set.

Eigenvectors are most interpretable after parameters have been scaled. For example, let \(D\) be a diagonal matrix of chosen parameter scales and define dimensionless coordinates \(\mathbf{z}\) by \(\boldsymbol{\theta}=\boldsymbol{\theta}_{\mathrm{ref}}+D\mathbf{z}\). The information in these coordinates is \(I_z=D^\mathsf{T}I_\theta D\). Eigenvectors of \(I_z\) describe combinations of scaled parameter changes, not unscaled changes in parameters with different units. Scaling affects eigenvalues and eigenvectors, so state the coordinates used when interpreting them.

For log-transformed positive parameters, useful coordinates can be changes in log parameters. For instance, a direction involving \(\log(CL)\) and \(\log(V)\) describes relative changes in clearance and volume. A direction with similar signs changes them together; one with opposite signs changes one upward while changing the other downward.

Why a Small Eigenvalue Means Weak Precision

Near a well-behaved maximum, the log-likelihood can be approximated by a quadratic:

Suppose \(I\) is symmetric and positive definite, with orthonormal eigenvectors \(\mathbf{v}_j\) and eigenvalues \(\lambda_j\). Write a displacement as \(\boldsymbol{\delta}=t\mathbf{v}_j\). Substitution gives the approximate log-likelihood change in that direction:

Thus, for the same size of displacement \(t\), a smaller \(\lambda_j\) gives a smaller drop in log-likelihood. Under the usual local asymptotic approximation, the covariance matrix of the estimate is \(I^{-1}\). The variance of the estimate projected onto \(\mathbf{v}_j\) is therefore

This relationship explains the directional interpretation: small information means large approximate variance along that direction. It relies on a locally quadratic likelihood and a valid covariance approximation. Near boundaries, in small samples, or with a markedly non-quadratic likelihood, the inverse information may not adequately represent uncertainty.

Reading an Eigenvector as a PK Combination

An eigenvector’s entries are weights for a linear combination of the coordinates. If \(\mathbf{v}=(v_1,\ldots,v_p)^\mathsf{T}\), the projected quantity is \(\mathbf{v}^\mathsf{T}\mathbf{z}\). For example, a direction proportional to \((1,1,-1)\) in coordinates for log clearance, log volume, and log absorption rate describes joint movement in which clearance and volume move together while absorption rate moves in the opposite direction. It is a local direction of parameter change, not necessarily a literal biological mechanism.

The sign of an eigenvector is arbitrary: \(\mathbf{v}\) and \(-\mathbf{v}\) represent the same direction and have the same eigenvalue. If eigenvalues are equal or nearly equal, individual eigenvectors in that group may not be uniquely determined; interpret the associated subspace rather than assigning strong meaning to a particular basis vector. Also, the entries describe weights in the coordinates analyzed. A combination of standardized log parameters does not have the same interpretation as a combination of raw parameters measured in different units.

Diagnostics, Alternatives, and Limitations

Inspect eigenvalues and eigenvectors together. A very small eigenvalue relative to the others suggests a weak direction, but a threshold is not universal: conclusions depend on parameter scaling, the likelihood approximation, and the inferential purpose. An eigenvalue near zero can also indicate a singular or numerically unstable information matrix; in that case an ordinary inverse may not exist or may be unreliable.

Useful checks include confirming that the matrix is symmetric and positive definite when that is expected, checking that the optimizer converged to an interior solution, and verifying that the eigenvectors are expressed in clearly stated coordinates. Compare the implied weak combination with parameter correlations, profile likelihoods, and sensitivity to reasonable changes in the model or data. A nearly flat direction in a profile or large changes under small perturbations can support the concern; disagreement between diagnostics calls for investigation rather than choosing one summary uncritically.

Eigenanalysis complements, rather than replaces, other precision summaries. Marginal standard errors describe uncertainty in individual coordinates, and parameter correlations show pairwise linear association in the approximate covariance matrix. A condition number summarizes the ratio of largest to smallest eigenvalue for a positive definite matrix, but does not identify the weak parameter combination. Profile likelihood examines likelihood shape beyond the local quadratic approximation, while simulation-based checks can show how estimates behave under repeated data generation. Each approach answers a related but distinct question.

A common mistake is to see similar marginal standard errors and conclude that every important parameter combination is well estimated. The first worked example shows why this is wrong: both marginal standard errors are equal, yet uncertainty in the contrast is much larger than uncertainty in the sum. The correction is to inspect directional information, while also checking scale and the validity of the local approximation.

Eigenanalysis has limits. It is local to the estimate, depends on the chosen coordinates, and can be unstable when eigenvalues are close. In nonlinear mixed-effects models, an approximate information matrix may not capture all uncertainty associated with variance components, boundaries, or non-Gaussian likelihood shapes. Treat a weak eigenvector as a diagnostic clue about the fitted model and data, not as a standalone declaration of identifiability or a causal explanation.