Why Sensitivity Collinearity Matters
An individual PK design is informative when its observations respond differently to changes in the parameters we want to estimate. If two parameters produce nearly the same pattern of changes across the scheduled samples, the data may struggle to distinguish them. The model can fit the observations well while allowing very different combinations of those parameters. This is practical non-identifiability: the design contains some distinguishing information in principle, but too little to estimate the parameters precisely.
Sensitivity collinearity makes this problem visible in the local behavior of a model. It complements the information-matrix calculations in the earlier material on individual Fisher information: instead of looking only at a determinant or information criterion, we examine the directions and relative sizes of the parameter effects themselves.
Constructing Sensitivity Vectors
Suppose a design collects \(m\) concentration observations, with model-predicted means \(\mu_i(\theta)\) at the scheduled times. Let \(\theta\) contain \(p\) positive PK parameters. For diagnostics, express each parameter in dimensionless log coordinates, \(\eta_j=\log(\theta_j/\theta_{j,\mathrm{ref}})\), where \(\theta_{j,\mathrm{ref}}\) is a fixed positive reference value. A one-unit change in \(\eta_j\) corresponds to multiplying \(\theta_j\) by \(e\), regardless of the parameter's physical units.
Assume the observations have independent normal errors with known standard deviations \(\sigma_i\), and that the model and error variances are evaluated at a specified nominal parameter vector. The weighted sensitivity of observation \(i\) to parameter \(j\) is \(J_{ij}=\frac{1}{\sigma_i}\frac{\partial \mu_i}{\partial \eta_j}\). Thus \(J\) is an \(m\)-by-\(p\) matrix. Its \(j\)th column, denoted \(J_j\), is the sensitivity vector for parameter \(j\): it records how that parameter changes the standardized predictions across all samples.
The angle measures similarity of direction, not strength. A parameter can have a sensitivity vector almost orthogonal to the others yet have a very small norm, meaning the observations barely respond to it. Conversely, two strong vectors can be nearly parallel and still provide weak separation of their parameters. Assess angles together with singular values and column magnitudes.
Singular Values, Rank, and Conditioning
The singular value decomposition writes \(J=U\Sigma V^\mathsf{T}\). The diagonal entries of \(\Sigma\), ordered as \(\sigma_1\geq \cdots \geq \sigma_p\geq 0\) when \(J\) has at least \(p\) columns, are the singular values. A zero smallest singular value means that some nonzero parameter-change direction produces no first-order change in any weighted prediction. A small but positive smallest singular value means that one direction is weakly informed.
The condition number is \(\kappa(J)=\sigma_1/\sigma_p\) when \(\sigma_p>0\); if \(J\) is rank deficient, its condition number is infinite. A large condition number signals that the design informs some parameter combinations far more strongly than others. It is a relative diagnostic: it does not say whether even the best-informed direction has adequate information. The absolute singular values and the implied uncertainty matter too.
Under the stated normal-error assumptions, the local Fisher information matrix is \(I=J^\mathsf{T}J\). If \(J\) has full column rank, the local covariance approximation for the estimated log parameters is \(I^{-1}\). This approximation is based on linearizing the model near the nominal parameter values. It is useful for planning and diagnosis, but it is not a guarantee that a nonlinear fit will be well behaved or that the realized uncertainty will equal the approximation.
Why Collinearity Inflates Uncertainty
The link between weak singular values and uncertainty follows directly from the local quadratic information. For any parameter-change direction \(v\), the information in that direction is \(v^\mathsf{T}Iv=v^\mathsf{T}J^\mathsf{T}Jv=\lVert Jv\rVert^2\). If \(v\) is a unit right singular vector associated with singular value \(\sigma\), then \(Jv\) has length \(\sigma\), so the directional information is \(\sigma^2\). The local variance along that direction is therefore proportional to \(1/\sigma^2\). As a singular value approaches zero, uncertainty in its associated parameter combination grows sharply.
For two normalized sensitivity columns, let their inner product be \(\rho=\cos(\phi)\). Their information matrix is \(\begin{pmatrix}1&\rho\\ \rho&1\end{pmatrix}\), with eigenvalues \(1+\rho\) and \(1-\rho\). When \(|\rho|\) approaches one, the smaller eigenvalue approaches zero. The columns then point in almost the same or opposite direction, and a parameter combination is poorly determined. This also explains why two-parameter estimates can be strongly correlated even when both parameters have visible effects on the observations.
Worked Example: Near-Parallel Sensitivities
Consider two hypothetical standardized sensitivity vectors from a two-sample individual PK design: \(J_1=(1,0)^\mathsf{T}\) and \(J_2=(0.99,\sqrt{0.0199})^\mathsf{T}\). These could represent the local effects of two log-transformed PK parameters after weighting by observation error. Both vectors have length one because \(0.99^2+0.0199=1\).
Their cosine is \(J_1^\mathsf{T}J_2=0.99\), so the angle is \(\phi=\cos^{-1}(0.99)\approx 8.11^\circ\). The design therefore produces very similar standardized prediction changes for the two parameters.
The information matrix is \(I=J^\mathsf{T}J=\begin{pmatrix}1&0.99\\0.99&1\end{pmatrix}\). Its eigenvalues are \(1.99\) and \(0.01\), so the singular values of \(J\) are \(\sqrt{1.99}\approx1.411\) and \(\sqrt{0.01}=0.1000\). Hence \(\kappa(J)=1.411/0.1000=14.11\), while the information-matrix condition number is \(199\).
The inverse information matrix is \(I^{-1}=\frac{1}{1-0.99^2}\begin{pmatrix}1&-0.99\\-0.99&1\end{pmatrix}\). Each diagonal element is \(1/0.0199\approx50.25\), giving a local standard error of \(\sqrt{50.25}\approx7.089\) log-parameter units for each parameter. The local correlation is \(-0.99\). The large marginal uncertainty and strong negative correlation reflect the difficulty of separating the parameter effects; they are not evidence that either parameter has no effect on the predictions.
Worked Diagnostics for Individual PK Designs
Worked Example: Exact Local Non-Identifiability
Suppose a hypothetical three-sample design has weighted sensitivity columns \(J_1=(1,2,3)^\mathsf{T}\) and \(J_2=(2,4,6)^\mathsf{T}\). These are exactly proportional: \(J_2=2J_1\). Their angle is \(0^\circ\), and the matrix has rank one rather than two.
For the parameter-change vector \(v=(2,-1)^\mathsf{T}\), the first-order change in weighted predictions is \(Jv=2J_1-J_2=(0,0,0)^\mathsf{T}\). Thus, moving the two log parameters in this direction leaves all three predictions unchanged to first order. The smallest singular value is zero and the condition number is infinite. Under this local model, the design cannot separately identify both parameters; adding more observations with the same proportional sensitivity pattern would not resolve the problem.
This conclusion concerns local separation at the stated nominal values. It does not establish that the parameters are structurally non-identifiable for every possible design or throughout the full nonlinear model.
Worked Example: A Design With Better Separation
Compare a hypothetical alternative schedule whose weighted sensitivity matrix has columns \(J_1=(1,0,1)^\mathsf{T}\) and \(J_2=(0.8,0.6,1)^\mathsf{T}\). Both columns have squared length \(2\), and their inner product is \(1.8\). Therefore \(\cos(\phi)=1.8/2=0.9\), giving \(\phi\approx25.84^\circ\).
The information matrix is \(I=\begin{pmatrix}2&1.8\\1.8&2\end{pmatrix}\). Its eigenvalues are \(3.8\) and \(0.2\), so the singular values are \(\sqrt{3.8}\approx1.949\) and \(\sqrt{0.2}\approx0.4472\). The condition number is about \(4.359\), substantially smaller than in the preceding near-parallel example.
The determinant is \(2(2)-1.8^2=0.76\), and \(I^{-1}=\frac{1}{0.76}\begin{pmatrix}2&-1.8\\-1.8&2\end{pmatrix}\). Each local variance is \(2/0.76\approx2.632\), and each local standard error is about \(1.622\) log-parameter units. The alternative has better local separation and lower approximate uncertainty for these coordinates. In a real design comparison, also check absolute sensitivity, the same error model, parameter scaling, and whether the nominal values are clinically plausible; a lower condition number alone does not establish that a design is adequate.
Practical Diagnostic Workflow
State the nominal parameter values, sampling times, observation-error standard deviations, and parameter coordinates. The diagnostics describe this local setup.
Form each derivative of the predicted observation with respect to the chosen parameter coordinate, then divide by the observation standard deviation. For positive PK parameters, log coordinates give a dimensionless relative parameter scale.
Calculate pairwise angles and column norms. Near-parallel columns indicate confounding; a small norm signals weak sensitivity even if its direction is distinct.
Check numerical rank, the smallest singular value, and \(\kappa(J)\), then examine the local covariance approximation when the matrix has full rank. Interpret these jointly rather than using one threshold as a pass/fail rule.
Recalculate at plausible alternative parameter values and under reasonable error assumptions. A diagnosis that changes sharply across this range is itself a warning about the local nature of the result.
Assumptions, Failure Modes, and Alternatives
These diagnostics assume a differentiable prediction model near the nominal parameters and a specified observation-error model. The weighted sensitivity matrix is meaningful only relative to those choices. If observation errors are correlated rather than independent, the correct information uses the full error covariance matrix \(R\): \(I=G^\mathsf{T}R^{-1}G\), where \(G\) is the unweighted prediction Jacobian. Equivalently, sensitivities must be whitened using a factorization of \(R\). Ignoring relevant error correlation can make the apparent information too large.
A frequent pitfall is to compute a condition number from unscaled derivatives with incompatible units, then treat it as an intrinsic property of the sampling times. Rescaling parameters can change that number. Use explicit, scientifically meaningful parameter coordinates, and report them. Log coordinates are often useful for positive PK parameters, but they do not eliminate dependence on the chosen parameterization or nominal values.
Other failure modes include numerical derivative noise, parameters near boundaries, predictions with near-zero or poorly modeled residual variance, and a strongly nonlinear response over the range of plausible estimates. Check derivative stability by comparing analytic derivatives with carefully chosen finite differences, inspect singular vectors to see which parameter combinations are weak, and compare the local approximation with profile likelihoods or simulation when needed. A full-rank sensitivity matrix is not proof of global identifiability; it only indicates first-order local separation at the evaluated point.
Angles are intuitive for pairwise comparisons, while singular values assess combinations of all parameters. Neither replaces the other: pairwise angles can miss a weak combination involving three or more parameters, and singular values alone do not identify the scientific meaning of that combination. Profile likelihood and simulation-based evaluation can reveal nonlinear or non-Gaussian behavior, but they require more computation. These approaches are complementary: sensitivity diagnostics are fast screening tools, not universal substitutes for model-based uncertainty assessment.
A large condition number is not a universal failure threshold. It is affected by parameter scaling, and a moderate condition number can coexist with weak absolute information. Conversely, a large value may be tolerable for a parameter combination that is not scientifically important. Report the smallest singular value, the parameter scaling, uncertainty estimates, and the poorly informed parameter combination rather than reducing the assessment to one number.
Check Your Understanding
Try each question, then compare your reasoning with the solutions.
- Two unit-length sensitivity vectors have inner product \(0.8\). What is their angle, and what does it suggest about the design?
- What does a zero smallest singular value imply about a local sensitivity matrix with two parameter columns?
- Why can two orthogonal sensitivity vectors still yield poor parameter estimates?
- Does a large condition number by itself prove structural non-identifiability? Explain.
Solutions. (1) The angle is \(\cos^{-1}(0.8)\approx36.87^\circ\). The effects are not orthogonal but are more separated than vectors with cosine near one; the angle alone does not establish adequate information. (2) The matrix is rank deficient, so at least one nonzero parameter-change direction has zero first-order effect on the weighted predictions. The corresponding local parameter combination cannot be separated by this design at the evaluated point. (3) Orthogonality describes direction only. If one or both columns have very small norms, the predictions change little as those parameters change, so the information and precision can still be poor. (4) No. A large condition number signals relative weakness in one or more directions under the chosen scaling, but it can depend on parameterization and is a local diagnostic. It does not establish structural non-identifiability of the full model across all possible designs and parameter values.