Why Parameter Combinations Matter
A fitted QSP model can reproduce observations well while leaving some parameter combinations poorly determined. For example, a change in one rate constant may be offset by a change in another, so predictions barely move even though the individual parameters change substantially. Singular-value decomposition (SVD) gives a systematic local way to find such directions: it separates parameter changes that strongly alter predicted observations from changes that have little observable effect.
The analysis is local: it describes behavior near a specified parameter estimate. It does not, by itself, establish global identifiability or guarantee that the likelihood is approximately quadratic over a large region. The sensitivity matrix introduced in the earlier material on sensitivity matrices is the starting point; here it is scaled, weighted for observation error, and decomposed into informative directions.
The scales in \(D\) should represent scientifically meaningful changes, such as a typical parameter value or a prespecified increment. They are not automatically standard errors. If a parameter is measured in different units, changing its scale changes the numerical coordinates of the analysis; the choice must therefore be reported and justified. Whitening by \(C^{-1/2}\) puts residual changes on an observation-error scale. For independent observations with standard deviations \(\sigma_i\), this amounts to dividing the \(i\)th row of \(SD\) by \(\sigma_i\).
What the Singular Values Say
Write the SVD of \(A\) as \(A=U\Sigma V^T\). The columns \(v_j\) of \(V\) are orthonormal directions in the dimensionless parameter coordinates \(z\), and the diagonal entries \(\sigma_j\) of \(\Sigma\) are nonnegative singular values, conventionally ordered from largest to smallest. For a displacement \(z=t v_j\), the local change in whitened predictions is \(Az=t\sigma_j u_j\), where \(u_j\) is the corresponding left singular vector. Thus \(\sigma_j\) measures the size of the prediction change, in observation-error units, for a unit displacement along \(v_j\).
A large singular value indicates a direction to which the observations are locally sensitive. A small singular value indicates a weakly constrained combination: movement along that direction produces little change in predictions compared with their error scale. If \(\sigma_j=0\), the local linearized model has no sensitivity in that direction. If it is merely small, information exists locally but may be insufficient for precise estimation. There is no universal cutoff separating “small” from “large”; interpretation depends on the parameter scaling, error model, and scientific precision required.
These statements use the local Gaussian or Gauss–Newton approximation, not a guarantee that the true likelihood has ellipsoidal contours everywhere. If the error covariance is known only up to an estimated common variance multiplier, that multiplier must also be included in the covariance calculation. The formula above assumes the stated \(C\) is the known observation-error covariance.
Worked Example: Finding a Weak Contrast
Suppose a hypothetical model has two scaled parameters, and its whitened, scaled sensitivity matrix at the fit is
This symmetric matrix has orthonormal right singular vectors \(v_1=(1,1)^T/\sqrt{2}\) and \(v_2=(1,-1)^T/\sqrt{2}\). Multiplication by \(A\) gives \(Av_1=4v_1\) and \(Av_2=0.2v_2\). Therefore the singular values are \(4\) and \(0.2\), respectively.
The sum direction, \(v_1\), is strongly constrained: a unit scaled displacement changes the whitened predictions by 4 units. The contrast direction, \(v_2\), is weakly constrained: the corresponding change is only 0.2 error units. Under the local covariance approximation, the standard errors of the coordinates along these directions are \(1/4=0.25\) and \(1/0.2=5\), respectively. The contrast is therefore much less precisely estimated than the sum, even though both are combinations of the same two parameters.
These are standard errors for dimensionless coordinates. To express uncertainty in physical parameter units, multiply the covariance by \(D\) on the left and \(D^T\) on the right. A numerical uncertainty statement in physical units cannot be made without specifying the parameter scales.
Why the Local Uncertainty Formula Holds
Let the residual at the fitted value be \(r_0=y-f(\hat{\theta})\), where \(y\) is the observation vector. For a small scaled displacement \(z\), linearize the predictions as \(f(\hat{\theta}+Dz)\approx f(\hat{\theta})+SDz\). The residual is then approximately \(r_0-SDz\). With the weighted least-squares objective \(Q=r^TC^{-1}r\), its local change is
The linear term must not be silently discarded. At an interior least-squares optimum, the gradient condition is \(S^TC^{-1}r_0=0\); consequently \(r_0^TC^{-1}SD=0\), and the linear term vanishes. Under this condition and the local linear approximation, \(\Delta Q(z)\approx z^TA^TAz\). For a nonlinear model this is a Gauss–Newton approximation: it uses the local sensitivity matrix and does not include all curvature terms from second derivatives of \(f\).
The covariance result can also be seen from the local linear estimation problem. Let \(\epsilon_w=C^{-1/2}\epsilon\) be the whitened observation error, with covariance equal to the identity matrix. The local change in the fitted scaled parameters is approximately
Taking its covariance and using \(\operatorname{Cov}(\epsilon_w)=I\) gives
Because \(A^TA=V\operatorname{diag}(\sigma_j^2)V^T\), inverting it gives the stated variance \(1/\sigma_j^2\) in each singular direction. This derivation requires full column rank; if a singular value is zero, the inverse does not exist.
Worked Example: Weighting and Parameter Scales
Consider two hypothetical parameters \(\theta_1\) and \(\theta_2\), with scales \(D=\operatorname{diag}(0.5,4)\). At a fitted value, suppose the sensitivity matrix and observation covariance are
The observation standard deviations are 2 and 1, so \(C^{-1/2}=\operatorname{diag}(0.5,1)\). First, \(SD=\operatorname{diag}(1,2)\); after whitening,
The singular values are 2 and 0.5. The first scaled coordinate is weakly constrained relative to the second, with standard error \(1/0.5=2\); the second has standard error \(1/2=0.5\). In scaled coordinates the covariance is \(\operatorname{diag}(4,0.25)\). Transforming back gives
Thus the local standard errors are 1 unit of \(\theta_1\) and 2 units of \(\theta_2\). The larger raw standard error for \(\theta_2\) does not contradict its stronger scaled information: the chosen scale for that parameter is 4 units, rather than 0.5.
Worked Example: The Residual Term Away From an Optimum
In a hypothetical one-dimensional whitened problem, let the residual at a trial point be \(r_0=1\), the local sensitivity be \(A=1\), and consider a scaled displacement \(z=0.1\). Under the local linear prediction approximation, the new residual is \(r_0-Az=0.9\). With \(Q=r^2\), the objective change is
The general expansion gives the same result:
Keeping only the quadratic term would incorrectly give \(+0.01\). This trial point is not a least-squares optimum, since its gradient is nonzero, so the first-order term does not vanish. The local uncertainty calculation is made at an optimum and relies on its stationarity condition; away from that point, the full first-order term must be retained.
Diagnostics, Alternatives, and Limitations
Before interpreting an SVD, check that the sensitivity matrix and covariance correspond to the fitted model, observation times, and error assumptions being analyzed. Numerical differentiation can be inaccurate if perturbations are too small, too large, or poorly scaled. Recompute sensitivities with reasonable step-size changes or an independent differentiation method when small singular values are driving an important conclusion. A singular value near numerical precision may reflect computational error rather than meaningful information.
Scaling is also a diagnostic choice, not a cosmetic detail. An unscaled matrix can make units dominate the decomposition: a parameter expressed in a smaller unit may appear to have a different sensitivity solely because of that unit. Report \(D\), the observation-error weighting, and the singular values together. A small singular value should be interpreted relative to an explicitly stated scale and scientific question, not as a universal pass/fail threshold.
SVD of \(A\) and eigendecomposition of the Fisher information matrix \(A^TA\) encode the same local directions: the eigenvectors are the right singular vectors, and the eigenvalues are \(\sigma_j^2\). SVD is often preferable numerically when the matrix is ill-conditioned, because forming \(A^TA\) squares the condition number. Neither decomposition supplies information beyond the local sensitivity and error model. Profile likelihood or other nonlinear analyses can be useful when local quadratic approximations are doubtful, but they answer a broader question about behavior away from the fitted point.
If \(A\) is rank deficient, the local covariance formula based on its inverse is unavailable; a pseudoinverse can summarize estimable directions but does not create information in the null space. Near rank deficiency, approximate standard errors may be very large and sensitive to model changes, data choices, or numerical error. Bound constraints, strong nonlinearity, multimodal likelihoods, and parameters at boundaries can all make the local Gaussian description misleading. Singular vectors identify combinations locally; they do not prove a global symmetry or establish that any individual parameter is uniquely determined.
- Dropping the residual term away from a fit. The quadratic objective change alone is justified only when the first-order term vanishes, such as at an interior least-squares optimum. Otherwise retain the linear term.
- Calling a small singular value proof of structural nonidentifiability. A small value indicates weak local information under the chosen data and scales; exact structural nonidentifiability requires a different argument.
- Reporting singular values without the scales. Their interpretation depends on \(D\) and on whitening by the observation-error covariance.
- Treating local standard errors as reliable in every direction. A nearly singular direction, active bound, or strongly curved likelihood can invalidate the local approximation.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- If a singular value is 0.25, what is the local standard error along its singular direction under the stated covariance approximation?
- Why must the scales \(D\) be reported when singular values are used to describe parameter information?
- For \(Q=r^TC^{-1}r\), what condition makes the first-order term in the local objective expansion vanish?
- What does an exactly zero singular value imply locally, and what does it not establish by itself?
Solutions. (1) The approximate standard error is \(1/0.25=4\) units of the dimensionless coordinate along that direction. (2) Scaling defines the coordinates of parameter displacement, so changing parameter units or the chosen meaningful increments changes the numerical singular values and their interpretation. (3) At an interior least-squares optimum, \(S^TC^{-1}r_0=0\), so the residual is orthogonal to the sensitivity columns in the weighted inner product and the linear term vanishes. (4) It indicates a null direction of the local linearized sensitivity matrix, so infinitesimal movement in that direction produces no first-order prediction change. By itself it does not prove global structural nonidentifiability or describe behavior far from the fitted point.