Reading Parameter Correlations in Candidate Designs

Why Read the Whole Covariance Matrix?

A design criterion can compress a complicated estimation problem into one number. That is useful for searching over candidate sampling schedules, but it can hide which parameters are estimated precisely, which remain uncertain, and whether two parameter estimates move together. Reading the covariance matrix makes those consequences visible. Two schedules with similar determinant criteria can have different marginal standard errors and very different uncertainty for a parameter combination that matters to a PK decision.

The discussion here concerns local individual-model precision: it describes uncertainty near specified nominal parameter values under a specified model and residual error assumption. It is not a guarantee about finite-sample performance or about model misspecification. The earlier material on identifiability diagnostics and D-, A-, and c-optimal design supplies useful context: the determinant summarizes joint uncertainty, whereas other summaries can emphasize marginal or target-specific uncertainty.

Covariance, Correlation, and Marginal Standard Error

Let \(\hat{\theta}\) be an estimator of an individual parameter vector \(\theta=(\theta_1,\ldots,\theta_p)^\mathsf{T}\). Its covariance matrix, denoted \(\Sigma\), has the variance of each estimate on its diagonal and covariances between pairs of estimates off the diagonal. In a local design calculation, \(\Sigma\) is commonly approximated by the inverse Fisher information matrix, when that inverse exists.

A marginal standard error describes uncertainty in one parameter estimate without conditioning on the other estimates. A correlation is dimensionless and describes the direction and strength of linear co-movement in the joint estimation uncertainty. A large positive correlation means that upward deviations in one estimate tend to accompany upward deviations in the other under the local approximation; a large negative correlation means they tend to move in opposite directions. Neither statement, by itself, says whether either parameter has a small or large marginal standard error.

Covariances have units equal to the product of the units of their two parameters, whereas variances have squared parameter units. Correlations have no units. Standard errors are on the scale of the parameters used in the calculation. Consequently, always identify the parameterization: an SE for log clearance is not an SE in litres per hour. A nonlinear reparameterization can change the covariance approximation and its marginal SEs.

Assumptions Behind the Local Covariance

For a common approximation, observations have a specified mean model \(f_i(\theta)\), independent errors with known variances \(\sigma_i^2\), and a locally adequate linear approximation around the nominal parameter value. Write \(J\) for the matrix whose \(i\)th row is the gradient of \(f_i(\theta)\) with respect to \(\theta\), evaluated at that value, and \(W=\operatorname{diag}(1/\sigma_i^2)\). The local Fisher information and covariance approximation are

This expression requires \(F\) to be nonsingular for an ordinary inverse-based covariance. Its use also depends on the assumed mean model, error variances, and local approximation being reasonable. If residual variance is estimated rather than known, its estimation may affect the covariance calculation. More generally, a covariance matrix obtained from a fitted model can use a different likelihood-based or robust approximation; its entries should be interpreted according to the calculation that produced it.

The inverse information connects design sensitivity to uncertainty: observations that add information in a parameter direction tend to reduce variance in that direction. But inspecting only the determinant of \(F\) or \(\Sigma\) does not show how uncertainty is allocated among the named parameters or their combinations.

Why Similar Determinants Can Hide Different Precision

For two parameters, write their covariance matrix as

Here \(\sigma_1\) and \(\sigma_2\) are marginal standard errors and \(\rho\) is their correlation. Direct calculation gives

Thus the determinant depends jointly on both marginal SEs and their correlation. A fixed determinant does not fix either marginal SE individually. In two dimensions, the area of a covariance uncertainty ellipse is proportional to \(\sqrt{\det(\Sigma)}\); equal determinants therefore summarize equal ellipse area under the same parameter scaling, not identical shape or orientation. For an invertible information matrix, \(\det(\Sigma)=1/\det(F)\), so maximizing information determinant is equivalent to minimizing covariance determinant. Neither operation alone identifies the uncertainty of every parameter or target.

Uncertainty in Parameter Combinations

Individual parameters are not always the only quantities of interest. For a fixed vector \(c\), the variance of the linear combination \(c^\mathsf{T}\hat{\theta}\) is \(c^\mathsf{T}\Sigma c\). For two parameters, the sum and difference variances are

These expressions show why covariance matters: a positive covariance increases uncertainty in the sum but reduces it in the difference. A negative covariance has the opposite effect. The relevant combination depends on the scientific target and the parameterization; a high correlation is not automatically harmful or beneficial for every target.

Eigenvalues, Near Singularity, and Directional Precision

The eigenvectors of a covariance matrix describe orthogonal directions in parameter space, and their corresponding eigenvalues give the variances along those directions. A large covariance eigenvalue indicates a direction of high uncertainty; a small eigenvalue indicates a direction of low uncertainty. This distinction is important when diagnosing a nearly singular covariance matrix: near singularity means at least one eigenvalue is close to zero, not that every direction is poorly estimated.

For an invertible covariance matrix, the eigenvalues of \(F=\Sigma^{-1}\) are the reciprocals of those of \(\Sigma\), with the same eigenvectors. Accordingly, weak precision corresponds to a small information eigenvalue and a large covariance eigenvalue. Inspecting eigenvalues together with eigenvectors identifies which parameter combination is uncertain or precise; the diagonal alone can obscure these directions.

How to Compare Candidate Schedules

A practical comparison should connect each matrix summary to the question being asked. Begin with the parameter scale and units, then inspect marginal SEs and correlations, and finally calculate uncertainty for any scientifically relevant combination. Determinants remain useful as a joint-volume summary, but they are not a substitute for these targeted checks.

This reading complements rather than replaces alternative design summaries. A determinant-based comparison emphasizes total joint uncertainty volume; a trace-based summary of covariance emphasizes aggregate marginal variance on the stated scale; and a c-optimal comparison targets the variance of a specified linear combination. Rankings may differ because the summaries answer different questions. When the intended decision depends on a particular parameter or combination, report that target-specific uncertainty rather than relying on a determinant ranking alone.

Diagnostics, Limitations, and a Common Pitfall

Check that a reported covariance matrix is symmetric and has nonnegative variances. For a two-parameter positive-definite matrix, the diagonal entries must be positive and \(\det(\Sigma)>0\), equivalently \(|\rho|<1\). Small numerical asymmetries may arise from computation, but large asymmetries or impossible correlations suggest a calculation or reporting problem. Large correlations, very different marginal SEs, and sensitivity of eigenvalues to numerical precision warrant closer inspection of the model and design; they do not, by themselves, establish a particular cause.

The local covariance is conditional on the model and assumptions used to calculate it. It does not account automatically for model misspecification, uncertainty in nominal values, or all nonlinear and finite-sample behavior. A matrix from one parameterization should not be compared mechanically with one from another. Likewise, equal determinants can only be interpreted as equal volume summaries when the parameter scales and ordering are comparable.

Common mistake: “The correlation is close to one, so both parameters must have large standard errors.” This is incorrect: correlation describes standardized co-movement, while marginal SEs come from the diagonal entries. The correction is to report both marginal SEs and correlations, and to examine relevant combinations or covariance eigen-directions. A related error is to call a nearly singular covariance matrix uniformly imprecise. Near singularity signals a very small variance eigenvalue; weak precision instead lies along a direction with a large covariance eigenvalue.