Mean and Covariance Contributions to Information

Why separate the two sources of information?

In a Gaussian population model, a design can be informative because a parameter changes the model's predicted values, because it changes the variability around those values, or both. These are distinct mechanisms. A sampling time may make a predicted concentration highly sensitive to a typical parameter value, while another design feature may make between-subject variability more visible. Looking at only the predicted mean can therefore miss information carried by the spread or correlation of observations.

The Gaussian Fisher matrix makes this distinction explicit. Its mean term quantifies changes in the expected observation relative to its covariance; its covariance term quantifies changes in the covariance itself. The two terms add, but they need not be equally important, and their relative sizes can depend on both the parameter and the design.

Interpretation and conditions

The mean contribution measures how large the parameter-induced change in the mean is after accounting for the covariance of the observation. In particular, \(V^{-1}\) weights directions with less variability more heavily. A mean shift in a low-variance direction can be more informative than an equally sized shift in a high-variance direction.

The covariance contribution measures how distinguishable the parameter-induced change in covariance is relative to the covariance already present. It can reflect changes in marginal variances, correlations, or both. It is not simply a measure of how large the variance is: if a parameter changes the covariance very little, this contribution can be small even when the observations are highly variable.

The decomposition applies when the Gaussian model is correctly specified for the likelihood being used, \(V(\theta)\) is symmetric positive definite in the parameter region of interest, and the mean and covariance are differentiable with respect to the parameters. The standard Fisher-information interpretation also relies on regularity conditions that allow differentiation of the likelihood and interchange of differentiation and expectation. In population design, \(m\) and \(V\) may be approximations to the population mean and covariance; the resulting decomposition is then information under that Gaussian approximation, not a guarantee that the original nonlinear mixed-effects likelihood is exactly Gaussian.

Why the two terms add

A short derivation explains why the terms can be interpreted separately. Let \(r=Y-m(\theta)\). The Gaussian log likelihood, apart from a constant, is \(-\tfrac12\log|V|-\tfrac12 r^\mathsf{T}V^{-1}r\). Differentiating with respect to \(\theta_a\) gives the score

The first part is linear in the centered observation and arises from changes in the mean. The second part is a centered quadratic form and arises from changes in covariance. To see their expected products, define \(Z=V^{-1/2}r\), which has distribution \(N(0,I_q)\), \(u_a=V^{-1/2}m_a\), and \(A_a=V^{-1/2}V_aV^{-1/2}\). Then

The expected product of the two linear parts is \(u_a^\mathsf{T}u_b=m_a^\mathsf{T}V^{-1}m_b\). Their cross-products with the centered quadratic parts have expectation zero: each is a centered Gaussian third moment, which is zero. For a standard Gaussian vector and symmetric matrices, the covariance of the two centered quadratic forms is \(2\operatorname{tr}(A_aA_b)\). The factors of one half in the scores therefore give \(\tfrac12\operatorname{tr}(A_aA_b)\), equal to the covariance term in the definition. This establishes the decomposition of the expected score product.

Each contribution is positive semidefinite as a matrix. For any parameter-direction vector \(c\), let \(m_c\) and \(V_c\) denote the directional derivatives of the mean and covariance along \(c\). Then

Thus neither mechanism subtracts information in a parameter direction. The matrix may nevertheless have off-diagonal entries: those describe information shared between parameter components, not a negative contribution from one mechanism.

How to decide which contribution is primary

For a single parameter, compare \(I^{\mathrm{mean}}_{aa}\) and \(I^{\mathrm{cov}}_{aa}\) at the same nominal parameter values and design. Their ratio describes the relative contributions to local scalar information for that parameter. For a parameter combination \(c\), compare \(c^\mathsf{T}I^{\mathrm{mean}}c\) with \(c^\mathsf{T}I^{\mathrm{cov}}c\). This is more informative than comparing the traces of the matrices, which can depend on the units and scaling of different parameters.

The directional form also gives a geometric interpretation. The mean contribution is the squared length of the whitened mean change, \(V^{-1/2}m_c\). The covariance contribution is half the sum of squared eigenvalues of the whitened covariance change, \(V^{-1/2}V_cV^{-1/2}\). If those eigenvalues are all small, the covariance changes little relative to the existing variability. If they are large, the covariance may be a strong source of information.

These are local quantities: the derivatives are evaluated at specified parameter values. A conclusion such as “information is mainly from the mean” can change at another nominal value, another design, or another parameter direction. The decomposition also does not imply that one term is always more reliable. If a Gaussian approximation represents the population distribution poorly, a large calculated covariance contribution may be an artifact of that approximation.

Diagnostics, alternatives, and limitations

A useful check is to calculate the mean and covariance derivatives independently, then verify that each resulting contribution is symmetric and positive semidefinite up to numerical error. For any direction \(c\), the formulas above require both quadratic contributions to be nonnegative. A materially negative value often signals an implementation error, an incorrectly computed derivative, or a covariance matrix that is not valid at the evaluation point.

Check that \(V\) is positive definite before using \(V^{-1}\), and inspect its conditioning. A nearly singular covariance matrix can make the inverse and the resulting information highly sensitive to small numerical errors. Also check derivative calculations by finite differences or another independent method, using parameter-appropriate step sizes. In particular, covariance derivatives must preserve symmetry, and the trace term must use the full matrix products in the stated order.

A mean-only calculation treats the covariance as fixed with respect to the parameters. It is appropriate only when that is the intended model or when the covariance dependence is being deliberately omitted as an approximation. A covariance-only calculation similarly ignores mean sensitivity and is not a general substitute for the full matrix. The full Gaussian Fisher matrix includes both terms. If the Gaussian approximation itself is questionable, a more faithful likelihood-based or simulation-based assessment may be needed; the decomposition does not correct misspecification.

Finally, “primary” is a local comparison, not a causal explanation or a universal ranking of design quality. Information depends on the chosen parameterization, nominal values, and design. In particular, comparing entries for differently scaled parameters without considering units can mislead. For a fixed parameter, or a clearly defined parameter direction, the two nonnegative contributions provide a principled comparison.