Information About Residual-Error Parameters

Why residual-error parameters need their own information calculation

A population design must provide information not only about typical pharmacokinetic or pharmacodynamic parameters, but also about the sources of variability in the observations. Residual-error parameters describe variation remaining at the observation level after accounting for the model prediction and random effects. Their information depends on how the assumed residual variance changes across observations and on how that change can be distinguished from between-subject variability.

The key distinction is the covariance structure each parameter changes. A random-effect variance changes covariance through the subject-level sensitivity to random effects. An additive residual variance adds observation-level variance, while a proportional residual variance adds variance that grows with the squared predicted response. These contributions can overlap, so information about one variance component need not be separable from information about another.

Model and assumptions

Consider subject \(i\)'s vector of observations \(Y_i\), approximated by a multivariate normal distribution with mean \(\mu_i\) and covariance \(V_i\). Under a first-order population approximation, write \(V_i=J_i\Omega J_i^\mathsf{T}+R_i\): \(J_i\) is the sensitivity matrix to random effects, \(\Omega\) is their covariance matrix, and \(R_i\) is the residual-error covariance matrix. This tutorial uses independent residual errors, so \(R_i\) is diagonal.

For observation \(j\), let \(f_{ij}\) be the predicted response at the typical parameter values, in the same units as the measured response. Use the variance-component parameterization \(\lambda_a=\sigma_a^2\) for additive error and \(\lambda_p=\sigma_p^2\) for proportional error. The residual variance for that observation is \(\lambda_a+\lambda_p f_{ij}^2\). Thus, in the stated approximation, \(R_i=\operatorname{diag}(\lambda_a+\lambda_p f_{i1}^2,\ldots,\lambda_a+\lambda_p f_{in_i}^2)\).

The calculation below assumes the Gaussian approximation is adequate, \(V_i\) is positive definite, the observations have the specified residual structure, and the model predictions used in \(R_i\) are treated as fixed when differentiating with respect to residual parameters. These assumptions define the approximation; they do not establish that a particular residual model fits actual data. If the implementation defines proportional error using a different scale or parameterization, the covariance derivatives must be changed accordingly.

Covariance derivatives and the information formula

For a Gaussian model with mean \(\mu_i\) and covariance \(V_i\), the expected Fisher information for parameters that affect the covariance includes \(\tfrac12\operatorname{tr}(V_i^{-1}V_{i,a}V_i^{-1}V_{i,b})\), where \(V_{i,a}\) denotes the derivative of \(V_i\) with respect to parameter \(a\). For residual parameters in this model, the mean has no direct dependence on \(\lambda_a\) or \(\lambda_p\). Their covariance derivatives are \(V_{i,\lambda_a}=I\) and \(V_{i,\lambda_p}=\operatorname{diag}(f_{i1}^2,\ldots,f_{in_i}^2)\).

Information accumulates across independent subjects by summing their contributions. For residual parameters \(a\) and \(b\), the resulting expression is

This formula accounts for the full subject-level covariance. In particular, \(V_i^{-1}\) is generally not diagonal when random effects induce correlation among a subject's observations. Ignoring that correlation and calculating as though all observations were independent can therefore give a different information value.

Why the covariance formula holds

For one subject, let \(r_i=Y_i-\mu_i\). Ignoring terms constant in the parameters, the log likelihood is \(-\tfrac12\{\log|V_i|+r_i^\mathsf{T}V_i^{-1}r_i\}\). For a covariance parameter \(a\), differentiation gives the score

To see the expected information, whiten the residual by writing \(r_i=V_i^{1/2}z\), where \(z\sim N(0,I)\). The score becomes a centered quadratic form in \(z\). For symmetric matrices \(A\) and \(B\), the covariance of \(z^\mathsf{T}Az\) and \(z^\mathsf{T}Bz\) is \(2\operatorname{tr}(AB)\); this follows by expanding the products and using \(E(z_kz_lz_mz_n)=\delta_{kl}\delta_{mn}+\delta_{km}\delta_{ln}+\delta_{kn}\delta_{lm}\). Applying this identity to the two centered scores gives \(\operatorname{Cov}(U_{i,a},U_{i,b})=\tfrac12\operatorname{tr}(V_i^{-1}V_{i,a}V_i^{-1}V_{i,b})\), the stated Fisher information.

Separating residual information from random-effect variance information

The same formula applies to random-effect variance parameters, but their covariance derivatives differ. For example, with a random intercept of variance \(\tau^2\), \(V_i\) includes \(\tau^2\mathbf{1}\mathbf{1}^\mathsf{T}\), so \(V_{i,\tau^2}=\mathbf{1}\mathbf{1}^\mathsf{T}\). By contrast, the additive residual derivative is \(I\), and the proportional residual derivative is diagonal with entries \(f_{ij}^2\). The matrices describe different patterns of covariance change: a random intercept links observations within the same subject, whereas independent residual errors add variance along the observation-level diagonal.

Different derivative patterns can help distinguish the sources of variability, but they need not be orthogonal under the Fisher information metric. The off-diagonal information between parameters \(a\) and \(b\) is the same trace expression with \(V_{i,a}\) and \(V_{i,b}\); it can be nonzero when their effects on covariance overlap. The next example makes this distinction explicit.

Diagnostics, alternatives, and limitations

A useful diagnostic is to inspect both the covariance derivative matrices and the resulting information matrix. Identical or nearly proportional derivatives indicate exact or weak local confounding. A singular or poorly conditioned information matrix warns that the design cannot reliably separate the corresponding parameters, even if its diagonal entries appear large. Also check that \(V_i\) is positive definite and that predictions span levels that make the proposed residual components respond differently.

The Gaussian population Fisher calculation is a local approximation. Its conclusions depend on the assumed model, parameter values, design, and approximation to the marginal subject-level distribution. Strong nonlinearities, heavy-tailed or correlated residual errors, outliers, or residual variance that depends on random effects can make the assumed \(R_i\) inadequate. In such cases, a model with the appropriate residual structure or a more faithful likelihood calculation may be needed.

The main alternative is to calculate information from a fuller marginal likelihood rather than the Gaussian population approximation. That can better represent nonlinear integration over random effects, but may require more computation and numerical integration. Within the Gaussian approximation, the derivative-matrix formula is preferable to treating the residual and random-effect variances as separate scalar calculations: it retains their cross-information and the covariance among repeated observations. Neither approach removes structural confounding in the design.

Common pitfall

• Wrong reasoning: “There are two residual parameters and two observations, so both parameters must be identifiable.” The count of observations alone does not establish identifiability.
• Correction: Inspect whether the covariance derivatives are linearly independent in the information calculation. At one prediction level, the proportional derivative is a constant multiple of the additive derivative, even with many repeated observations at that same level. Variation in prediction levels, or another source of distinct covariance structure, is needed to separate their effects.
• Wrong reasoning: “A random-effect variance and a residual variance both increase variance, so their information is interchangeable.”
• Correction: Compare the full derivative matrices. A random intercept adds shared covariance across observations; independent residual error adds diagonal variance. Their cross-information may be nonzero, but they are not the same covariance contribution.