What Sparse Sampling Can Identify In Omega

Why Sparse Sampling Constrains Omega

In a population pharmacokinetic model, the matrix \(\Omega\) describes the covariance of individual random effects. A diagonal element is a variance; an off-diagonal element is a covariance between two random effects. A sparse concentration schedule may contain many subjects but only a few measurements per subject. The central question is therefore not simply how many observations were collected, but whether the scheduled measurements respond differently enough to the random effects to distinguish the components of \(\Omega\).

This is a question about information in the design, not a guarantee about what an estimation algorithm will return. A model may produce numerical estimates for every element of a full covariance matrix even when the data mainly support only a few combinations of those elements. Sensitivity analysis helps reveal that distinction before interpreting estimates.

Definition: Let \(\eta_i\) be the vector of individual random effects for subject \(i\), with mean zero and covariance matrix \(\Omega\). At the scheduled times, write the individual predicted concentrations as a vector \(f_i(\eta_i)\). The local sensitivity matrix \(J_i\) has one row per observation and one column per random effect; its entry is the derivative of that predicted concentration with respect to that random effect, evaluated at a reference value such as \(\eta_i=0\). A component of \(\Omega\) is locally distinguishable from other components when changing it produces a covariance pattern across the scheduled observations that cannot be reproduced by changing the other components.

For a short derivation, linearize the predictions around the reference value: \(f_i(\eta_i)\approx f_i(0)+J_i\eta_i\). If the residual errors have covariance matrix \(R_i\) and are independent of the random effects, the resulting approximate covariance of the observations is

V_i \approx J_i\Omega J_i^\mathsf{T}+R_i.

This expression shows how the schedule enters the problem: it determines \(J_i\), and thus which parts of \(\Omega\) affect the joint pattern of concentrations. For random effects \(k\) and \(l\), the covariance sensitivity patterns are

\frac{\partial V_i}{\partial \omega_{kk}}=j_{ik}j_{ik}^\mathsf{T}, \qquad \frac{\partial V_i}{\partial \omega_{kl}} =j_{ik}j_{il}^\mathsf{T}+j_{il}j_{ik}^\mathsf{T} \quad (k\ne l),

where \(j_{ik}\) is column \(k\) of \(J_i\), and \(\omega_{kl}\) is an element of \(\Omega\). If two such patterns are identical or proportional, their components cannot be separately distinguished from this local covariance information. If they are nearly proportional, separate estimates may be possible in principle but imprecise in practice. When residual parameters are estimated rather than known, their covariance sensitivity patterns must also be considered: a random-effect contribution that resembles a change in residual variance may be difficult to separate from it.

One Observation Can Support Only a Combination

Consider two random effects, \(\eta_{\mathrm{CL}}\) and \(\eta_{\mathrm{V}}\), on log clearance and log volume. A concentration at one scheduled time may respond to both. The example below is deliberately local: its sensitivities are illustrative, not a claim that every time point in a particular pharmacokinetic model has these values.

Worked Example: Two Random Effects at One Sampling Time

Suppose a single log-concentration measurement has local sensitivities \(1\) to each random effect, so \(J=[1\ \ 1]\). Let \(\Omega=\begin{bmatrix}\omega_{\mathrm{CL,CL}}&\omega_{\mathrm{CL,V}}\\ \omega_{\mathrm{CL,V}}&\omega_{\mathrm{V,V}}\end{bmatrix}\), and suppose the residual variance is known to be \(0.02\). The observation variance is

V=J\Omega J^\mathsf{T}+0.02 =\omega_{\mathrm{CL,CL}}+2\omega_{\mathrm{CL,V}}+\omega_{\mathrm{V,V}}+0.02.

Thus the one observation informs only the displayed combination of three covariance components, not each component separately. For example, the two hypothetical covariance matrices \(\Omega_A=\begin{bmatrix}0.04&0.01\\0.01&0.09\end{bmatrix}\) and \(\Omega_B=\begin{bmatrix}0.05&0.025\\0.025&0.05\end{bmatrix}\) are both valid positive-definite covariance matrices. For \(\Omega_A\), the random-effect contribution is \(0.04+2(0.01)+0.09=0.15\). For \(\Omega_B\), it is \(0.05+2(0.025)+0.05=0.15\). In both cases the total observation variance is \(0.15+0.02=0.17\).

Therefore, this schedule cannot choose between these two covariance structures on the basis of the local observation variance. More subjects can estimate the supported combination more precisely, but cannot make these identical contributions distinguishable at this one sensitivity pattern. The log-concentration variances and covariances here are on the squared log-concentration scale.

The assumptions matter. This argument uses a local linear approximation, a specified sensitivity at the sampling time, and a residual variance treated as known. If residual variance were also unknown, even the total variance \(0.17\) would not by itself separate random-effect variation from residual variation. Nonlinear model behavior across subjects can provide additional information, but it does not justify assuming that a sparse schedule identifies every covariance component.

Different Sensitivities Can Separate Components

A schedule is more informative when its observations have different sensitivity patterns. The following simple example shows how off-diagonal covariances across measurements can carry information not present in a single concentration. It assumes the local approximation is adequate, the residual errors are independent across the three measurements, and their variances are known.

Worked Example: Three Distinct Sensitivity Patterns

Suppose three hypothetical log-concentration observations have sensitivity matrix \(J=\begin{bmatrix}1&0\\0&1\\1&1\end{bmatrix}\). The first observation responds only to \(\eta_{\mathrm{CL}}\), the second only to \(\eta_{\mathrm{V}}\), and the third to both. Let \(\Omega=\begin{bmatrix}0.04&0.01\\0.01&0.09\end{bmatrix}\), and let each residual variance be \(0.01\), with independent residual errors.

The random-effect contribution to the covariance matrix is \(J\Omega J^\mathsf{T}\). Its diagonal entries are \(0.04\), \(0.09\), and \(0.04+0.09+2(0.01)=0.15\). Its off-diagonal entries are \(0.01\), \(0.04+0.01=0.05\), and \(0.01+0.09=0.10\). Adding \(0.01\) to each diagonal gives

V= \begin{bmatrix} 0.05&0.01&0.05\\ 0.01&0.10&0.10\\ 0.05&0.10&0.16 \end{bmatrix}.

Because residual errors are independent, they add nothing to the off-diagonal entries. In this constructed example, \(V_{12}=\omega_{\mathrm{CL,V}}=0.01\); then \(V_{13}=\omega_{\mathrm{CL,CL}}+\omega_{\mathrm{CL,V}}=0.05\), giving \(\omega_{\mathrm{CL,CL}}=0.05-0.01=0.04\). Likewise, \(V_{23}=\omega_{\mathrm{V,V}}+\omega_{\mathrm{CL,V}}=0.10\), giving \(\omega_{\mathrm{V,V}}=0.10-0.01=0.09\). The scheduled measurements supply distinct patterns that separate all three components locally.

This is an identification demonstration, not a claim that three observations per subject always suffice for reliable estimation. Actual precision depends on the number of subjects, the noise level, the distribution of sensitivities across individuals, and how closely the linear approximation describes the model over the relevant range of random effects.

Rank and Near-Dependence in Sensitivity Patterns

For several random effects, there are \(q(q+1)/2\) distinct elements in a symmetric \(q\times q\) matrix \(\Omega\). A useful local check is to form a sensitivity vector for each candidate variance or covariance component by collecting the relevant entries of its derivative matrix across observations and subjects. These vectors can be scaled and weighted to reflect the observation-error model. If they are linearly dependent, at least one component cannot be distinguished locally. If they are almost dependent, estimates can be highly uncertain even when the model-fitting procedure reports a finite answer.

For example, suppose the three normalized, weighted sensitivity vectors for two variances and their covariance have the following Gram matrix, whose entries are the pairwise inner products of those vectors:

G= \begin{bmatrix} 0.64&0.04&0.32\\ 0.04&0.64&0.16\\ 0.32&0.16&0.68 \end{bmatrix}.

The determinant can be expanded along the first row:

\begin{aligned} \det(G) &=0.64(0.64\cdot0.68-0.32\cdot0.16) -0.04(0.04\cdot0.68-0.32\cdot0.16)\\ &\quad+0.32(0.04\cdot0.16-0.64\cdot0.16)\\ &=0.64(0.384)-0.04(-0.024)+0.32(-0.096)\\ &=0.24576+0.00096-0.03072\\ &=0.216. \end{aligned}

The positive determinant means these three sensitivity vectors are linearly independent in this illustrative calculation. It does not establish that the components are estimated precisely: the smallest eigenvalue, the scale of the information, and the full model likelihood also matter. A determinant close to zero would indicate near-dependence, but a nonzero determinant alone is not a precision guarantee.

Worked Example: A Correlation Estimate With Weak Support

For two random effects, the correlation is \(\rho=\omega_{12}/\sqrt{\omega_{11}\omega_{22}}\), when both variances are positive. Suppose an estimated standard deviation for a volume effect is \(0.02\) on the log scale, while the clearance standard deviation is \(0.20\), and the estimated covariance is \(0.002\). The corresponding correlation is

\hat{\rho} =\frac{0.002}{\sqrt{(0.20^2)(0.02^2)}} =\frac{0.002}{0.004} =0.50.

The arithmetic is correct, but the value \(0.50\) should not be read as strong evidence for a well-determined relationship. If the small volume variance is weakly supported by the schedule, modest changes in its estimate can substantially change the denominator and hence the estimated correlation. For instance, keeping the other two estimates fixed but changing the volume standard deviation from \(0.02\) to \(0.01\) changes the correlation to \(0.002/\sqrt{(0.20^2)(0.01^2)}=1.00\), the boundary of the allowable range. In practice, covariance estimates must also respect the requirement that \(\Omega\) be positive semidefinite. The correlation is therefore especially fragile when either variance is near zero or poorly informed.

Diagnostics, Alternatives, and Limitations

Sensitivity checks are most useful alongside diagnostics from model fitting. Warning signs include very large standard errors for covariance parameters, correlations near \(1\) or \(-1\), variance estimates at or near zero, changes in estimates when starting values change, and instability when a small number of sampling times are altered. These patterns do not prove that the design is uninformative, but they are reasons to inspect the schedule and the covariance sensitivity patterns rather than treating a converged fit as sufficient evidence.

There are several practical responses, each with a different trade-off:

Use a simpler covariance structure. Estimating fewer components can avoid asking the data to distinguish several similar patterns. This is a modeling restriction, not evidence that the omitted covariance is exactly zero.
Improve the schedule. Add or move sampling times where the concentration predictions respond differently to the random effects of interest. A denser schedule is useful when it adds distinct sensitivities, not merely because it adds observations.
Use prior or external information when justified. Such information can stabilize weakly supported components, but the resulting estimates then depend partly on that information rather than on the sparse concentration data alone.

The local calculation has limitations. Sensitivities depend on the structural model, typical parameter values, and scheduled times. A linear approximation can miss information arising from nonlinear behavior, while a schedule that appears informative locally may be practically weak once residual variability and finite sample size are considered. Conversely, exact local dependence is a strong warning: increasing the number of subjects under the same design improves information about supported patterns but does not remove the dependence. The correct conclusion is about what the design supports, not about whether a particular software package can return a number.

Common pitfall: “The covariance model converged and reported a correlation, so the schedule identified that correlation.” Convergence only means the fitting procedure reached a numerical result under its settings. Check whether the sensitivity patterns for the variance and covariance components are distinct, whether the estimates are stable, and whether the variances supporting a correlation are themselves informed.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

With one observation whose sensitivities to two random effects are both \(1\), which combination of their two variances and covariance contributes to the random-effect observation variance?
Why can increasing the number of subjects fail to distinguish two Omega components in an exactly rank-deficient design?
In the three-observation example, which covariance entry identifies \(\omega_{\mathrm{CL,V}}\) when residual errors are independent?
What does a positive determinant of a sensitivity Gram matrix establish, and what does it not establish?
Why can a correlation estimate be unstable when one random-effect variance is close to zero?

Solutions. (1) The contribution is \(\omega_{11}+2\omega_{12}+\omega_{22}\), because \(J=[1\ \ 1]\). (2) Each subject contributes information in the same dependent sensitivity directions; additional subjects can sharpen estimates of supported combinations but do not create a new direction that separates the confounded components. (3) It is the off-diagonal observation covariance \(V_{12}\), equal to \(\omega_{\mathrm{CL,V}}\) in that example because the residual errors are independent. (4) A positive determinant establishes linear independence of the sensitivity vectors in the stated calculation; it does not establish small standard errors or adequate practical information. (5) The correlation divides the covariance by the product of the standard deviations, so uncertainty or small changes in a near-zero variance can substantially change the denominator and the resulting correlation.