Individual Sensitivities in a Nonlinear Mixed-Effects Model

Why Individual Sensitivities Matter

In a nonlinear mixed-effects model, a subject’s predicted observations depend on both population-level parameters and that subject’s random effects. A sensitivity is a derivative of the prediction with respect to one of these inputs, evaluated at specified parameter values, random effects, and observation times. It measures the local change in a prediction when an input changes slightly while the other inputs are held fixed.

These derivatives are useful for describing how a particular sampling schedule responds to parameter changes and for constructing local calculations later in population design. Here the object is the conditional, subject-level prediction and its derivatives. A sensitivity is not itself a likelihood, an estimate of a parameter, or a statement about the distribution of predictions across subjects.

Definition: For subject \(i\), let \(t_{ij}\) denote the fixed time of observation \(j\), let \(\beta\) be a vector of population fixed effects, and let \(\eta_i\) be that subject’s vector of random effects. Write the conditional prediction as \(m_i(t_{ij};\beta,\eta_i)\). The prediction vector is \(\mathbf m_i(\beta,\eta_i)=[m_i(t_{i1};\beta,\eta_i),\ldots,m_i(t_{in_i};\beta,\eta_i)]^\mathsf{T}\). Its sensitivity matrices are \(J_{\beta,i}=\partial\mathbf m_i/\partial\beta^\mathsf{T}\) and \(J_{\eta,i}=\partial\mathbf m_i/\partial\eta_i^\mathsf{T}\), with dimensions \(n_i\) by the number of elements in \(\beta\) and \(n_i\) by the number of elements in \(\eta_i\), respectively.

Each row corresponds to an observation time; each column corresponds to a parameter or random effect. The derivatives are evaluated at the parameter and random-effect values of interest. Unless otherwise stated, observation times and doses are treated as fixed in taking the derivatives.

Differentiate the Conditional Prediction

Suppose an individual parameter vector \(\phi_i\) is generated from population fixed effects and random effects through a differentiable transformation \(\phi_i=g(\beta,\eta_i)\). If the prediction is \(m_i=h(t_i,\phi_i)\), the chain rule gives the sensitivities:

\frac{\partial m_i}{\partial \beta^\mathsf{T}} = \frac{\partial h}{\partial \phi_i^\mathsf{T}} \frac{\partial g}{\partial \beta^\mathsf{T}}, \qquad \frac{\partial m_i}{\partial \eta_i^\mathsf{T}} = \frac{\partial h}{\partial \phi_i^\mathsf{T}} \frac{\partial g}{\partial \eta_i^\mathsf{T}}.

For multiple observations, apply these derivatives at each time and stack the resulting rows. If the model is expressed directly as \(m_i(t;\beta,\eta_i)\), differentiate that expression directly; the chain rule is still the underlying principle. Differentiability is required at the evaluation point. A model can be smooth in its parameters even when it is complicated to evaluate, but a discontinuity or a boundary such as a parameter exactly at a non-differentiable kink requires special care.

Specify the conditional model.
Write the prediction as a function of observation time, population fixed effects, and the subject’s random effects.

Identify the parameterization.
State how individual parameters depend on population parameters and random effects, including any exponential or other link transformation.

Differentiate and evaluate.
Use the chain rule, evaluate at the stated values, then stack rows by observation and columns by parameter.

Check the result.
Check matrix dimensions and units, and compare analytic derivatives with a suitable numerical check when possible.

The usual conditional prediction depends directly on \(\beta\) and \(\eta_i\), not on the random-effect covariance matrix \(\Omega\). Thus, with \(\eta_i\) held fixed, its direct derivative with respect to \(\Omega\) is zero. The covariance matrix describes the distribution of random effects across subjects, rather than changing a given conditional prediction when that subject’s random effects are held fixed. If instead a model is parameterized using a standardized random vector \(z_i\) and a transformation such as \(\eta_i=L(\Omega)z_i\), a derivative with respect to \(\Omega\) must include the derivative through that transformation. That is a different choice of what is held fixed.

Worked Sensitivities for Nonlinear Models

Worked Example: A Saturable Prediction

Consider the hypothetical individual prediction \(m(C)=\theta_0+\theta_1C/(\theta_2+C)\), where \(C\) is a fixed concentration in mg/L. Let \(\theta_0=1\) mg/L, \(\theta_1=4\) mg/L, and \(\theta_2=2\) mg/L. Calculate the sensitivities at \(C=2\) mg/L.

Differentiate with respect to each parameter:

\frac{\partial m}{\partial\theta_0}=1,\qquad \frac{\partial m}{\partial\theta_1}=\frac{C}{\theta_2+C},\qquad \frac{\partial m}{\partial\theta_2}=-\frac{\theta_1C}{(\theta_2+C)^2}.

The prediction is \(m=1+4(2)/(2+2)=3\) mg/L. Substitution gives \(\partial m/\partial\theta_0=1\), \(\partial m/\partial\theta_1=2/4=0.5\), and \(\partial m/\partial\theta_2=-4(2)/4^2=-0.5\). For this one observation, the sensitivity matrix is \(J_{\theta}=[1,\ 0.5,\ -0.5]\).

The first and second sensitivities are dimensionless: changing either \(\theta_0\) or \(\theta_1\) by 1 mg/L changes the prediction locally by 1 or 0.5 mg/L, respectively. The third derivative is \(-0.5\) mg/L per mg/L of \(\theta_2\). It is dimensionless when both output and \(\theta_2\) are expressed in mg/L, but its interpretation remains a change in prediction per change in \(\theta_2\). The negative sign agrees with the model: increasing \(\theta_2\) reduces the saturable contribution at fixed positive \(C\).

Worked Example: Population and Random-Effect Sensitivities in a PK Model

Consider a hypothetical one-compartment intravenous bolus prediction \(m(t)=D/V_i\exp(-CL_i t/V_i)\), with \(CL_i=\exp(\beta_{CL}+\eta_{CL,i})\) in L/h and \(V_i=\exp(\beta_V+\eta_{V,i})\) in L. Use \(D=100\) mg, \(t=4\) h, population values \(\exp(\beta_{CL})=5\) L/h and \(\exp(\beta_V)=20\) L, and random effects \(\eta_{CL,i}=0.10\), \(\eta_{V,i}=-0.05\). Find the prediction and the sensitivity row with columns \((\beta_{CL},\beta_V,\eta_{CL,i},\eta_{V,i})\).

The individual parameters are \(CL_i=5\exp(0.10)=5.526\) L/h and \(V_i=20\exp(-0.05)=19.02\) L. Hence \(k_i=CL_i/V_i=0.2905\) h\(^{-1}\), and \(m(4)=100/19.02\times\exp[-0.2905(4)]=1.645\) mg/L.

Because the fixed effects and their corresponding random effects enter as sums on the log scale, the derivatives with respect to \(\beta_{CL}\) and \(\eta_{CL,i}\) are equal. The same is true for \(\beta_V\) and \(\eta_{V,i}\). Differentiating the log prediction gives \(\partial\log m/\partial\beta_{CL}=-tCL_i/V_i\) and \(\partial\log m/\partial\beta_V=-1+tCL_i/V_i\). Multiplying each by \(m\) yields the prediction derivatives:

J_{(\beta,\eta),i} = \left[ m(-tk_i),\quad m(tk_i-1),\quad m(-tk_i),\quad m(tk_i-1) \right].

Here \(tk_i=4(0.2905)=1.162\), so the row is approximately \([-1.911,\ 0.2663,\ -1.911,\ 0.2663]\) mg/L per unit change in the corresponding log-scale input. Increasing clearance decreases the concentration at this time, as indicated by the negative derivative. Increasing volume has two opposing effects: it lowers the initial concentration but also lowers the elimination rate. At this particular time the latter effect slightly dominates, giving a positive derivative. The units are concentration per unit of the log-scale parameter; the random effects are dimensionless.

Worked Example: Building a Two-Time Sensitivity Matrix

For a hypothetical saturable response, let \(m(C)=E_{\max,i}C/(K_i+C)\), with \(E_{\max,i}=\exp(\beta_E+\eta_{E,i})\) in response units and \(K_i=\exp(\beta_K+\eta_{K,i})\) in mg/L. Evaluate the population-fixed-effect and random-effect matrices for one subject at \(C=1\) and \(4\) mg/L, with \(E_{\max,i}=8\) response units and \(K_i=2\) mg/L. These values could, for example, arise when the random effects are zero and the population values imply \(E_{\max}=8\) and \(K=2\).

The prediction and derivatives are

m(C)=\frac{E_{\max,i}C}{K_i+C},\qquad \frac{\partial m}{\partial\beta_E} =\frac{\partial m}{\partial\eta_{E,i}}=m(C),\qquad \frac{\partial m}{\partial\beta_K} =\frac{\partial m}{\partial\eta_{K,i}} =-\frac{E_{\max,i}C K_i}{(K_i+C)^2}.

At \(C=1\), \(m=8/3=2.667\) and the \(K_i\) derivative is \(-8(1)(2)/3^2=-1.778\). At \(C=4\), \(m=32/6=5.333\) and the \(K_i\) derivative is \(-8(4)(2)/6^2=-1.778\). In the stated column order, the matrices are

J_{\beta,i}= \begin{bmatrix} 2.667 & -1.778\\ 5.333 & -1.778 \end{bmatrix}, \qquad J_{\eta,i}= \begin{bmatrix} 2.667 & -1.778\\ 5.333 & -1.778 \end{bmatrix}.

The first column has units of response units per unit log-scale change; the second has units of response units per unit log-scale change in \(K_i\). The equal matrices follow from the additive log parameterization, not from a general rule that population and random-effect sensitivities must match. With these two observation times, changes in a population log effect and changes in its corresponding random effect act identically on this subject’s conditional predictions. The population distribution of random effects, rather than these conditional derivatives alone, distinguishes their roles in a full mixed-effects analysis.

Checks, Alternatives, and Limitations

A reliable sensitivity calculation has several simple checks. First, verify dimensions: for \(n_i\) observations, \(J_{\beta,i}\) must have \(n_i\) rows and one column per population fixed effect. Second, check units for each derivative. Third, check signs or limiting behavior when the model has an interpretable monotonic relationship. For example, in the saturable model above, increasing \(K_i\) at fixed positive \(C\) must lower the prediction. Such checks can reveal a missing chain-rule factor or a sign error.

Analytic differentiation is often the clearest approach for a simple model. For a complicated model, automatic differentiation or finite differences can be useful. A central finite-difference approximation for parameter \(a\), with small perturbation \(h\), is

\frac{\partial m}{\partial a}\approx\frac{m(a+h)-m(a-h)}{2h}.

This is a numerical check, not an exact derivative. Results can be unstable if \(h\) is too large (so curvature matters) or too small (so rounding error dominates). Compare a sequence of reasonable step sizes and, where feasible, compare with an analytic or automatic derivative. For predictions obtained by solving differential equations, numerical solver tolerances can also affect finite-difference checks.

Sensitivity matrices depend on the parameterization and the evaluation point. A derivative with respect to a parameter on the natural scale is not numerically interchangeable with a derivative with respect to its logarithm. Likewise, a matrix evaluated at \(\eta_i=0\) need not describe the local behavior for a subject with a substantially different random effect. State whether the evaluation uses a typical subject, a specified individual, or another clearly defined point.

Near-zero or nearly proportional columns indicate that the predictions at the chosen times respond similarly to small changes in different inputs. This is a warning that the schedule may provide little local separation of those effects; it is not, by itself, a complete identifiability analysis. A matrix may also be ill-conditioned because columns use very different units or scales, so interpret numerical conditioning with parameter scaling in mind.

Wrong: “The random-effect covariance \(\Omega\) changes this subject’s conditional prediction, so differentiate the prediction with respect to \(\Omega\) while holding \(\eta_i\) fixed.” Correction: Under the usual conditional parameterization, \(\Omega\) governs the distribution of \(\eta_i\); it has no direct effect on the prediction when \(\eta_i\) is held fixed.
Wrong: “If the individual parameter is exponential in a log-scale input, its derivative is one.” Correction: The derivative of \(\exp(a)\) with respect to \(a\) is \(\exp(a)\); carry that factor through the chain rule.
Wrong: “The derivative has the same units as the prediction.” Correction: A sensitivity has units of prediction divided by units of the differentiated input.

Individual sensitivities are local descriptions: they approximate changes for sufficiently small changes in the inputs at the stated evaluation point. They do not, on their own, quantify the full range of population predictions or replace an analysis of the subject-level likelihood. They are also not guaranteed to be informative at every design point; for example, a prediction can be nearly insensitive to a parameter at an extreme observation time. The principal alternatives for obtaining derivatives are hand differentiation, automatic differentiation, and numerical differences. These are complementary ways to calculate or check the same local quantity, rather than different definitions of sensitivity.

Check Your Understanding

Answer the questions, then compare with the brief solutions below.

For \(m(C)=\theta_0+\theta_1C/(\theta_2+C)\), what is \(\partial m/\partial\theta_2\), and what is its sign when all three quantities are positive?
A subject has three observation times and two random effects. What are the dimensions of \(J_{\eta,i}\)?
If \(CL_i=\exp(\beta_{CL}+\eta_{CL,i})\), what relationship holds between \(\partial m/\partial\beta_{CL}\) and \(\partial m/\partial\eta_{CL,i}\) for a conditional prediction depending on these inputs only through \(CL_i\)?
Why is the direct derivative of a conditional prediction with respect to \(\Omega\) zero when \(\eta_i\) is held fixed?
A finite-difference derivative changes substantially as the step size is reduced. Give one reason this may happen and one useful diagnostic response.

Solutions. (1) \(\partial m/\partial\theta_2=-\theta_1C/(\theta_2+C)^2\), which is negative for positive \(\theta_1,C,\theta_2\). (2) \(J_{\eta,i}\) is \(3\) by \(2\). (3) The derivatives are equal because both inputs enter through the same sum \(\beta_{CL}+\eta_{CL,i}\); the chain-rule factors are identical. (4) \(\Omega\) determines the random-effect distribution, not the fixed conditional prediction at a specified \(\eta_i\). (5) A step that is too small can amplify rounding error or solver tolerance; compare several step sizes and, if possible, an analytic or automatic derivative.