Taylor Linearization of Individual Predictions

Why Linearize an Individual Prediction?

In a nonlinear mixed-effects model, an individual prediction depends on that individual's random effects. Write the prediction at a fixed observation time or set of covariates as $f(\eta)$, where $\eta$ is a vector of random effects and model parameters and covariates are held fixed. The function may be nonlinear even when the random effects enter the model simply. A Taylor expansion describes how the prediction changes near a chosen random-effect value.

The key is to treat linearization as a local approximation. It replaces the curved prediction function near an expansion point with its tangent (or tangent plane). The prediction and its first derivatives determine that tangent. Curvature, represented by second and higher derivatives, is not captured by the first-order term.

Definition: Let $f:\mathbb{R}^q\to\mathbb{R}$ be a scalar individual prediction, and let $\eta^\ast$ be an expansion point. For a displacement $\Delta\eta=\eta-\eta^\ast$, its first-order Taylor linearization is $$ f(\eta)\approx f(\eta^\ast)+\nabla f(\eta^\ast)^\mathsf{T}\Delta\eta. $$ Here $\nabla f(\eta^\ast)$ is the gradient: the vector of first partial derivatives with respect to the random effects, evaluated at $\eta^\ast$. The approximation keeps the value and slope at the expansion point and omits curvature.

If $\eta$ has $q$ components, a vector of $r$ predictions has an $r\times q$ Jacobian matrix $J(\eta^\ast)$. Its entry in row $k$, column $j$ is the derivative of prediction $k$ with respect to random effect $j$, evaluated at $\eta^\ast$. The vector expansion is $f(\eta)\approx f(\eta^\ast)+J(\eta^\ast)\Delta\eta$.

A common expansion point is $\eta^\ast=0$, the typical individual in a model with centered random effects. But zero is not required: any point in the domain where the needed derivatives exist can be used. The expansion point must be stated because both the intercept and the derivatives depend on it.

What the First-Order Term Leaves Out

Suppose $f$ has continuous second derivatives along the line from $\eta^\ast$ to $\eta$. Define $g(s)=f(\eta^\ast+s\Delta\eta)$ for $0\leq s\leq1$. The ordinary one-variable Taylor formula with integral remainder gives $g(1)=g(0)+g'(0)+\int_0^1(1-s)g''(s)\,ds$. By the chain rule, $g'(0)=\nabla f(\eta^\ast)^\mathsf{T}\Delta\eta$ and $g''(s)=\Delta\eta^\mathsf{T}H_f(\eta^\ast+s\Delta\eta)\Delta\eta$, where $H_f$ is the Hessian matrix of second partial derivatives. Therefore,

f(\eta)=f(\eta^\ast)+\nabla f(\eta^\ast)^\mathsf{T}\Delta\eta +\int_0^1(1-s)\Delta\eta^\mathsf{T}H_f(\eta^\ast+s\Delta\eta)\Delta\eta\,ds.

The integral is the remainder. It depends on the Hessian along the path, not just at the expansion point. If the absolute value of the quadratic form $\Delta\eta^\mathsf{T}H_f(\eta)\Delta\eta$ is bounded by $M$ along that path, the remainder has absolute value at most $M/2$, because $\int_0^1(1-s)\,ds=1/2$. This makes the local nature of the approximation explicit: small displacements and modest curvature tend to make the omitted term small, but a first-order expansion alone does not guarantee that it is small.

For a vector prediction, this argument applies to each component, with a separate Hessian and remainder for each predicted quantity. Derivative units also matter. If a concentration is measured in mg/L and a random effect is dimensionless, the derivative has units mg/L per unit of random effect; multiplying it by a change in that random effect gives a change in mg/L.

Worked Example: A Scalar Prediction Near Zero

Suppose a hypothetical individual prediction is $f(\eta)=100\exp(-0.4\eta)$ mg/L, with dimensionless random effect $\eta$. Linearize at $\eta^\ast=0$, then compare the approximation with the exact prediction at $\eta=0.1$.

At zero, $f(0)=100$ mg/L. Differentiating gives $f'(\eta)=-40\exp(-0.4\eta)$ mg/L per unit of $\eta$, so $f'(0)=-40$ mg/L per unit. The first-order approximation is

f(\eta)\approx100-40\eta\quad\text{mg/L}.

At $\eta=0.1$, the linearized value is $100-40(0.1)=96.0$ mg/L. The exact value is $100\exp(-0.04)=96.0789$ mg/L, so the approximation is lower by about $0.0789$ mg/L. The second derivative is $f''(\eta)=16\exp(-0.4\eta)$ mg/L per squared unit of $\eta$; at zero the second-order Taylor contribution is $\tfrac12(16)(0.1)^2=0.0800$ mg/L. Its closeness to the observed discrepancy is consistent with the small higher-order terms in this example.

Expansion Point and Derivatives in a Pharmacokinetic Prediction

A useful way to see how random effects affect a prediction is to start with a structural model and differentiate its individual prediction with respect to the random effects. Consider the following hypothetical one-compartment bolus model, with dose $D$, clearance $CL$, volume $V$, and time $t$: $C(t)=D/V\exp[-(CL/V)t]$. Assume log-scale random effects, $CL=CL_{\mathrm{typ}}\exp(\eta_{CL})$ and $V=V_{\mathrm{typ}}\exp(\eta_V)$. At $\eta_{CL}=\eta_V=0$, clearance and volume equal their typical values. The derivatives must include the way each random effect changes both the prefactor and, where relevant, the exponential term.

Worked Example: Linearizing Concentration With Two Random Effects

For a hypothetical subject, let $D=100$ mg, $CL_{\mathrm{typ}}=4$ L/h, $V_{\mathrm{typ}}=20$ L, and $t=2$ h. Use $\eta^\ast=(0,0)$ and consider the displacement $\Delta\eta=(0.1,-0.05)$, in the order $(\eta_{CL},\eta_V)$. Calculate the gradient, the first-order change, and compare the resulting prediction with the exact one.

Writing the prediction directly as a function of the random effects gives $C(t;\eta)=5\exp(-\eta_V)\exp[-0.4\exp(\eta_{CL}-\eta_V)]$ mg/L. At the expansion point, $C(t;0,0)=5\exp(-0.4)=3.351600$ mg/L. Differentiating the log of the prediction and then multiplying by $C$ gives

\frac{\partial C}{\partial\eta_{CL}} =C\left[-0.4\exp(\eta_{CL}-\eta_V)\right], \qquad \frac{\partial C}{\partial\eta_V} =C\left[-1+0.4\exp(\eta_{CL}-\eta_V)\right].

At $(0,0)$, the gradient is approximately $(-1.340640,-2.010960)$ mg/L per unit of the respective random effect. The first-order change is the gradient dotted with the displacement: $(-1.340640)(0.1)+(-2.010960)(-0.05)=-0.134064+0.100548=-0.033516$ mg/L. Thus the linearized prediction is approximately $3.351600-0.033516=3.318084$ mg/L. The displayed products and their sum agree at the shown precision.

For comparison, the exact prediction at $(0.1,-0.05)$ is $5\exp(0.05)\exp[-0.4\exp(0.15)]$, which is approximately $3.30258$ mg/L. The exact change from the expansion-point prediction is about $-0.04902$ mg/L, rather than the linearized change of $-0.033516$ mg/L. The difference shows that curvature is material for this displacement; it does not indicate an arithmetic error in the derivative calculation.

Multiple Predictions and a Nonzero Expansion Point

When several quantities are predicted for one individual, linearize each component at the same stated expansion point and collect the derivatives into a Jacobian. A nonzero expansion point is useful for illustrating that the tangent is local to the point chosen, rather than inherently tied to the population value.

Worked Example: A Two-Output Prediction at a Nonzero Point

Consider hypothetical predictions $f_1(\eta)=10+2\eta_1+\eta_2^2$ units and $f_2(\eta)=4\exp(\eta_1)$ units. Linearize at $\eta^\ast=(0.2,-0.1)$ for $\Delta\eta=(0.05,0.2)$, so the target point is $(0.25,0.1)$.

At the expansion point, $f_1=10+2(0.2)+(-0.1)^2=10.41$ units and $f_2=4\exp(0.2)=4.885611$ units. The Jacobian is

J(\eta^\ast)= \begin{pmatrix} 2 & 2\eta_2^\ast\\ 4\exp(\eta_1^\ast) & 0 \end{pmatrix} = \begin{pmatrix} 2 & -0.2\\ 4.885611 & 0 \end{pmatrix}.

The first-order change in $f_1$ is $2(0.05)+(-0.2)(0.2)=0.0600$ units, giving the linearized value $10.47$ units. For $f_2$, the change is $4.885611(0.05)+0(0.2)=0.244281$ units, giving $5.129892$ units. At the target point, the exact values are $f_1=10+2(0.25)+(0.1)^2=10.51$ units and $f_2=4\exp(0.25)=5.136102$ units. The first component misses by $0.0400$ units because it omits the quadratic change in $\eta_2$; the second differs by about $0.00621$ units because the exponential function is curved.

Checking the Approximation and Avoiding Common Errors

A Taylor expansion is a mathematical description of a function near a point, not a claim that a particular displacement is small enough for a chosen application. When the exact prediction can be evaluated, a direct check is to compare it with the linearized prediction over relevant random-effect values. Examine the size and direction of the discrepancy for each output, especially near the edges of the range that matters. A close match at the expansion point is guaranteed by construction; it is not evidence of accuracy far from that point.

If exact evaluation is not convenient, inspect the curvature analytically or numerically and consider how far the random effects may move from the expansion point. Derivatives can also be checked by small finite differences: for component $j$, compare the analytic derivative with $[f(\eta^\ast+h e_j)-f(\eta^\ast)]/h$ for a suitably small $h$, where $e_j$ changes only component $j$. This is a check on the derivative calculation, not a replacement for checking the neglected curvature over the range of interest.

Wrong reasoning: “The random effects have mean zero, so the prediction for every individual is $f(0)$.” Correction: $f(0)$ is the prediction at zero random effects. An individual with $\eta\ne0$ has prediction $f(\eta)$; the tangent approximation describes that prediction relative to the stated expansion point.
Wrong reasoning: “The first derivative is the prediction change.” Correction: The first-order change is the derivative (or gradient) multiplied by the displacement. A derivative alone is a local rate of change, with units per unit of random effect.
Wrong reasoning: “A small derivative means the linearization is accurate.” Correction: Accuracy also depends on displacement and curvature. A function can have a modest slope at the expansion point and still curve substantially nearby.
Wrong reasoning: “Expanding at zero and expanding at a subject-specific point give the same tangent.” Correction: Both the prediction and its derivatives generally change with the expansion point, so the two tangents need not agree.

The main alternative for prediction at a specified random-effect value is to evaluate the original nonlinear function directly; that gives the model prediction without Taylor truncation, when the function can be evaluated. A first-order expansion is simpler and gives a transparent local sensitivity calculation, but can be inaccurate over a wide range. Expanding around a different point can improve local fidelity near that point, though it does not make the approximation uniformly accurate elsewhere. Higher-order expansions explicitly add curvature terms, but require additional derivatives and still have a remainder. These are alternatives for representing the prediction; none removes the need to state the expansion point and assess the range where the approximation is used.

Key takeaway: A first-order Taylor linearization is the prediction at the expansion point plus the gradient times the random-effect displacement. Its validity is local; the omitted Hessian-based remainder is the mathematical source of its approximation error.

Check Your Understanding

Answer each question using the expansion point and units explicitly. Short solutions follow.

For $f(\eta)=20+3\eta$ mg/L, what is the first-order approximation at $\eta^\ast=0$, and what value does it give at $\eta=0.2$?
For $f(\eta)=\exp(\eta)$ units, expand at $\eta^\ast=0$. Compare the linearized and exact values at $\eta=0.1$.
For a two-component random effect, what does entry $J_{2,1}(\eta^\ast)$ represent?
Why does knowing the gradient at the expansion point not by itself establish accuracy at a distant random-effect value?

Solutions. (1) The value is $f(0)=20$ mg/L and the derivative is $3$ mg/L per unit, so the approximation is $20+3\eta$ mg/L; at $0.2$ it is $20.6$ mg/L. (2) Since $f(0)=1$ and $f'(0)=1$, the approximation is $1+\eta$, giving $1.1$ units. The exact value is $\exp(0.1)\approx1.10517$ units. (3) It is the derivative of the second prediction with respect to the first random-effect component, evaluated at $\eta^\ast$; it measures local change in prediction 2 per unit change in random effect 1. (4) The gradient specifies only the tangent at the expansion point. Accuracy away from that point also depends on displacement and on the function's curvature along the path.