First-Order Taylor Expansion Around Zero Random Effects

Why Expand Around Zero Random Effects?

In a nonlinear mixed-effects model, an individual prediction depends on both population parameters and that individual's random effects. The prediction at zero random effects is the model's typical-subject prediction under the chosen parameterization. A Taylor expansion around that point describes how the prediction changes for small departures in the random effects. It is useful because it separates the prediction at the expansion point from its local sensitivity to each random effect.

The expansion is a local approximation, not a claim that the individual model is linear. Its accuracy depends on how much the prediction curves over the range of random effects that matters. A small random effect alone does not guarantee accuracy: strong curvature can make even a modest departure consequential. Conversely, a curved model can be approximated well over a sufficiently narrow range.

Definition: Let $f(\theta,\eta)$ be a scalar individual prediction, where $\theta$ denotes fixed-effect parameters and $\eta=(\eta_1,\ldots,\eta_q)^T$ is the vector of random effects. For fixed $\theta$, the first-order Taylor approximation around zero random effects is $$ f(\theta,\eta)\approx f(\theta,0)+\nabla_\eta f(\theta,0)^T\eta. $$ Here, $\nabla_\eta f(\theta,0)$ is the gradient with respect to the random effects, evaluated at $\eta=0$. The approximation retains the prediction and its local slope at zero and omits terms involving curvature and higher derivatives.

In an application, the prediction may also depend on a design point, such as a sampling time $t$. That design point is held fixed when taking derivatives with respect to $\eta$; it can be included among the arguments of $f$ without changing the expansion. If the prediction is vector-valued, apply the expansion to each component.

Derivation and Conditions

Suppose $f(\theta,\eta)$ has continuous second partial derivatives with respect to $\eta$ along the line from zero to the effect value under consideration. Define the one-dimensional function $g(s)=f(\theta,s\eta)$, for $0\leq s\leq1$. The chain rule gives $g'(s)=\nabla_\eta f(\theta,s\eta)^T\eta$, and differentiating again gives $g''(s)=\eta^T H_\eta f(\theta,s\eta)\eta$, where $H_\eta f$ is the Hessian matrix of second derivatives with respect to the random effects. The one-dimensional Taylor theorem with remainder then gives the exact expression

f(\theta,\eta)=f(\theta,0)+\nabla_\eta f(\theta,0)^T\eta +\frac{1}{2}\eta^T H_\eta f(\theta,s^*\eta)\eta

for some $s^*$ between 0 and 1. Thus the first-order approximation omits a remainder determined by curvature along the path from zero to $\eta$, not necessarily by the curvature at zero alone. If the Hessian's operator norm is at most $M$ everywhere on that path, then

\left|f(\theta,\eta)-f(\theta,0)-\nabla_\eta f(\theta,0)^T\eta\right| \leq \frac{M}{2}\|\eta\|^2.

This bound explains the two main ingredients in the local error: curvature and the squared size of the effect vector. It is a useful justification, not a guarantee of a tight numerical bound; a uniform upper bound $M$ may be difficult to obtain or much larger than the curvature actually encountered.

The expansion is pointwise and does not require the random effects to follow a normal distribution. To use it as a practical approximation for a population of individuals, however, it matters whether the effects typically stay in a region where the neglected terms are small. A mean-zero random-effect distribution does not ensure that each individual's effect is close to zero. Also, the meaning of $\eta=0$ depends on the model parameterization. For example, if a positive parameter is modeled as a typical value multiplied by $\exp(\eta)$, zero effect corresponds to that typical value; it does not generally mean that the population arithmetic mean of the parameter equals the typical value.

Fix the prediction context.
Hold the time, dose, covariates, and fixed-effect parameters at the values for which the individual prediction is being approximated.

Evaluate at zero effects.
Calculate $f(\theta,0)$, the prediction at the expansion point.

Calculate the random-effect gradient.
Evaluate each partial derivative $\partial f/\partial\eta_j$ at $\eta=0$, using analytic derivatives or a suitably checked numerical method.

Form and assess the approximation.
Use $f(\theta,0)+\nabla_\eta f(\theta,0)^T\eta$, then compare with exact predictions over plausible effect values or inspect the second-order remainder.

Worked Examples

Worked Example: A Log-Random-Effect Clearance Model

Consider a hypothetical one-compartment concentration prediction at a fixed time, with volume held fixed and clearance varying by a log-random effect: $C(\eta)=10\exp[-0.6\exp(\eta)]$ mg/L. The effect $\eta$ is dimensionless, and $\eta=0$ gives the typical-subject prediction. Find the first-order approximation and compare it with the exact prediction at $\eta=0.20$.

At zero, $C(0)=10\exp(-0.6)=5.488$ mg/L. Differentiating with respect to $\eta$, $C'(\eta)=C(\eta)[-0.6\exp(\eta)]$, so $C'(0)=-0.6(5.488)=-3.293$ mg/L per unit of $\eta$. The first-order expansion is therefore

C(\eta)\approx 5.488-3.293\eta \quad \text{mg/L}.

At $\eta=0.20$, the linear approximation is $5.488-3.293(0.20)=4.8294$ mg/L. Direct evaluation of the model gives $10\exp[-0.6\exp(0.20)]=4.805415$ mg/L. The signed difference, exact minus approximate, is $4.805415-4.8294=-0.02398$ mg/L. Relative to the exact prediction, this is $100(-0.0239849/4.805415)=-0.4991\%$. The first-order approximation overpredicts the exact value by about $0.02398$ mg/L, or $0.4991\%$ of the exact prediction. In this case the local approximation is close at the specified effect value; that does not establish accuracy for larger effects or at other times.

Worked Example: Two Random Effects and Directional Curvature

For a hypothetical prediction $f(\eta_1,\eta_2)=100\exp(\eta_1+2\eta_2)$ mg/L, calculate the first- and second-order approximations at $(\eta_1,\eta_2)=(0.10,0.05)$, and compare each with the exact value. Both effects are dimensionless.

At zero, $f(0,0)=100$ mg/L. The gradient there is $(100,200)^T$ mg/L, so the first-order approximation is $100+100\eta_1+200\eta_2$. Substitution gives $100+100(0.10)+200(0.05)=120$ mg/L.

The Hessian at zero is $\begin{pmatrix}100&200\\200&400\end{pmatrix}$ mg/L. The second-order term at the specified effects is

\frac{1}{2} \begin{pmatrix}0.10&0.05\end{pmatrix} \begin{pmatrix}100&200\\200&400\end{pmatrix} \begin{pmatrix}0.10\\0.05\end{pmatrix} =\frac{1}{2}(100)(0.20)^2=2.000\ \text{mg/L}.

The second-order approximation is $120+2.000=122.0$ mg/L. The exact prediction is $100\exp[0.10+2(0.05)]=100\exp(0.20)=122.1403$ mg/L. Thus the first-order approximation is low by $2.1403$ mg/L, which is $1.752\%$ of the exact prediction. The second-order approximation is low by $0.1403$ mg/L, or $0.1148\%$. The quadratic term here depends on the combined direction $\eta_1+2\eta_2$; inspecting each effect separately would not capture the full directional curvature.

Worked Example: Effect Scale Can Make the Omission Matter

Consider the hypothetical scalar prediction $f(\eta)=10\exp(\eta)$ mg/L. Compare first- and second-order Taylor approximations with the exact value at $\eta=0.30$.

The value, first derivative, and second derivative at zero are all 10, with the derivatives expressed in mg/L per unit effect and mg/L per squared unit effect, respectively. The first-order approximation is $10+10(0.30)=13.0$ mg/L. The second-order term is $\tfrac{1}{2}(10)(0.30)^2=0.4500$ mg/L, so the second-order approximation is $13.45$ mg/L. The exact value is $10\exp(0.30)=13.4986$ mg/L. The first-order approximation is low by $0.4986$ mg/L, about $3.694\%$ of the exact prediction; adding the quadratic term leaves a difference of $-0.04859$ mg/L, about $-0.3600\%$ of the exact value. The improvement is substantial here, but the remaining error shows that including the quadratic term does not make the expansion exact.

When Do the Omitted Terms Matter?

A practical assessment should consider the random-effect scale in relation to model curvature, rather than judging the size of $\eta$ in isolation. For a single effect, the leading omitted term is approximately $\tfrac{1}{2}f''(0)\eta^2$. A steep slope can make the linear change large, while curvature determines how far that change departs from a straight line. With multiple effects, cross-partial derivatives matter: the terms involving $H_{jk}\eta_j\eta_k$ can add to or offset the diagonal contributions.

The approximation deserves particular scrutiny when plausible effects reach regions where the model changes rapidly, approaches a boundary, or behaves very differently from its behavior at zero. Examples include concentrations near zero, saturable processes over a wide range of exposure, and nonlinear transformations of random effects. It can also fail when the effect distribution is broad or has appreciable probability far from zero, even if zero is its mean.

Useful diagnostics are direct and local to the prediction being approximated:

Evaluate the exact nonlinear prediction and its first-order approximation at representative effect values, including values toward the tails of the assumed random-effect distribution.
Where feasible, calculate the second-order term and compare its magnitude with the prediction and with the linear effect. A large quadratic contribution is a warning, not a complete error assessment, because higher-order terms remain.
Repeat the comparison at different design points, such as sampling times. Curvature and sensitivity can vary substantially with time.
Check that derivatives are calculated with respect to the actual random-effect parameterization. A derivative for a log effect is not interchangeable with a derivative for the underlying positive parameter.

Numerical derivatives require their own checks: finite-difference results can depend on step size, particularly near boundaries or when predictions vary sharply. Compare derivative estimates across reasonable step sizes or verify them against analytic derivatives when possible. A visually plausible linearized curve at one effect value is not a diagnostic across an entire random-effect range.

Comparison and Limitations

Approach	What it does	Main consideration
Exact individual evaluation	Evaluates the original nonlinear model at the specified random effects.	Retains the model's nonlinearity but does not, by itself, summarize behavior over a random-effect distribution.
First-order Taylor expansion	Uses the prediction and gradient at zero effects.	Simple and interpretable locally; can be inaccurate when curvature or effect scale is substantial.
Second-order expansion	Adds the Hessian-based quadratic term.	Can improve a local approximation, but higher-order terms are still omitted and the Hessian must be computed reliably.
Numerical evaluation over effects	Evaluates the original model at selected or simulated effect values.	Can reveal failures of a local expansion, but conclusions depend on the range and weighting of evaluated values.

The first-order expansion is not a replacement for the nonlinear individual model. Nor does it alone specify a population-level approximation: that requires additional steps involving the distribution of the random effects. The present expansion supplies the local prediction and sensitivity components; it does not establish that a particular population summary is accurate.

Common Pitfall

Wrong reasoning: “The random effects have mean zero, so the omitted Taylor terms average to zero and the first-order approximation must be adequate.” Mean zero only implies that the linear term has zero expectation under appropriate integrability conditions. It does not eliminate quadratic terms: for example, a scalar second-order term is proportional to $\eta^2$, which is nonnegative and generally has a positive expectation when the effect varies. Nor does a zero mean prevent individual effects from being far from zero.

Correction: Assess curvature and the range of plausible effects directly. A centered distribution can still produce important departures from the prediction at zero. Check representative individual predictions and do not infer accuracy from centering alone.

Key takeaway: The first-order expansion at zero random effects is $f(\theta,0)+\nabla_\eta f(\theta,0)^T\eta$. It is justified locally by smoothness, but its usefulness depends on the curvature of the individual prediction over the effect range that matters.

Check Your Understanding

Try each question, then compare your reasoning with the brief solutions below.

For a scalar prediction, what two quantities determine the first-order Taylor approximation around zero random effects?
Suppose $f(\eta)=8+3\eta+4\eta^2$. Find its first-order approximation at zero and the omitted quadratic term at $\eta=0.25$.
Why does a mean-zero random-effect distribution not guarantee that a first-order expansion is accurate for individual predictions?
For a prediction with two random effects, why can checking each effect separately miss an important source of approximation error?

Solutions. (1) The prediction $f(0)$ and its gradient with respect to the random effects at zero; for a scalar effect, these are the intercept and slope. (2) Since $f(0)=8$ and $f'(0)=3$, the first-order approximation is $8+3\eta$. The omitted term is $4(0.25)^2=0.2500$, so the exact value is $8+0.75+0.25=9.000$, while the linear approximation is $8.750$. (3) Mean zero concerns the distributional average, not the size of each effect or the curvature of the prediction; effects can be far from zero and quadratic terms need not average to zero. (4) The Hessian can have nonzero cross-partial derivatives, so terms proportional to $\eta_1\eta_2$ contribute when both effects vary together.