Tutorials › Nonlinear Mixed-Effects Models (NLME) › Conditional Linearization in FOCE

Home  /  Tutorials  /  The Marginal Likelihood and Its Approximations

The Marginal Likelihood and Its Approximations · Tutorial 152 of 797

Conditional Linearization in FOCE

Learn how FOCE centers a prediction’s local linear approximation at each individual’s conditional mode, and why its derivatives can differ from those in a zero-centered expansion.

Advanced 12 min read

What You'll Learn

  • State the local smoothness conditions needed to expand an individual prediction at a conditional mode.
  • Write the FOCE prediction approximation using the mode-specific Jacobian.
  • Compare predictions and slopes from expansions at a conditional mode and at zero random effects.
  • Calculate first-order approximations for scalar and vector random effects.
  • Recognize when a local prediction expansion may be inaccurate or insufficient.

Why the Expansion Point Matters

An individual prediction in a nonlinear mixed-effects model can respond differently to a change in random effects at different locations in the random-effects space. A slope calculated at zero random effects need not describe the prediction well near the random-effect values most compatible with a particular individual’s data. FOCE addresses this local mismatch by expanding the prediction around that individual’s conditional mode.

The focus here is the approximation of the prediction itself. It is not yet the construction of the approximate subject likelihood. The distinction to keep in view is that a zero-centered expansion uses the prediction and its derivatives at zero, whereas conditional linearization uses the prediction and its derivatives at the individual-specific conditional mode.

Definition: Let \(y_i\) denote the observations for individual \(i\), let \(\boldsymbol{\eta}_i\) be that individual’s random-effects vector, and let \(f_{ij}(\boldsymbol{\theta},\boldsymbol{\eta}_i)\) be the model prediction for observation \(j\). The conditional mode, written \(\hat{\boldsymbol{\eta}}_i\), is a value of \(\boldsymbol{\eta}_i\) that maximizes the individual’s conditional log density given the observations, for the current model parameters. The FOCE conditional linearization of the prediction is its first-order Taylor approximation in \(\boldsymbol{\eta}_i\), evaluated at \(\hat{\boldsymbol{\eta}}_i\).

For the usual normal random-effects model, \(\boldsymbol{\eta}_i\sim N(\mathbf{0},\boldsymbol{\Omega})\), the conditional log density combines the individual’s data contribution with the random-effects density. The mode is therefore generally data-dependent; it is not simply a population average or a fixed substitute for the random effects. In the expansions below, treat the current \(\boldsymbol{\theta}\) as fixed and abbreviate \(f_{ij}(\boldsymbol{\theta},\boldsymbol{\eta}_i)\) as \(f_{ij}(\boldsymbol{\eta}_i)\).

Deriving the Conditional Linearization

Write \(\mathbf{h}=\boldsymbol{\eta}_i-\hat{\boldsymbol{\eta}}_i\). If the prediction is continuously differentiable in a neighborhood of the mode, its first-order expansion there has a remainder that is smaller than the size of \(\mathbf{h}\) as \(\mathbf{h}\) approaches zero:

$$ f_{ij}(\hat{\boldsymbol{\eta}}_i+\mathbf{h}) = f_{ij}(\hat{\boldsymbol{\eta}}_i) + \nabla_{\boldsymbol{\eta}_i} f_{ij}(\hat{\boldsymbol{\eta}}_i)^{\mathsf T}\mathbf{h} + o(\|\mathbf{h}\|). $$

Thus the FOCE linearized prediction is

$$ f_{ij}(\boldsymbol{\eta}_i) \approx f_{ij}(\hat{\boldsymbol{\eta}}_i) + \nabla_{\boldsymbol{\eta}_i} f_{ij}(\hat{\boldsymbol{\eta}}_i)^{\mathsf T} (\boldsymbol{\eta}_i-\hat{\boldsymbol{\eta}}_i). $$

For \(m_i\) predictions on individual \(i\), stack them into the vector \(\mathbf{f}_i\), and let \(\mathbf{J}_i(\hat{\boldsymbol{\eta}}_i)\) be the matrix whose \(j\)th row is \(\nabla_{\boldsymbol{\eta}_i} f_{ij}(\hat{\boldsymbol{\eta}}_i)^{\mathsf T}\). Then the same approximation is

$$ \mathbf{f}_i(\boldsymbol{\eta}_i) \approx \mathbf{f}_i(\hat{\boldsymbol{\eta}}_i) + \mathbf{J}_i(\hat{\boldsymbol{\eta}}_i) (\boldsymbol{\eta}_i-\hat{\boldsymbol{\eta}}_i). $$

This is a local approximation: its intercept is the prediction at the mode, and its slope is the Jacobian at the mode. If the prediction is twice continuously differentiable along the segment from \(\hat{\boldsymbol{\eta}}_i\) to \(\hat{\boldsymbol{\eta}}_i+\mathbf{h}\), the remainder can be written as an integral involving the Hessian:

$$ f_{ij}(\hat{\boldsymbol{\eta}}_i+\mathbf{h}) = f_{ij}(\hat{\boldsymbol{\eta}}_i) + \nabla f_{ij}(\hat{\boldsymbol{\eta}}_i)^{\mathsf T}\mathbf{h} + \int_0^1 (1-s)\, \mathbf{h}^{\mathsf T} H_{ij}(\hat{\boldsymbol{\eta}}_i+s\mathbf{h}) \mathbf{h}\,ds. $$

Here \(H_{ij}\) is the Hessian matrix of the prediction with respect to the random effects. If its operator norm is bounded by \(M\) along that segment, the absolute remainder is at most \(M\|\mathbf{h}\|^2/2\). This follows by bounding the integrand and using \(\int_0^1(1-s)\,ds=1/2\). The integral formula requires the stated second-derivative condition; continuous differentiability alone justifies the first-order expansion, not a Hessian remainder formula.

Conditional and Zero-Centered Derivatives

A zero-centered first-order expansion instead uses the prediction and gradient at \(\boldsymbol{\eta}_i=\mathbf{0}\):

$$ f_{ij}(\boldsymbol{\eta}_i) \approx f_{ij}(\mathbf{0}) + \nabla_{\boldsymbol{\eta}_i} f_{ij}(\mathbf{0})^{\mathsf T}\boldsymbol{\eta}_i. $$

The two approximations generally differ in both intercept and slope. Even if the conditional mode is close to zero, nonlinear predictions can have noticeably different local derivatives at the two points. For a vector of observations, each row of the conditional Jacobian is evaluated at the same individual-specific mode, while the rows may still differ because the predictions depend on different observation times or covariates.

The conditional mode and its derivatives are specific to the individual and to the current model parameters. When parameters change during estimation, the mode and the local expansion may change as well. This contrasts with using derivatives at zero, which are evaluated at a common random-effects value. The important distinction is the evaluation point, not a claim that the conditional derivative is always numerically larger or that it always improves every approximation.

Worked Example: A Scalar Exponential Prediction

Suppose a hypothetical prediction is \(f(\eta)=100\exp(\eta)\), in concentration units. For illustration, take the conditional mode to be \(\hat{\eta}=\log(1.2)=0.1823\), and evaluate the approximation at \(\eta=0.30\). Assume the prediction is differentiable on the interval between these values, as it is for this exponential function.

At the mode, \(f(\hat{\eta})=100(1.2)=120\), and \(f'(\eta)=100\exp(\eta)\) gives \(f'(\hat{\eta})=120\). The displacement is \(h=0.30-0.1823=0.1177\). Therefore the conditional linearization gives

$$ f(0.30)\approx 120+120(0.1177)=134.1. $$

The exact prediction is \(100\exp(0.30)=135.0\) concentration units. For comparison, the zero-centered expansion uses \(f(0)=100\) and \(f'(0)=100\), giving \(100+100(0.30)=130.0\) concentration units. In this example, the conditional expansion is closer to the exact value. That numerical result illustrates the effect of changing the expansion point; it is not a guarantee that conditional linearization is more accurate in every setting.

Worked Example: Two Random Effects and a Local Gradient

Consider the hypothetical prediction \(f(\eta_1,\eta_2)=80\exp(\eta_1)/(1+0.5\eta_2)\), in arbitrary concentration units, over values where the denominator is nonzero. Suppose the individual’s conditional mode is \((0.20,-0.10)\), and evaluate at \((0.25,0)\). The prediction is smooth along the segment connecting these points.

At the mode, \(80\exp(0.20)=97.71\) and \(1+0.5(-0.10)=0.95\), so \(f(0.20,-0.10)=97.71/0.95=102.9\). Its gradient is

$$ \nabla f(\eta_1,\eta_2) = \left( \frac{80\exp(\eta_1)}{1+0.5\eta_2}, -\frac{40\exp(\eta_1)}{(1+0.5\eta_2)^2} \right). $$

At the mode, this is approximately \((102.9,-54.13)\). The displacement is \((0.05,0.10)\), so the conditional approximation is

$$ 102.9+(102.9)(0.05)+(-54.13)(0.10) = 102.6. $$

The exact prediction at \((0.25,0)\) is \(80\exp(0.25)=102.7\). At zero, the prediction is \(80\), and the gradient is \((80,-40)\), so the zero-centered expansion at the same point gives \(80+(80)(0.25)+(-40)(0)=100.0\). The different results arise from both the changed intercept and the changed gradient.

Worked Example: A Saturating Prediction

For a hypothetical saturating prediction \(f(\eta)=100\exp(\eta)/(2+\exp(\eta))\), suppose the conditional mode is \(\hat{\eta}=\log(2)=0.6931\), and evaluate at \(\eta=0.80\). The denominator is positive throughout the segment, and the function is smooth there.

At the mode, \(f(\hat{\eta})=100(2)/(2+2)=50.00\). Differentiation gives \(f'(\eta)=200\exp(\eta)/(2+\exp(\eta))^2\), so \(f'(\hat{\eta})=25.00\). Since \(h=0.80-0.6931=0.1069\), the conditional approximation is \(50.00+25.00(0.1069)=52.67\). The exact value is \(100\exp(0.80)/(2+\exp(0.80))=52.67\) to two decimal places.

At zero, \(f(0)=100/3=33.33\) and \(f'(0)=200/9=22.22\). The zero-centered approximation is \(33.33+22.22(0.80)=51.11\). The mode-centered slope reflects the prediction’s local behavior near \(\log(2)\), rather than near zero.

Assumptions, Diagnostics, and Limitations

Conditional linearization requires a well-defined conditional mode for the individual and a prediction differentiable at that mode. To use the second-order remainder expression above, require continuous second derivatives along the relevant segment. The approximation is most defensible for random-effects values near the mode; it does not assert that the linear function describes the nonlinear prediction equally well over the entire random-effects space.

Several checks help reveal problems with the local approximation:

  • Check the derivatives. Compare analytic gradients with finite-difference estimates at the mode, using step sizes small enough to assess numerical stability.
  • Check local curvature. Compare the linearized prediction with the original prediction at nearby random-effects values. Large discrepancies over relevant displacements indicate that the local linear approximation may be inadequate.
  • Check the mode and its stability. A mode that changes substantially with small parameter changes, or that is poorly determined, can make the local expansion sensitive to numerical optimization.
  • Check the prediction’s range and domain. A linear approximation can produce values outside the prediction’s possible range, or values incompatible with its mathematical domain, even when the nonlinear model does not.

A common mistake is to calculate the gradient at zero, then call the result a conditional expansion because the individual’s mode was used elsewhere in the analysis. That is not conditional linearization: the gradient must be evaluated at \(\hat{\boldsymbol{\eta}}_i\), and the linear term must use \(\boldsymbol{\eta}_i-\hat{\boldsymbol{\eta}}_i\). Another mistake is to assume that a first-order expansion automatically has a Hessian remainder at some intermediate point. Such a remainder needs stronger smoothness conditions; continuous second derivatives along the segment justify the integral form given above.

Finally, this expansion concerns the prediction. If the residual variance also depends on random effects, linearizing the prediction alone does not account for that dependence in the full conditional data model. Conditional linearization is also local, so it can be inaccurate when relevant random-effects values are far from the mode or when curvature is strong. These limitations matter when the approximation is used as part of a larger likelihood calculation.

Key takeaway: FOCE linearizes each individual prediction at that individual’s conditional mode: use the prediction at the mode, the Jacobian at the mode, and the displacement from the mode. A zero-centered expansion instead uses the prediction and Jacobian at zero; nonlinear models generally make these local approximations different.

Check Your Understanding

Answer each question, then compare with the brief solutions.

  1. Write the conditional first-order approximation for a scalar prediction \(f(\eta)\) at mode \(\hat{\eta}\).
  2. For \(f(\eta)=20\exp(\eta)\) with \(\hat{\eta}=\log(1.5)\), calculate the conditional linear approximation at \(\eta=0.50\).
  3. For \(f(\eta)=\eta^2+4\), compare the conditional and zero-centered first-order approximations at \(\eta=1.2\) when \(\hat{\eta}=1\).
  4. What additional smoothness condition supports the displayed integral Hessian remainder?

Solutions. (1) \(f(\eta)\approx f(\hat{\eta})+f'(\hat{\eta})(\eta-\hat{\eta})\). (2) \(f(\hat{\eta})=30\), \(f'(\hat{\eta})=30\), and \(h=0.50-\log(1.5)=0.09453\), so the approximation is \(30+30(0.09453)=32.84\). (3) At the mode, \(f(1)=5\) and \(f'(1)=2\), giving \(5+2(0.2)=5.4\). At zero, \(f(0)=4\) and \(f'(0)=0\), giving \(4\). (4) The prediction must be twice continuously differentiable along the segment from the expansion point to the point being evaluated.

Previous

Next

Back to Tutorials