FOCE and Laplace Objective Differences

Why the Two Objectives Can Differ

FOCE and Laplace both use information about a subject’s conditional random-effect mode. The distinction is what happens after evaluating the subject’s contribution there. A mode-based FOCE construction uses the conditional fit at that mode, with the model’s relevant density normalizers. A Laplace construction also accounts for how sharply the joint subject contribution falls away from the mode. Its extra local-curvature term can change the objective even when the conditional modes are identical.

The comparison is useful when two methods produce different objective values or parameter estimates. It does not, by itself, say which method is more accurate for a particular model. The difference depends on the local curvature, and Laplace is still an approximation to the marginal likelihood unless the log integrand is quadratic.

Definition: For subject $i$, let $y_i$ denote the observations, $\eta_i$ the random effects with distribution $N(0,\Omega)$, and $\theta$ the population parameters. Define the subject-level integrand and its negative log kernel by $$ q_i(\eta_i;\theta)=p(y_i\mid\eta_i,\theta)\,p(\eta_i\mid\Omega), \qquad h_i(\eta_i;\theta)=-\log q_i(\eta_i;\theta). $$ A conditional mode $\hat\eta_i$ minimizes $h_i$, or equivalently maximizes $q_i$. The Hessian $H_i$ is the matrix of second derivatives of $h_i$ with respect to $\eta_i$, evaluated at $\hat\eta_i$.

The subject’s marginal likelihood is the integral $L_i(\theta)=\int q_i(\eta_i;\theta)\,d\eta_i$. In a mode-based comparison, the shared contribution is the integrand evaluated at the conditional mode. Using the convention of a deviance, or negative twice log likelihood, write this contribution as $J_{i,\mathrm{mode}}=-2\log q_i(\hat\eta_i;\theta)$. Terms that are constant across parameter values may be omitted from an optimization objective, but must be handled consistently when comparing reported values.

FOCE constructs a conditional approximation using a first-order expansion around the conditional mode and uses the resulting mode-based subject contribution. Its precise implementation can retain density normalizers associated with the linearized conditional model. Those normalizers are not the same thing as the Laplace correction from the Hessian of the full negative log integrand. Laplace’s additional term is specifically about the volume of the integrand near its mode.

What Laplace Adds

Suppose $h_i$ is smooth near an interior conditional mode, the mode is locally unique, and $H_i$ is positive definite. A second-order Taylor expansion gives a local quadratic approximation:

h_i(\eta_i)\approx h_i(\hat\eta_i) +\frac{1}{2}(\eta_i-\hat\eta_i)^\mathsf{T} H_i(\eta_i-\hat\eta_i).

Exponentiating the negative of this expression approximates $q_i$ by a Gaussian-shaped function. Integrating that function over the $d$-dimensional random-effect vector gives

L_i(\theta)\approx q_i(\hat\eta_i;\theta)(2\pi)^{d/2}|H_i|^{-1/2},

where $d$ is the dimension of $\eta_i$, and $|H_i|$ is the determinant of the Hessian. Taking negative twice the logarithm yields the Laplace subject objective:

J_{i,\mathrm{Laplace}} \approx J_{i,\mathrm{mode}}+\log|H_i|-d\log(2\pi).

Thus, when both constructions use the same mode contribution and the same density conventions, their objective difference is $J_{i,\mathrm{Laplace}}-J_{i,\mathrm{mode}}\approx\log|H_i|-d\log(2\pi)$. The determinant is not an arbitrary penalty: it adjusts for the width of the integrand around its peak. A sharply concentrated integrand has a larger curvature determinant and, other things equal, a smaller local volume.

Key distinction: FOCE and Laplace share a conditional-mode contribution in this comparison. Laplace adds the local-curvature adjustment $\log|H_i|-d\log(2\pi)$. A determinant already present as a normalizing constant in a conditional density is not automatically this Hessian correction.

Worked Objective Comparisons

Worked Example: A One-Dimensional Gaussian Subject

Consider hypothetical data consisting of one observation $y=1.00$, with $\theta=0$, $\eta\sim N(0,0.25)$, and $y\mid\eta\sim N(\eta,0.16)$. The conditional model is linear and both distributions are Gaussian. Find the conditional mode, compare the mode contribution with the Laplace objective, and check the marginal likelihood.

The negative log kernel, including its density normalizers, is $h(\eta)=\tfrac12\log(2\pi\cdot0.16)+(1-\eta)^2/(2\cdot0.16)+\tfrac12\log(2\pi\cdot0.25)+\eta^2/(2\cdot0.25)$. Setting its derivative to zero gives $(1-\eta)/0.16-\eta/0.25=0$, so $\hat\eta=0.25/(0.25+0.16)=0.6098$.

The Hessian is constant: $H=1/0.16+1/0.25=6.25+4.00=10.25$. At the mode, the deviance contribution is $J_{\mathrm{mode}}=\log(2\pi\cdot0.16)+(1-0.6098)^2/0.16+\log(2\pi\cdot0.25)+0.6098^2/0.25\approx2.896$. For $d=1$, the Laplace adjustment is $\log(10.25)-\log(2\pi)=2.327-1.838=0.489$. Therefore $J_{\mathrm{Laplace}}\approx2.896+0.489=3.385$.

As a check, integrating the two Gaussian distributions gives $y\sim N(0,0.25+0.16)$. Its marginal deviance at $y=1.00$ is $\log(2\pi\cdot0.41)+1^2/0.41\approx0.946+2.439=3.385$. Laplace is exact here because the log integrand is quadratic. The mode-only contribution is not the integrated marginal deviance.

Worked Example: A Scalar Curvature Correction

Suppose a hypothetical subject has a one-dimensional conditional mode and a computed mode contribution $J_{\mathrm{mode}}=7.200$. At that mode, the negative log integrand has Hessian $H=25.0$. Assume a smooth log integrand and a locally unique interior mode with positive curvature. Calculate the Laplace objective.

Here $d=1$, so $J_{\mathrm{Laplace}}\approx7.200+\log(25.0)-\log(2\pi)$. Using $\log(25.0)=3.219$ and $\log(2\pi)=1.838$, the correction is $1.381$, and $J_{\mathrm{Laplace}}\approx8.581$.

The positive correction reflects the specified curvature on this scale. It is not a test statistic or a measure of model fit by itself. It is the change from the mode contribution to the local Laplace approximation under the stated density and objective conventions.

Worked Example: A Two-Dimensional Curvature Correction

For a hypothetical subject with two random effects, suppose the mode contribution is $J_{\mathrm{mode}}=12.00$ and the Hessian at the mode is $H=\begin{pmatrix}9&3\\3&4\end{pmatrix}$. Assume the mode is interior and the Hessian is positive definite. Find the determinant correction and Laplace objective.

The determinant is $|H|=9(4)-3(3)=27$. The first leading principal minor is $9>0$, and the determinant is positive, so this symmetric matrix is positive definite. With $d=2$, the adjustment is $\log(27)-2\log(2\pi)=3.296-3.676=-0.380$. Hence $J_{\mathrm{Laplace}}\approx12.00-0.380=11.62$.

A negative adjustment is possible: the determinant term is not constrained to be positive. It reflects curvature in all random-effect directions, including the off-diagonal association in this Hessian. Reporting only its diagonal entries would give the wrong determinant and therefore the wrong adjustment.

Assumptions, Diagnostics, and Alternatives

The local Laplace calculation relies on more than the existence of a numerical optimizer. The conditional mode should be interior and sufficiently well identified; $h_i$ should be twice differentiable nearby; and $H_i$ should be positive definite for a local minimum of $h_i$. The approximation is most persuasive when the integrand is concentrated near that mode and is well described by its quadratic expansion over the region carrying appreciable probability.

Useful checks include confirming that the gradient of $h_i$ is close to zero at $\hat\eta_i$, checking the Hessian’s symmetry and positive definiteness, and verifying that the determinant is finite and numerically stable. A near-zero eigenvalue indicates weak curvature in at least one direction; a negative eigenvalue means the computed point is not a strict local minimum. These findings call for checking optimization, parameterization, and model identification rather than blindly applying the formula. Sensitivity to starting values can also reveal that different modes are being found.

FOCE and Laplace are not the only ways to approximate the subject integral. A first-order approximation expands individual predictions around the population random-effect mean rather than conditioning the expansion on each subject’s mode. Numerical quadrature can directly approximate the integral, though cost rises with random-effect dimension. Simulation-based integration is another alternative when computational resources and a suitable integration strategy are available. These methods differ in approximation and computational demands; none is guaranteed to agree with Laplace in a strongly nonquadratic problem.

Construction	Role of the conditional mode	Curvature treatment	Main caution
FOCE, in a mode-based comparison	Evaluates a conditional approximation at the subject-specific mode	Does not add the full Hessian determinant correction shown for Laplace	First-order conditional linearization can be inadequate when the local approximation is poor
Laplace	Evaluates the joint subject integrand at its mode	Uses the determinant of the local Hessian	A local quadratic shape may not represent the full integrand well
Numerical quadrature or simulation-based integration	May use a mode to guide computation, but integrates beyond the mode alone	Approximates the integral by other numerical means	Can be computationally demanding, especially as dimension grows

The determinant correction also has limitations as a diagnostic. Its value depends on the scale and parameterization of the random effects, while the full likelihood calculation must transform consistently with that parameterization. A large difference between mode-based and Laplace objectives can flag substantial curvature contributions, but it does not establish that Laplace is accurate: asymmetry, broad tails, or multiple important modes can undermine a single local quadratic approximation. These are reasons to inspect the model and consider other integration checks, not reasons to interpret an objective difference as a direct measure of bias.

Common Mistake

Wrong reasoning: “Laplace and FOCE found the same conditional modes, so their objectives should be the same.” The mode fixes the height of the integrand, not its local width. Correction: compare the shared mode contribution and then account for Laplace’s $\log|H_i|-d\log(2\pi)$ term, keeping any other density normalizers consistent. Also do not label every determinant in a software objective as the Laplace Hessian correction; identify what quantity that determinant normalizes.

Check Your Understanding

Work through these questions, then check the brief solutions.

What does the Laplace determinant term represent that a mode evaluation alone does not?
For $d=1$, $J_{\mathrm{mode}}=5.00$, and $H=2$, calculate the Laplace objective to three significant figures.
For $d=2$, explain why a negative value of $\log|H|-2\log(2\pi)$ is not automatically an error.
Name two conditions needed for the stated local Laplace calculation and one diagnostic for each.

Solutions. (1) It adjusts for the local volume, or width, of the integrand around its mode. (2) $J_{\mathrm{Laplace}}\approx5.00+\log(2)-\log(2\pi)=5.00+0.6931-1.838=3.86$. (3) The correction is a difference of terms and is not required to be positive; check that the Hessian is positive definite and its determinant is computed correctly. (4) For example, an interior stationary mode can be checked by verifying a small gradient, and positive-definite curvature can be checked from the Hessian’s eigenvalues or principal minors. A poor check indicates that the local minimum or curvature calculation needs investigation.