Asymmetric Curvature and Laplace Error

Why Curvature Asymmetry Matters

For an individual in a nonlinear mixed-effects model, the marginal likelihood contribution is obtained by integrating over the individual’s random effect. The Laplace approximation replaces the log of the integrand near its conditional mode with a quadratic function. This is effective when the mode is locally well behaved and the important probability mass is concentrated in a region that the quadratic describes adequately.

A quadratic is symmetric around its center. But a nonlinear log integrand can fall away from its mode at different rates to the left and right. In that case, a single curvature describes the infinitesimal neighborhood of the mode, but may not describe the wider region contributing materially to the integral. The resulting error is not determined by the curvature at the mode alone.

Definition: For a one-dimensional log integrand $h(\eta)$ with an interior mode $\hat{\eta}$, define its effective curvature at displacement $u\ne 0$ by $$ J_{\mathrm{eff}}(u)=-\frac{2\{h(\hat{\eta}+u)-h(\hat{\eta})-h'(\hat{\eta})u\}}{u^2}. $$ At a stationary mode, $h'(\hat{\eta})=0$. If $h$ is twice differentiable at the mode, then $J_{\mathrm{eff}}(u)\to -h''(\hat{\eta})$ from either side as $u\to 0$. Different values at finite positive and negative displacements indicate that the log integrand bends differently over those neighborhoods.

The limit matters: if $h$ is twice continuously differentiable at the mode, its left and right second derivatives there are equal. Thus “asymmetric curvature” in a smooth model usually means curvature that changes differently over finite neighborhoods on the two sides, not two different Hessians at the exact same point. A model with different one-sided second derivatives at the mode is a useful illustration of asymmetry, but it does not meet the usual smoothness condition for the standard Laplace approximation.

Quadratic Laplace Approximation and Its Conditions

For subject $i$, write the one-dimensional log integrand as $h_i(\eta_i;\theta)=\log p(y_i\mid\eta_i,\theta)+\log p(\eta_i\mid\Omega)$, where $y_i$ denotes the subject’s data, $\eta_i$ the scalar random effect, and $\theta$ the model’s fixed-effect parameters. The marginal contribution is $L_i=\int \exp\{h_i(\eta_i;\theta)\}\,d\eta_i$. Let $\hat{\eta}_i$ be an interior conditional mode and define $J_i=-h_i''(\hat{\eta}_i;\theta)$.

Laplace approximation: If the mode is unique and interior, $h_i$ is sufficiently smooth near it, $J_i>0$, and the integral is dominated by a neighborhood where the quadratic expansion is adequate, then $$ L_i\approx \exp\{h_i(\hat{\eta}_i;\theta)\}\sqrt{\frac{2\pi}{J_i}}. $$ The factor $\sqrt{2\pi/J_i}$ is the area adjustment implied by the local Gaussian approximation. It does not account for substantial nonquadratic behavior or important mass far from the mode.

The assumptions are substantive. A flat or saddle-like mode has $J_i\le 0$, so this expression is not a valid local Gaussian approximation. A boundary mode, a nondifferentiable log integrand, or substantial mass in distant regions also undermines the usual argument. Even with a unique mode and positive curvature, strong changes in effective curvature across the region containing most of the integral can make the approximation inaccurate.

A useful comparison is between the local curvature $J_i$ and effective curvatures at displacements of practical interest. For example, the local quadratic predicts a drop of $J_i u^2/2$ in log integrand at displacement $u$. Comparing this prediction with the actual drops on both sides reveals whether the local Gaussian shape is representative. The relevant displacement is not arbitrary: it should include the region in which the quadratic approximation would place appreciable mass.

How Asymmetry Enters the Approximation Error

Set $u=\eta_i-\hat{\eta}_i$, and temporarily suppress the subject and parameter subscripts. Suppose the log integrand has a smooth local expansion $$ h(\hat{\eta}+u)=h(\hat{\eta})-\frac{J u^2}{2}+\frac{\kappa_3u^3}{6}+\frac{\kappa_4u^4}{24}+\cdots, $$ where $J=-h''(\hat{\eta})>0$, $\kappa_3=h'''(\hat{\eta})$, and $\kappa_4=h''''(\hat{\eta})$. The cubic term makes the log integrand asymmetric around the mode. Although its first-order contribution integrates to zero over a symmetric Gaussian approximation, it does not follow that asymmetry has no effect on the integral.

To see why, factor out the quadratic term and expand the exponential of the remaining terms. Under the centered Gaussian density proportional to $\exp(-Ju^2/2)$, odd moments vanish, while $E(u^4)=3/J^2$ and $E(u^6)=15/J^3$. The quartic term contributes $\kappa_4 E(u^4)/24$; the square of the cubic term contributes $\kappa_3^2 E(u^6)/72$. Therefore the first correction to the quadratic approximation is

L \approx \exp\{h(\hat{\eta})\}\sqrt{\frac{2\pi}{J}} \left[1+\frac{\kappa_4}{8J^2}+\frac{5\kappa_3^2}{24J^3}+\cdots\right].

This is a local expansion, not a universal error bound. Its terms are informative when the integral is concentrated near the mode and higher derivatives are controlled in that region. The positive squared-cubic contribution shows how skewness can affect the integral even though the cubic term itself is odd. The quartic contribution can be positive or negative. When the standardized derivative terms are not small, omitted terms and behavior outside the local neighborhood may be important, so the correction should not be treated as a guaranteed improvement.

Worked Example: Comparing Curvature on the Two Sides

Consider a hypothetical one-dimensional log integrand with mode $\hat{\eta}=0$, $h(0)=0$, and $h'(0)=0$. Suppose $h''(0)=-4$, while its values at $-0.5$ and $0.5$ are $-0.45$ and $-0.65$, respectively. The local curvature is $J=4$.

At $u=-0.5$, the effective curvature is $J_{\mathrm{eff}}(-0.5)=-2(-0.45)/(0.5^2)=3.60$. At $u=0.5$, it is $J_{\mathrm{eff}}(0.5)=-2(-0.65)/(0.5^2)=5.20$.

The local quadratic predicts $h(\pm0.5)-h(0)\approx-4(0.5^2)/2=-0.500$ on both sides. The actual drops are $-0.450$ and $-0.650$: the left side is less sharply curved over this displacement, and the right side is more sharply curved. The local Laplace contribution is $\exp(0)\sqrt{2\pi/4}=1.253$. This is a local approximation to the integral, not the exact integral; the two curvature comparisons show why the single value $J=4$ may not adequately represent the contributing region.

A Limiting Example: Different One-Sided Curvatures at the Mode

A piecewise quadratic example makes the effect of unequal left- and right-side curvature exactly calculable. It is deliberately nonsmooth in second derivative at the mode, so it illustrates a failure of the ordinary smooth Laplace assumptions rather than a routine application of the formula.

Worked Example: Exact Integral Versus a Single Quadratic

Let the hypothetical log integrand be $h(x)=-x^2/2$ for $x<0$ and $h(x)=-2x^2$ for $x\ge 0$. The mode is $x=0$. The left side has curvature 1 and the right side has curvature 4.

The exact integral is the sum of two half-Gaussian integrals: $$ \int_{-\infty}^0 e^{-x^2/2}\,dx+\int_0^\infty e^{-2x^2}\,dx =\sqrt{\frac{\pi}{2}}+\sqrt{\frac{\pi}{8}} =1.2533+0.6267=1.880. $$

There is no unique second derivative at the mode, and so no ordinary Laplace approximation based on a single Hessian. If one uses the left curvature $J=1$, the formula gives $\sqrt{2\pi}=2.507$, which is 33.3% above the exact integral. If one uses the right curvature $J=4$, it gives $\sqrt{2\pi/4}=1.253$, which is 33.3% below the exact integral. An arbitrary average curvature $J=2.5$ gives $\sqrt{2\pi/2.5}=1.585$, 15.7% below the exact value.

Averaging the curvatures happens to reduce the error here, but there is no general rule that an average curvature is the correct substitute. The calculation demonstrates that neither one-sided shape alone describes the full area. Because the second derivative jumps at the mode, these results diagnose the limitation of a quadratic summary; they do not establish the standard Laplace formula as valid for this nonsmooth integrand.

Using Derivative Corrections Carefully

Worked Example: A Local Higher-Order Error Check

Suppose a hypothetical smooth log integrand has $h(\hat{\eta})=0$, $J=4$, $\kappa_3=2$, and $\kappa_4=8$, in consistent units for the scalar random effect. The leading quadratic Laplace value is $\sqrt{2\pi/4}=1.2533$.

The quartic correction is $\kappa_4/(8J^2)=8/(8\cdot16)=0.06250$. The cubic contribution is $5\kappa_3^2/(24J^3)=5(4)/(24\cdot64)=0.01302$. Together, these terms give a correction factor $1+0.06250+0.01302=1.07552$, and the corrected local expansion is $1.2533(1.07552)=1.348$.

This calculation says that the displayed higher-order terms suggest a 7.55% upward correction to the quadratic value. It does not prove that the exact integral is 1.348: the remaining terms and nonlocal tails have not been evaluated. If the derivative terms are large, or if curvature changes rapidly across the region with appreciable mass, the local series itself may be unreliable.

Diagnostics, Alternatives, and Limitations

In practice, curvature asymmetry is a warning that local Gaussian shape may not represent the full individual contribution. Useful checks include evaluating $h$ at a few positive and negative displacements from $\hat{\eta}_i$, comparing observed log-integrand drops with $-J_i u^2/2$, and checking whether the mode and curvature are stable under tighter numerical optimization. A large difference in $J_{\mathrm{eff}}(u)$ across sides, a visibly unequal tail decline, or sensitivity to the displacement chosen indicates that the quadratic summary may be fragile.

These checks are not substitutes for evaluating approximation error. A good local fit does not rule out a distant region of material integrand mass, and a poor fit at a displacement where the integrand is already negligible may have little effect on the integral. Interpretation should therefore consider both the shape and the location of the mass. In a one-dimensional problem, plotting the log integrand and its fitted quadratic is often a particularly direct diagnostic.

FOCE and Laplace also summarize the individual contribution differently. As discussed in the earlier material on FOCE and Laplace objective differences, FOCE uses a conditional linearization, whereas Laplace uses local curvature to approximate integration. Neither is automatically accurate when the conditional log integrand is strongly nonquadratic. If an exact integral is available for a special model, it provides a direct benchmark; otherwise, numerical integration can be used as a check when feasible. These checks compare the marginal contribution itself rather than assuming that a local curvature diagnostic fully determines the error.

The main limitation is that a local quadratic contains only the mode height and second derivative. It cannot encode left–right shape differences, higher-order features, or separated regions of mass. The usual asymptotic justification is strongest when the integrand becomes increasingly concentrated around a smooth, well-defined mode, so that the local neighborhood dominates and standardized higher-order contributions shrink. For a fixed subject with weak concentration or pronounced skewness, that justification may be poor.

Common Mistake

Common mistake: “The Hessian is negative at the mode, so the Laplace approximation must be accurate.” A negative second derivative establishes local concavity and supplies a positive $J$; it does not establish that curvature is nearly constant over the region contributing to the integral. The correction is to check the log-integrand shape on both sides, assess how concentrated the integral is, and treat the quadratic formula as an approximation whose accuracy depends on those features.

Key takeaway: In a smooth one-dimensional problem, the left and right curvatures agree in the infinitesimal limit at an interior mode. Laplace error arises when the curvature and higher-order shape differ over the wider region that contributes to the integral. The local Hessian is necessary for the approximation, but not sufficient to guarantee its accuracy.

Check Your Understanding

Try each question, then compare your reasoning with the brief solutions.

Why can a smooth log integrand have asymmetric curvature near a mode even though it has only one second derivative at the mode?
A log integrand has $h(\hat{\eta})=0$ and $J=9$. Calculate its quadratic Laplace contribution.
For $J=4$, $\kappa_3=0$, and $\kappa_4=-16$, calculate the quartic correction in the leading expansion. What does its sign indicate, and what does it not establish?
Why is a piecewise quadratic log integrand with unequal one-sided curvatures not a standard smooth Laplace example?

Solutions. (1) The second derivative agrees in the limit as displacement approaches zero, but curvature can change differently over finite positive and negative displacements. (2) The contribution is $\exp(0)\sqrt{2\pi/9}=0.8355$. (3) The correction is $-16/(8\cdot16)=-0.125$, or $-12.5\%$ relative to the quadratic term at this order. It indicates a downward quartic contribution in the local expansion, not the exact integral’s total error. (4) The second derivative does not exist as a single continuous value at the mode, violating the usual smoothness condition used to justify the quadratic expansion.