Exactness of Laplace Integration for Gaussian Integrands

Why Exactness Is a Useful Check

In a generalized linear mixed model, integrating over random effects produces a marginal likelihood. The Laplace formula replaces that integral by a Gaussian integral formed from the log integrand’s value and curvature at its mode. When the log integrand is exactly quadratic over the entire integration domain, this replacement is not an approximation: it reproduces the integral exactly. That fact provides both a theoretical result and a practical check on calculations.

The result depends on the whole log integrand, not merely on how closely it looks quadratic near its mode. A local Taylor expansion is enough to define the Laplace calculation, but exactness is guaranteed when the quadratic form describes the integrand everywhere on the domain. The distinction matters: this tutorial establishes a sufficient condition for exactness, not a necessary condition.

The Exactness Result

Let $b$ be a $q$-dimensional integration variable, and let $h(b)$ be the log integrand. Suppose the integral is over all of $\mathbb{R}^q$, and for some scalar $c$, vector $m$, and symmetric positive-definite matrix $A$,

h(b)=c-\frac{1}{2}(b-m)^{T}A(b-m).

The unique maximum is at $\hat{b}=m$, because the quadratic term is nonpositive and equals zero only at $b=m$. The Hessian matrix of second derivatives at the mode is $H=-A$, so $-H=A$ is positive definite. In this notation, the Laplace formula for the integral is

\exp\{h(\hat{b})\}(2\pi)^{q/2}|-H|^{-1/2},

where $|-H|$ denotes the determinant of $-H$. Under the stated conditions, this expression equals the integral exactly.

Theorem: If $h(b)=c-\frac{1}{2}(b-m)^TA(b-m)$ throughout $\mathbb{R}^q$, where $A$ is symmetric positive definite, then $$ \int_{\mathbb{R}^q}\exp\{h(b)\}\,db = \exp\{h(m)\}(2\pi)^{q/2}|A|^{-1/2}. $$ This is the Laplace formula evaluated at the unique mode $m$, with negative Hessian $-H=A$.

Why the Formula Is Exact

The justification reduces the multivariate integral to a product of one-dimensional Gaussian integrals. Since $A$ is symmetric positive definite, it has an orthogonal diagonalization $A=Q\Lambda Q^T$, where $Q^TQ=I$ and $\Lambda$ is diagonal with positive diagonal entries $\lambda_1,\ldots,\lambda_q$. Set $u=Q^T(b-m)$. This is a translation followed by an orthogonal change of coordinates, so the absolute value of its Jacobian determinant is one. Also,

(b-m)^TA(b-m) = u^T\Lambda u = \sum_{j=1}^{q}\lambda_j u_j^2.

Thus the exponent separates into $q$ one-dimensional terms. The standard Gaussian integral $\int_{-\infty}^{\infty}\exp(-\lambda u^2/2)\,du=\sqrt{2\pi/\lambda}$, valid for $\lambda>0$, gives

\begin{aligned} \int_{\mathbb{R}^q}\exp\{h(b)\}\,db &=\exp(c)\prod_{j=1}^{q}\int_{-\infty}^{\infty} \exp\left(-\frac{\lambda_j u_j^2}{2}\right)\,du_j\\ &=\exp(c)\prod_{j=1}^{q}\left(\frac{2\pi}{\lambda_j}\right)^{1/2}\\ &=\exp(c)(2\pi)^{q/2}\left(\prod_{j=1}^{q}\lambda_j\right)^{-1/2}\\ &=\exp(c)(2\pi)^{q/2}|A|^{-1/2}. \end{aligned}

At the mode $m$, $h(m)=c$, and $-H=A$. Substituting these into the Laplace formula gives the same expression. This proves the result, including the determinant and the factor $(2\pi)^{q/2}$.

Worked Example: A One-Dimensional Quadratic Log Integrand

Consider the hypothetical integral

I=\int_{-\infty}^{\infty} \exp\left\{-1.7-\frac{1}{2}\,4(b-2)^2\right\}\,db.

Here $q=1$, $c=-1.7$, $m=2$, and $A=4$, which is positive. The log integrand has its unique mode at $\hat b=2$; its second derivative is $H=-4$, so the negative curvature is $-H=4$. These satisfy the theorem’s conditions: the domain is the full real line and the log integrand is quadratic everywhere.

The integral, evaluated by the Laplace formula, is

\begin{aligned} I &=\exp\{h(2)\}(2\pi)^{1/2}|4|^{-1/2}\\ &=\exp(-1.7)\frac{\sqrt{2\pi}}{2}\\ &\approx 0.1827(1.2533)\\ &\approx 0.2290. \end{aligned}

The value is exact, not a local approximation: the exponent in the integral is precisely the quadratic used by the Laplace formula at every $b$.

Checking a Normalized Gaussian Density

A multivariate Gaussian density provides a particularly direct verification because its normalizing constant is chosen to make its integral equal to one. Let $b\sim N(\mu,\Sigma)$, where $\Sigma$ is a symmetric positive-definite $q\times q$ covariance matrix. Its log density is

h(b)=-\frac{q}{2}\log(2\pi)-\frac{1}{2}\log|\Sigma| -\frac{1}{2}(b-\mu)^T\Sigma^{-1}(b-\mu).

This is quadratic over all of $\mathbb{R}^q$. Its mode is $\mu$, and its Hessian is $H=-\Sigma^{-1}$. Since $\Sigma$ is positive definite, so is $\Sigma^{-1}$; hence the mode has negative-definite Hessian. At the mode,

\exp\{h(\mu)\}=(2\pi)^{-q/2}|\Sigma|^{-1/2}, \qquad |-H|=|\Sigma^{-1}|=|\Sigma|^{-1}.

Therefore the Laplace value is

(2\pi)^{-q/2}|\Sigma|^{-1/2} (2\pi)^{q/2}|\Sigma|^{1/2}=1.

This verifies the normalization directly. It also shows why the determinant term and the Gaussian density’s normalizing constant must both be retained: they cancel precisely.

Worked Example: A Two-Dimensional Normal Density

Take a hypothetical bivariate normal random effect with mean $\mu=(1,-1)^T$ and covariance matrix

\Sigma= \begin{pmatrix} 4&1\\ 1&2 \end{pmatrix}, \qquad |\Sigma|=4(2)-1(1)=7.

The matrix is positive definite: its first leading principal minor is $4>0$, and its determinant is $7>0$. Thus the Gaussian density is well defined and has a unique mode at $\mu$. For $q=2$, its value at the mode is

\exp\{h(\mu)\} =(2\pi)^{-1}7^{-1/2} =\frac{1}{2\pi\sqrt{7}}.

The Hessian is $-\Sigma^{-1}$, so $|-H|=|\Sigma^{-1}|=1/7$. Applying the Laplace formula gives

\begin{aligned} \int_{\mathbb{R}^2}\exp\{h(b)\}\,db &=\frac{1}{2\pi\sqrt{7}}(2\pi)^{2/2}(1/7)^{-1/2}\\ &=\frac{1}{2\pi\sqrt{7}}(2\pi)(\sqrt{7})\\ &=1. \end{aligned}

The value agrees with the required integral of a normalized probability density. The computation also checks the determinant exponent: using $|-H|^{1/2}$ instead of $|-H|^{-1/2}$ would not produce one.

A Gaussian Random-Effects Integral

Exactness also appears in Gaussian random-effects calculations when both the conditional data model and the random-effects distribution are Gaussian. Consider one hypothetical observation $y=1.2$, with conditional model $Y\mid b\sim N(b,v)$, $v=0.5$, and random effect $b\sim N(0,t)$, $t=1.5$. The marginal density of $y$ is the integral of the product of these two normalized densities. Its log integrand is

h(b)=-\frac{1}{2}\log(2\pi v)-\frac{(y-b)^2}{2v} -\frac{1}{2}\log(2\pi t)-\frac{b^2}{2t}.

This is quadratic over the full real line. Differentiating and setting the derivative to zero gives the mode:

h'(b)=\frac{y-b}{v}-\frac{b}{t}=0, \qquad \hat b=\frac{y/v}{1/v+1/t}.

Worked Example: Integrating a Gaussian Random Effect

Substitute $y=1.2$, $v=0.5$, and $t=1.5$. Then

\hat b=\frac{1.2/0.5}{1/0.5+1/1.5} =\frac{2.4}{2+0.6667}=0.9000, \qquad H=-\left(\frac{1}{v}+\frac{1}{t}\right)=-2.6667.

The negative Hessian is positive, as required for a maximum. At the mode, the two squared terms in $h$ sum to \[ \frac{(1.2-0.9)^2}{0.5}+\frac{(0.9)^2}{1.5} =0.1800+0.5400=0.7200. \] Thus the quadratic contribution to $h(\hat b)$ is $-0.3600$, and the constants give \[ \exp\{h(\hat b)\} =\frac{\exp(-0.3600)}{2\pi\sqrt{vt}} =\frac{\exp(-0.3600)}{2\pi\sqrt{0.75}} \approx 0.1283. \] The Laplace multiplier is $\sqrt{2\pi/2.6667}\approx1.535$, so

\int_{-\infty}^{\infty}\exp\{h(b)\}\,db \approx 0.1283(1.535)=0.1970.

As a check, integrating the Gaussian model gives $Y\sim N(0,v+t)=N(0,2)$, so its density at $y=1.2$ is \[ \frac{1}{\sqrt{2\pi(2)}}\exp\left(-\frac{1.2^2}{2(2)}\right) =\frac{1}{\sqrt{4\pi}}\exp(-0.3600) \approx 0.1970. \] The values agree, up to rounding. The integral is exact because the entire log integrand is quadratic and the full normalizing constants were included.

What Exactness Does—and Does Not—Claim

The theorem gives a reliable sufficient condition: a quadratic log integrand on the full integration domain, with positive-definite negative curvature, guarantees that the Laplace value equals the integral. These conditions are not necessary for equality. A nonquadratic integrand can have the same integral as its Laplace value by coincidence.

For example, start with $h_0(x)=-x^2/2$. Choose a nonzero smooth function $g$ supported away from zero such that \[ \int_{-\infty}^{\infty}e^{-x^2/2}g(x)\,dx=0. \] For sufficiently small nonzero $\epsilon$, $1+\epsilon g(x)$ remains positive, and define \[ h(x)=-x^2/2+\log(1+\epsilon g(x)). \] Because $g$ is zero near zero, $h$ has the same mode and curvature at zero as $h_0$; choosing $\epsilon$ sufficiently small also keeps zero as its unique mode. Yet $h$ is nonquadratic, while \[ \int e^{h(x)}\,dx =\int e^{-x^2/2}\{1+\epsilon g(x)\}\,dx =\int e^{-x^2/2}\,dx. \] Its Laplace value is therefore exact as well. This is an accidental equality, not an instance of the quadratic theorem.

The theorem also does not say that every integrand with a quadratic expression near its mode is integrated exactly. The global domain and the full log integrand matter. For example, restricting a quadratic integrand to a finite interval generally gives a truncated Gaussian integral, not the full Gaussian integral used in the Laplace formula. Likewise, if the negative Hessian is not positive definite, the stated Gaussian integral is not a finite real-valued Laplace expression.

Practical Checks and Common Pitfalls

When using a Gaussian case to check a Laplace calculation, confirm that the log integrand includes every normalizing term required for the integral being evaluated. In a random-effects marginal likelihood, dropping the random-effects density’s normalizing constant changes the integral. Check the curvature sign, use the determinant of the negative Hessian, and verify that the integration is over the full space assumed by the formula. If the integrand is a normalized density, its integral must be one.

A common mistake is to conclude, “The Hessian at the mode is negative definite, so the Laplace result is exact.” Negative-definite curvature establishes a locally well-behaved maximum and makes the Gaussian factor well defined; it does not establish that the log integrand is quadratic everywhere. The correction is to distinguish the local curvature calculation from the global condition in the theorem. For nonquadratic GLMM integrands, the Laplace formula is generally an approximation; exactness is guaranteed by the theorem only when its stated quadratic conditions hold.

For a Gaussian integrand, direct analytic integration and the Laplace formula are equivalent ways to evaluate the same integral. The analytic form is a useful benchmark when available. For a non-Gaussian integrand, the Laplace formula remains a mode-and-curvature approximation, while numerical integration methods provide an alternative way to evaluate the integral. Neither a single local curvature calculation nor agreement in one test case proves general exactness.

Key takeaway: A globally quadratic log integrand over $\mathbb{R}^q$, with a positive-definite quadratic matrix, makes the Laplace formula exactly equal to the integral. This is a sufficient guarantee; it is not a necessary condition for equality in an individual nonquadratic case.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

For $h(b)=c-\frac{1}{2}(b-m)^TA(b-m)$, what conditions on $A$ and the domain give the quadratic exactness guarantee?
A one-dimensional log integrand has mode $\hat b=3$ and Hessian $H=-9$. What is the Laplace multiplier, apart from $\exp\{h(\hat b)\}$?
Why must a normalized Gaussian density integrate to one under the Laplace formula?
Does a negative-definite Hessian at the mode alone guarantee exactness? Explain.

Solutions. (1) $A$ must be symmetric positive definite, and the integrand must have the stated quadratic form throughout the full domain $\mathbb{R}^q$. (2) Here $q=1$ and $-H=9$, so the multiplier is $\sqrt{2\pi}/3$. (3) Its log density is globally quadratic, its mode is the mean, and the Gaussian density’s normalization cancels the Laplace factor $(2\pi)^{q/2}|\Sigma|^{1/2}$, giving one. (4) No. A negative-definite Hessian supplies local curvature information but does not establish a globally quadratic log integrand; exactness is guaranteed under the theorem’s full conditions, though other integrands can match the Laplace value by coincidence.