Random-Effect Parameterization and Approximation

Why Random-Effect Coordinates Matter

A marginal likelihood integrates over latent individual effects. The integral itself does not depend on the coordinates used to describe those effects, provided the transformation is one-to-one and its Jacobian is included. A finite-order approximation is a different matter: it uses local expansions, and those expansions can change when the coordinates change. Distinguishing these two facts is essential when interpreting differences between approximate objectives.

For individual \(i\), let \(y_i\) denote the observations, \(\eta_i\) the random effects, and \(\theta\) the population parameters. The centered parameterization uses \(\eta_i\) directly, typically with \(\eta_i\sim N(0,\Omega)\). Its individual marginal likelihood is

L_i(\theta,\Omega)=\int p(y_i\mid\eta_i,\theta)\,p(\eta_i\mid\Omega)\,d\eta_i.

A transformed parameterization writes \(\eta_i=T(z_i)\), where \(T\) is a differentiable bijection. If \(J_T(z_i)\) is its Jacobian matrix, the transformed integral is

L_i(\theta,\Omega)= \int p(y_i\mid T(z_i),\theta)\,p(T(z_i)\mid\Omega) \left|\det J_T(z_i)\right|\,dz_i.

The determinant is not optional: it adjusts the density for the change in volume between coordinate systems. Exact equality follows from change of variables, not from an approximation.

Definition: A random-effect parameterization is a choice of coordinates and probability density used to represent the latent individual effects in the likelihood integral. Two parameterizations represent the same model only when the transformation, transformed density, and Jacobian together preserve the same probability distribution for the random effects.

Linear Changes and Laplace Approximation

A common change of coordinates is linear: \(\eta_i=Lz_i\), where \(LL^\mathsf{T}=\Omega\) and \(z_i\) has a standard multivariate normal distribution. This is an affine rescaling and rotation, not a change to the random-effect distribution. With the covariance and Jacobian handled consistently, it leaves the exact marginal likelihood unchanged.

It also leaves the ordinary Laplace calculation unchanged. To see why, write the centered integrand as \(\exp\{-h(\eta)\}\). At its interior minimum \(\hat\eta\), the gradient is zero. Under \(\eta=Lz\), the Hessian at the corresponding point is \(L^\mathsf{T}H_\eta L\), where \(H_\eta\) is the Hessian of \(h\). Its determinant is \(\det(L)^2\det(H_\eta)\), while the volume element contributes \(|\det L|\). These factors cancel in the Laplace formula. This argument assumes a nonsingular \(L\), a smooth integrand, and a positive-definite Hessian at the minimum.

Worked Example: A Centered and Linearly Transformed Gaussian Integral

Suppose, hypothetically, one observation \(y=2\) has conditional distribution \(Y\mid\eta\sim N(\eta,1)\), and \(\eta\sim N(0,3)\). The marginal likelihood is the integral of the conditional density times the random-effect density.

The negative log of the product of those densities, apart from its normalizing constants, is \(h(\eta)=(2-\eta)^2/2+\eta^2/6\). Differentiating gives \(h'(\eta)=-(2-\eta)+\eta/3\), so the mode is \(\hat\eta=1.5\). The second derivative is \(h''(\eta)=1+1/3=4/3\).

The two normal densities have marginal variance \(1+3=4\), so the exact marginal likelihood is

L(y)=\frac{1}{\sqrt{2\pi(4)}}\exp\left(-\frac{2^2}{2(4)}\right) =\frac{\exp(-0.5)}{\sqrt{8\pi}} \approx 0.1210.

Because the integrand is Gaussian, the Laplace calculation is exact. Now set \(\eta=\sqrt{3}\,z\), so \(z\sim N(0,1)\). This linear change contributes the constant Jacobian \(\sqrt{3}\); the transformed integral still equals \(0.1210\), and its Laplace approximation gives the same value. A different numerical value here would indicate an inconsistent covariance, density, Jacobian, or calculation—not a meaningful effect of linear coordinates.

Nonlinear Changes and Finite-Order Approximation

For a nonlinear transformation, the exact integral remains invariant, but the transformed integrand includes a Jacobian that varies with \(z_i\). Consequently, the mode of the transformed integrand need not be the image of the centered-coordinate mode. Its curvature can also differ. A Laplace approximation uses a local mode and curvature, so its finite-sample value can depend on the nonlinear coordinates even though the underlying integral does not.

This is one reason to distinguish the exact marginal likelihood from its approximation. Under regularity conditions, Laplace approximation is justified when the integrand is concentrated near a well-behaved interior mode. As concentration increases, the leading approximation captures the dominant contribution; finite-order error terms can still depend on the chosen coordinates. A complete asymptotic expansion, including the corresponding correction terms, describes the same integral under either coordinate system. Merely changing variables and applying the leading Laplace formula again does not generally include those corrections.

Worked Example: A Nonlinear Transformation Changes the Laplace Value

Consider the hypothetical integral \(I=\int_{-\infty}^{\infty}\exp(-\eta^2/2)\,d\eta=\sqrt{2\pi}\). In centered coordinates its integrand has mode \(0\), and the negative log-integrand \(h(\eta)=\eta^2/2\) has curvature \(h''(0)=1\). Thus the Laplace approximation is \(\exp(0)\sqrt{2\pi/1}=2.5066\), exactly the integral.

Now use the smooth, increasing bijection \(T(z)=z+0.5\tanh(z)\). Its derivative is \(T'(z)=1+0.5\,\mathrm{sech}^2(z)>0\), and it maps the real line onto itself. The transformed integrand is \(\exp\{-T(z)^2/2\}T'(z)\). The Jacobian is largest at zero, and the transformed integrand has its mode there. At zero, \(T(0)=0\), \(T'(0)=1.5\), and the second derivative of \(\log T'(z)\) is \(-2/3\). Therefore the transformed negative log-integrand has value \(-\log(1.5)\) and curvature \(1.5^2+2/3=2.9167\) at the mode.

Its leading Laplace approximation is

I_{\mathrm{Laplace},z} =\exp\{\log(1.5)\}\sqrt{\frac{2\pi}{2.9167}} \approx 2.202.

The exact transformed integral is still \(\sqrt{2\pi}\approx2.5066\), by the change-of-variable formula. The transformed-coordinate Laplace approximation is about 12.2% lower. This difference is a finite-order approximation effect; it does not mean the transformation changed the model or the exact integral.

Why Taylor-Based Approximations Can Respond to Coordinates

FO and FOCE use local expansions of the individual contribution. A Taylor expansion is local to its chosen coordinates and expansion point. If the transformation is affine and all means, covariances, derivatives, and expansion points are transformed consistently, first-order linearization represents the same local change in the model prediction. Under a nonlinear transformation, higher derivatives enter differently, and truncating the expansion can discard different terms in the two coordinate systems. In FOCE, the conditional expansion point can also move because the transformed density includes a varying Jacobian.

The curvature transformation makes one source of this sensitivity explicit. For a scalar function \(q(\eta)\) and a scalar transformation \(\eta=T(z)\), the chain rule gives

\frac{d^2}{dz^2}q(T(z)) = \left(\frac{d\eta}{dz}\right)^2\frac{d^2q}{d\eta^2} + \frac{dq}{d\eta}\frac{d^2\eta}{dz^2}.

The second term vanishes at a stationary point of \(q\), but not generally away from one. For a transformed density there is an additional contribution from the log-Jacobian. Thus a curvature calculated at an arbitrary expansion point cannot be transformed by simply multiplying it by the square of the first derivative.

Worked Example: The Extra Curvature Term Away From a Mode

Let \(q(\eta)=(\eta-1)^2/2\), and use the bijection \(T(z)=z+0.5(1-\exp(-z))\). At \(z=0\), \(T(0)=0\), \(T'(0)=1.5\), and \(T''(0)=-0.5\). In centered coordinates, \(q'(\eta)=\eta-1\) and \(q''(\eta)=1\), so at \(\eta=0\) the gradient is \(-1\) and the curvature is \(1\).

The transformed curvature of \(q(T(z))\) at zero is therefore \(1.5^2(1)+(-1)(-0.5)=2.75\). Using only the squared-slope term would give \(2.25\), which is incorrect because the expansion point is not a stationary point. If this is part of a transformed density, the curvature of the negative log-Jacobian must also be included. This calculation illustrates why local curvature and truncated expansions require careful coordinate bookkeeping.

Assumptions, Diagnostics, and Practical Comparisons

A coordinate comparison is meaningful only when both calculations represent the same probability model. The transformation should be differentiable and one-to-one on the relevant support, its Jacobian should be nonsingular there, and the transformed density must include the Jacobian. For a standard Laplace calculation, the chosen integrand should have an isolated interior mode and a positive-definite Hessian for its negative log at that mode. If these conditions fail, the usual local Gaussian approximation may be unreliable regardless of parameterization.

When results differ, check whether the difference is expected approximation sensitivity or an implementation error. Useful checks include:

Verify that both parameterizations produce the same exact likelihood under a numerical change-of-variable check on a low-dimensional test case.
Confirm that the transformed prior includes the Jacobian and has the intended support and covariance structure.
Record the expansion point, mode, and Hessian in each coordinate system; check that the Hessian is positive definite for Laplace.
Compare approximate values across sensible parameterizations and against a more direct integration calculation when feasible.
Look for unstable modes, nearly singular curvature, or large changes under small numerical perturbations. These suggest that a local approximation is fragile.

FO and FOCE are local Taylor-based alternatives, while Laplace uses a local quadratic approximation to the log-integrand. Each can be efficient, but their finite-order behavior can depend on the coordinates and expansion choices. Direct numerical integration targets the integral rather than one local quadratic representation, though its computational burden can become substantial as the number of random effects grows. None of these methods makes an incorrect transformed density valid.

Common pitfall: “The marginal likelihood is invariant to reparameterization, so the Laplace objective must be identical in every coordinate system.” The first clause is true for an exact integral with the correct Jacobian. The conclusion does not follow: a leading finite-order approximation uses a coordinate-dependent local representation. Conversely, a large discrepancy is not automatically harmless; first rule out an omitted Jacobian, an inconsistent covariance, a missed mode, or numerical failure.

Key takeaway: Exact likelihood invariance is a change-of-variable result. Approximation invariance is a separate question: consistent affine changes preserve the usual local calculation, but nonlinear changes can alter finite-order modes, curvature, and Taylor remainders.

Check Your Understanding

Answer each question, then compare your reasoning with the short solutions.

What factor must be included when a random effect is expressed through a differentiable bijection?
Why does a consistent nonsingular linear transformation preserve the ordinary Laplace calculation?
Does a difference in nonlinear-coordinate Laplace values prove that the exact marginal likelihood changed?
At a nonstationary expansion point, why is the transformed curvature not just the original curvature times the squared derivative?
Name two diagnostics that can distinguish approximation sensitivity from an implementation error.

Solutions. (1) Include the absolute determinant of the transformation’s Jacobian in the transformed density. (2) The Hessian determinant gains the square of the linear transformation’s determinant, which cancels with the volume-element factor. (3) No; the exact integral is invariant when the density and Jacobian are transformed correctly, although a finite-order Laplace value can differ. (4) The chain rule includes the additional term consisting of the original gradient times the transformation’s second derivative. (5) For example, check the Jacobian and transformed covariance, or compare the approximation with direct integration in a low-dimensional case; also inspect modes and Hessian definiteness.