Gaussian Importance Proposals from Conditional Modes

Why Center a Proposal at the Conditional Mode?

For subject \(i\), the marginal likelihood contribution integrates that subject’s likelihood over the random-effects distribution. When the data make the subject-specific likelihood concentrate far from the population center, drawing random effects only from the population distribution may yield few draws in the important region. A subject-specific importance proposal instead aims to place draws near the region that contributes most to the integral.

A useful starting point is the conditional mode: the random-effects value that maximizes the individual likelihood multiplied by the random-effects density. The local curvature around this mode gives a principled scale for a Gaussian proposal. This is a proposal-design method, not a replacement for the marginal likelihood integral: importance weights retain the correction for using a proposal rather than the original integration density.

The random-effects density belongs in this definition. Maximizing the individual likelihood alone generally gives a different value, because it omits the population model's contribution. The conditional mode is also defined at specified \(\theta\); if the population or residual parameters change, the mode and its local curvature can change too.

From Local Curvature to a Gaussian Proposal

Suppose \(h_i\) is twice differentiable near an interior conditional mode \(\hat\eta_i\). At a local maximum, its gradient is zero. Define the observed negative Hessian at that mode by \(J_i=-\nabla^2 h_i(\hat\eta_i)\). If \(J_i\) is positive definite, a second-order Taylor expansion gives a local quadratic approximation to the log target.

Exponentiating the quadratic term gives the shape of a multivariate Gaussian centered at \(\hat\eta_i\), with precision matrix \(J_i\) and covariance \(J_i^{-1}\). This motivates the proposal \(q_i(\eta_i)=N(\hat\eta_i,J_i^{-1})\). The approximation need only guide the proposal; the importance weight uses the actual target \(g_i\), not its quadratic approximation.

In practice, random effects are often written in standard-normal coordinates. If \(\eta_i=Lz_i\), where \(LL^\mathsf{T}=\Omega\), then \(z_i\sim N(0,I)\) under the population random-effects model. Define \(h_i(z_i)=\log L_i(Lz_i;\theta)+\log\phi(z_i)\), find its mode \(\hat z_i\), and calculate \(J_{z,i}=-\nabla^2h_i(\hat z_i)\). The proposal in these coordinates is \(z_i\sim N(\hat z_i,J_{z,i}^{-1})\). Transforming back gives mean \(L\hat z_i\) and covariance \(LJ_{z,i}^{-1}L^\mathsf{T}\). The prior contribution must be included in whichever coordinates are used.

The Taylor argument also explains the limitation of the method: the Hessian describes curvature only locally. If the target is skewed, has heavy tails, or has multiple modes, a Gaussian centered at one mode may not represent all regions that contribute appreciably to the integral.

What Proposal Location and Scale Do to Weights

For draws \(\eta_i^{(m)}\) from a proposal density \(q_i\), the importance weight is \(w_i^{(m)}=g_i(\eta_i^{(m)};\theta)/q_i(\eta_i^{(m)})\). If \(q_i\) is positive wherever \(g_i\) is positive and the integral is finite, then \[ L_i^{\mathrm{marg}}(\theta) =E_{q_i}\left[\frac{g_i(\eta_i;\theta)}{q_i(\eta_i)}\right]. \] Consequently, the average of \(M\) such weights is an unbiased estimator of the integral when the draws are independent from \(q_i\). A proposal that puts too little density in an important target region can produce occasional very large weights; a proposal that puts excessive density elsewhere can waste draws on regions contributing little to the integral.

Multivariate Curvature and Coordinate Choice

In multiple dimensions, curvature includes dependence between random-effects components. The off-diagonal entries of \(J_i\) matter: replacing the full covariance \(J_i^{-1}\) with independent marginal variances can miss tilted directions of the target. A positive-definite Hessian is also essential for the stated Gaussian construction. A singular or indefinite matrix does not define a finite positive-definite covariance through inversion.

Checking the Construction and Choosing Alternatives

A mode-based Gaussian is most defensible when the target is smooth near a well-defined interior mode and the negative Hessian is stable and positive definite. Check the numerical optimization by evaluating the gradient near the reported mode and confirming that the Hessian has the required sign. For multivariate problems, inspect whether small eigenvalues make the inverse covariance extremely large or whether numerical asymmetry or rounding has affected the Hessian.

These checks do not establish that the target is globally Gaussian. Compare the mode-based proposal with the actual conditional target over a range of plausible random-effects values, and examine whether proposal draws cover regions where the target density is appreciable. Strong skewness, a long tail, separated modes, or large changes in curvature away from the mode warn that local curvature may give a poor scale or miss important regions. When multiple modes are present, a proposal centered on just one mode may fail to represent the others.

A population-centered proposal is simpler and can be reasonable when the subject-specific conditional target remains close to the population random-effects distribution. It may be inefficient when the subject’s data pull the conditional target far from that center. The mode-and-curvature proposal adapts its center and local scale to each subject, but requires optimization and Hessian calculation, and remains a local approximation. Laplace approximation also uses a mode and local curvature, but approximates the integral from those quantities; importance sampling instead evaluates the target-to-proposal density ratio, so the exact target—not only its quadratic approximation—enters each weight. Neither the local Gaussian nor a good mode guarantees low weight dispersion in a skewed or multimodal target.