Hyperparameter Integration and Marginals in INLA

Why the Marginals Require Integration

In a joint model, a longitudinal outcome and an event outcome can both provide information about shared latent effects. INLA represents such models using a latent Gaussian vector and a relatively low-dimensional vector of hyperparameters. The posterior distribution of a latent effect is therefore not, in general, just its conditional distribution at one selected hyperparameter value. It must also account for uncertainty about those hyperparameters.

This distinction matters for interpretation. A posterior mean or interval for a subject-specific random effect should reflect both uncertainty remaining when the hyperparameters are held fixed and variation in the effect's conditional distribution across plausible hyperparameter values. Similarly, a hyperparameter marginal is a posterior distribution, not merely an optimizer's point estimate.

A typical model has prior density \(p(x\mid\theta)\), likelihood \(p(y\mid x,\theta)\), and hyperparameter prior \(p(\theta)\). Its joint posterior is proportional to \(p(y\mid x,\theta)p(x\mid\theta)p(\theta)\). For a joint longitudinal-survival model, the likelihood can combine the longitudinal and event contributions conditional on the latent effects, as in the earlier material on encoding joint likelihoods in INLA. The marginal calculations described here do not require the two outcomes to be independent after the latent effects have been accounted for; they require the chosen likelihood and latent structure to represent their dependence appropriately.

INLA’s Approximation and Integration

INLA, the Integrated Nested Laplace Approximation, targets posterior marginals in latent Gaussian models by combining approximations with numerical integration over \(\theta\). First, for given \(\theta\), it approximates the conditional posterior of \(x\) and its relevant marginals. It also approximates the hyperparameter posterior, whose exact density is

The integral over the usually high-dimensional \(x\) is handled with a Laplace-based approximation. In outline, at each candidate \(\theta\), INLA finds a conditional posterior mode \(\hat{x}(\theta)\) and uses the curvature around that mode to approximate the integral. A standard Laplace form is

where \(p_G(x\mid y,\theta)\) is a Gaussian approximation to the conditional posterior of \(x\), evaluated at its mode in the denominator. INLA then explores the hyperparameter space and uses numerical integration to approximate posterior quantities. The conditional marginals may themselves use Gaussian or Laplace-based approximations; the choice of approximation affects the accuracy of the resulting marginal.

Here \(\tilde p(x_i\mid y,\theta_k)\) is INLA’s approximation to the conditional marginal at integration point \(\theta_k\). The weights represent the posterior mass assigned through the numerical integration scheme; they are not necessarily equal to the prior weights or to a simple grid spacing. The corresponding hyperparameter marginal is obtained by integrating or summarizing the approximate posterior over the other hyperparameters.

What Integration Does to Uncertainty

The effect of integration is especially clear for posterior means and variances. Write \(m(\theta)=E(x_i\mid y,\theta)\) and \(v(\theta)=\operatorname{Var}(x_i\mid y,\theta)\). The law of total expectation and law of total variance give

The first variance term averages the uncertainty conditional on hyperparameters. The second measures how the conditional mean changes over the posterior distribution of the hyperparameters. It is nonnegative, so ignoring hyperparameter uncertainty can omit a genuine component of posterior variance. This decomposition is also a useful check on calculations: a fully integrated variance cannot be smaller than the average conditional variance, apart from numerical approximation error.

Reading Hyperparameter Marginals and Scales

A hyperparameter marginal describes posterior uncertainty about a model component, such as a random-effect precision \(\tau\). A posterior mean, median, mode, or credible interval may answer different questions, especially when the marginal is skewed. The reported scale matters: a model may work internally with \(\eta=\log\tau\), while the scientific interpretation concerns \(\tau\). A posterior summary on the log scale is not generally the same as the corresponding summary on the precision scale.

For continuous variables, if \(\eta=\log\tau\), then \(\tau=\exp(\eta)\) and the densities are related by the change-of-variables rule:

A summary of \(\tau\) should be calculated from its posterior distribution on the precision scale, or by transforming draws or integration points and then summarizing with their posterior weights. For example, exponentiating the posterior mean of \(\log\tau\) gives a geometric-mean-type summary, not generally the posterior mean of \(\tau\).

The same integration logic applies to posterior probabilities and quantiles. For an event \(A\) involving a latent or model parameter, \(P(A\mid y)\approx\sum_k\alpha_k P(A\mid y,\theta_k)\). For quantiles, the integrated marginal distribution is formed first, and the desired quantile is then found from that distribution. In general, averaging conditional quantiles does not give the quantile of the mixture.

Diagnostics, Alternatives, and Limitations

Numerical integration is part of the inferential calculation, so inspect whether it is representing the posterior mass adequately. Useful checks include reviewing the shape and range of hyperparameter marginals, checking whether substantial mass lies near a boundary or beyond the explored region, and assessing whether summaries change when integration settings are refined. Concentrated or irregular weights can signal that a small number of integration points dominate; this does not by itself prove an error, but it warrants checking that the posterior region has been adequately explored. Also verify that the reported parameter is on the intended scale and that all weights used in a manual summary are normalized.

These checks concern numerical representation, not whether the model is scientifically appropriate. Even accurate integration cannot correct a misspecified likelihood, prior, or latent structure. INLA results also inherit error from the nested approximations: the conditional posterior and the hyperparameter posterior are approximated, and numerical integration is finite. Consequently, extra displayed digits do not guarantee corresponding inferential accuracy.

A conditional or plug-in result can be useful as a computational shortcut or a focused diagnostic, provided it is labelled clearly. It should not be presented as the fully integrated posterior marginal. Conversely, integration does not make the result automatically reliable: the approximation and integration scheme still need suitable numerical behavior, and uncertainty summaries remain conditional on the model.

Common Mistake

A common mistake is to treat the conditional posterior at the estimated hyperparameters as though it were the marginal posterior. For example, one might use the conditional standard deviation of a random effect as its full posterior standard deviation. The correction is to integrate the conditional distribution over the hyperparameter posterior. For variances, this adds the variation in conditional means, as shown by \(\operatorname{Var}(x_i\mid y)=E\{v(\theta)\mid y\}+\operatorname{Var}\{m(\theta)\mid y\}\). A second mistake is to average conditional credible-interval endpoints and call the result an integrated credible interval; instead, form the mixture marginal and calculate its quantiles.