INLA Approximation for Shared Latent Effects

Why the Approximation Target Matters

A joint model connects repeated measurements and event times through shared subject-specific latent effects. For example, a subject’s random intercept or slope may help describe both their longitudinal trajectory and their event hazard. The model’s posterior therefore concerns many quantities at once: population-level coefficients, subject-level effects, and parameters governing how the two outcomes are associated.

INLA is designed to approximate posterior marginals in latent Gaussian models. Its target is not simply one estimate for each parameter. It approximates distributions for individual components, accounting for uncertainty in the other latent variables and in model hyperparameters. Identifying which quantities are treated as components of the latent field, and which are treated as hyperparameters, is essential for interpreting those marginals.

Definition: In a latent Gaussian model, the latent field \(x\), conditional on hyperparameters \(\theta\), has a Gaussian prior with mean \(\mu(\theta)\) and precision matrix \(Q(\theta)\). Observations are conditionally independent given \(x\) and \(\theta\), with likelihood contributions \(p(y_k\mid x,\theta)\). In a joint model, \(x\) can contain fixed-effect coefficients and subject-specific random effects, including effects shared across the longitudinal and event-time submodels. Hyperparameters \(\theta\) include parameters governing the latent-field distribution and may include the association parameter.

A schematic joint-model likelihood factors into longitudinal and event-time contributions conditional on the shared latent effects. This factorization is a modeling assumption: after conditioning on the relevant latent field and covariates, the observed outcomes are modeled as conditionally independent. The prior on the latent field is Gaussian conditional on \(\theta\). INLA also relies on a manageable, typically low-dimensional hyperparameter vector and on posterior shapes that can be represented adequately by its numerical approximations.

p(x,\theta\mid y)\ \propto\ p(\theta)\,p(x\mid\theta)\prod_k p(y_k\mid x,\theta)

Here \(y\) collects the observed longitudinal and event-time data. The association parameter, denoted here by \(\alpha\), is often included in \(\theta\), because it controls how the shared latent effect enters the event submodel. Fixed-effect coefficients and random effects are usually components of \(x\). This is a common modeling arrangement, not a mathematical requirement: software representation depends on the model specification. The posterior targets remain the distributions of the quantities of scientific interest, regardless of how the software organizes them.

What INLA Approximates

INLA uses nested approximations. For a fixed value of \(\theta\), it approximates the conditional posterior of the latent field \(p(x\mid\theta,y)\). It also approximates the marginal posterior density of the hyperparameters, \(p(\theta\mid y)\). It then averages conditional latent-field marginals over the hyperparameter distribution to approximate the marginal posterior of each component \(x_i\).

p(x_i\mid y) = \int p(x_i\mid\theta,y)\,p(\theta\mid y)\,d\theta \ \approx\ \int \widetilde p(x_i\mid\theta,y)\, \widetilde p(\theta\mid y)\,d\theta

The tildes indicate approximations. INLA approximates the integral numerically, commonly by evaluating the integrand at selected values of \(\theta\) and combining the results with weights. The purpose is not to report \(x_i\) at one “best” hyperparameter value. Integrating over \(\theta\) allows uncertainty in, for example, a random-effect variance or association parameter to affect the reported marginal for \(x_i\).

For a fixed \(\theta\), write the log of the latent-field posterior kernel as \(g(x;\theta)=\log p(y\mid x,\theta)+\log p(x\mid\theta)\). Let \(\hat{x}(\theta)\) maximize this function, and let \(H(\theta)=-\nabla_x^2 g(\hat{x}(\theta);\theta)\) be the negative Hessian at the mode. A local quadratic expansion of \(g\) gives a Gaussian approximation to the conditional posterior and a Laplace approximation to the integrated likelihood.

g(x;\theta)\ \approx\ g(\hat{x};\theta) -\frac{1}{2}(x-\hat{x})^\mathsf{T}H(\theta)(x-\hat{x})

\widetilde p(y\mid\theta) \ \approx\ \exp\{g(\hat{x};\theta)\}(2\pi)^{m/2}|H(\theta)|^{-1/2}, \qquad \widetilde p(\theta\mid y)\ \propto\ \widetilde p(y\mid\theta)p(\theta)

Here \(m\) is the dimension of \(x\). The determinant term arises because integrating the local Gaussian approximation contributes its volume, not just its height at the mode. The approximation to \(p(\theta\mid y)\) is normalized over \(\theta\) before it is used to average conditional marginals. The Taylor expansion justifies the local Gaussian step when the posterior is sufficiently smooth and approximately quadratic near its mode. INLA can use more refined conditional approximations than this basic Gaussian form, but the central target remains the same.

Three Distinct Marginal Targets

A fixed-effect coefficient, a subject-specific random effect, and an association parameter do not all enter the approximation in the same way. Their marginal posteriors are related, but the roles they play in INLA’s nested calculation differ.

Quantity	Typical representation	Marginal target
Fixed-effect coefficient, such as a longitudinal time slope	Component of the latent field \(x\)	\(p(\beta_j\mid y)\), obtained by integrating its conditional marginal over \(\theta\)
Subject-specific random effect, such as subject \(i\)’s intercept	Component of the latent field \(x\)	\(p(b_i\mid y)\), accounting for uncertainty in other latent components and \(\theta\)
Association parameter \(\alpha\)	Often a component of \(\theta\)	\(p(\alpha\mid y)\), a marginal of the hyperparameter posterior approximation

For the fixed effect, the marginal describes uncertainty in a population-level coefficient after accounting for the observed data, latent effects, and hyperparameters. For a random effect, it describes uncertainty about a particular subject’s deviation from the population pattern, not the distribution of random effects in a hypothetical new sample. For an association parameter, the marginal describes the data-supported values of the parameter linking the shared effect to the event process, conditional on the model and priors.

These are marginal distributions. Separate marginals do not, by themselves, describe the full posterior dependence between a fixed effect and a random effect, or between an association parameter and a variance parameter. A collection of marginal summaries is not a substitute for the full joint posterior when the question depends on such dependence.

Worked Examples

Worked Example: Integrating a Fixed-Effect Marginal

Consider hypothetical joint-model output for a longitudinal time-slope coefficient \(\beta\). Suppose the hyperparameter integration has two configurations with approximate posterior weights \(0.25\) and \(0.75\). At those configurations, the conditional marginals are approximately normal: \(\beta\mid\theta_1,y\sim N(1.2,0.09)\) and \(\beta\mid\theta_2,y\sim N(0.8,0.16)\). The second argument is the variance.

The integrated posterior mean is the weighted mean of the conditional means:

E(\beta\mid y)\ \approx\ 0.25(1.2)+0.75(0.8)=0.90

To calculate the integrated variance, add the average conditional variance to the variance of the conditional means:

\operatorname{Var}(\beta\mid y) \ \approx\ 0.25(0.09)+0.75(0.16) +0.25(1.2-0.90)^2+0.75(0.8-0.90)^2 =0.1725

Thus the approximate posterior standard deviation is \(\sqrt{0.1725}=0.4153\). The posterior mean time slope is \(0.90\) outcome units per time unit, and its marginal uncertainty reflects both within-configuration uncertainty and variation between the two conditional means. Treating \(\theta\) as fixed at \(\theta_2\) would instead give a conditional mean of \(0.8\) and would omit the between-configuration contribution.

Worked Example: A Subject-Specific Random-Effect Marginal

Suppose a hypothetical subject’s random intercept \(b_i\), measured in outcome units, has two conditional posterior approximations. The hyperparameter weights are \(0.40\) and \(0.60\); the conditional distributions are \(b_i\mid\theta_1,y\sim N(-0.2,0.04)\) and \(b_i\mid\theta_2,y\sim N(0.3,0.09)\).

The integrated mean is \(0.40(-0.2)+0.60(0.3)=0.10\) outcome units. The integrated variance is

\begin{aligned} \operatorname{Var}(b_i\mid y) &\approx 0.40(0.04)+0.60(0.09)\\ &\quad+0.40(-0.2-0.1)^2+0.60(0.3-0.1)^2\\ &=0.016+0.054+0.036+0.024\\ &=0.130 \end{aligned}

The approximate posterior standard deviation is \(\sqrt{0.130}=0.3606\) outcome units. This marginal concerns this subject’s latent intercept, after integrating over hyperparameter uncertainty. It is not a direct estimate of the population random-intercept standard deviation; that variance parameter has its own posterior target.

Worked Example: An Association-Parameter Marginal

Suppose an INLA-style numerical integration gives three hyperparameter configurations for a hypothetical association parameter \(\alpha\), measured as a log-hazard change per one-unit increase in the shared longitudinal latent value. The approximate posterior weights are \(0.20\), \(0.50\), and \(0.30\), at \(\alpha=0.2\), \(0.6\), and \(1.0\), respectively.

The approximate posterior mean and variance are

E(\alpha\mid y)\approx 0.20(0.2)+0.50(0.6)+0.30(1.0)=0.64

\begin{aligned} \operatorname{Var}(\alpha\mid y) &\approx 0.20(0.2-0.64)^2+ 0.50(0.6-0.64)^2+ 0.30(1.0-0.64)^2\\ &=0.0784 \end{aligned}

The approximate posterior standard deviation is \(0.28\), and the integrated weight on values above \(0.5\) is \(0.50+0.30=0.80\). Under the stated association model, \(\exp(\alpha)\) is the hazard ratio for a one-unit increase in the shared latent value. At the posterior mean \(\alpha=0.64\), that ratio is \(\exp(0.64)=1.896\), but this is the transform of the posterior mean, not the posterior mean of the hazard ratio. The weighted mean of the transformed values is approximately \(0.20\exp(0.2)+0.50\exp(0.6)+0.30\exp(1.0)=1.971\). The distinction matters because transformation and averaging do not generally commute.

Justification for the Integration Step

The mixture calculations in the examples follow directly from the law of total expectation and the law of total variance. These identities explain why integrating over hyperparameters changes both the center and spread of a latent-field marginal.

Condition on the hyperparameters.
For a latent component \(x_i\), use its conditional posterior mean \(m_i(\theta)=E(x_i\mid\theta,y)\) and variance \(v_i(\theta)=\operatorname{Var}(x_i\mid\theta,y)\), or their INLA approximations.

Average the conditional means.
Taking expectation first conditional on \(\theta\) gives \(E(x_i\mid y)=E_{\theta\mid y}\{m_i(\theta)\}\).

Account for both sources of variance.
Decompose \(x_i-E(x_i\mid y)\) into \(x_i-m_i(\theta)\) and \(m_i(\theta)-E(x_i\mid y)\). The cross-product has conditional expectation zero, so the total variance is \(E_{\theta\mid y}\{v_i(\theta)\}+\operatorname{Var}_{\theta\mid y}\{m_i(\theta)\}\).

\operatorname{Var}(x_i\mid y) = E_{\theta\mid y}\{\operatorname{Var}(x_i\mid\theta,y)\} + \operatorname{Var}_{\theta\mid y}\{E(x_i\mid\theta,y)\}

INLA replaces the exact conditional distributions and hyperparameter integration in these identities with approximations. A numerical integration scheme can be expressed as weighted sums, with weights that approximate posterior mass over hyperparameter configurations. If the configurations fail to cover a region with meaningful posterior mass, the resulting marginal can miss part of the uncertainty.

Diagnostics, Alternatives, and Limitations

A useful first check is whether the posterior shapes are compatible with the approximation being used. Strong skewness, heavy tails, multiple separated modes, or a nearly singular Hessian can make a local Gaussian approximation unreliable. Hyperparameter integration can also be inaccurate if the posterior is concentrated near a boundary or if the selected integration region does not capture its tails. Inspect marginal shapes and numerical stability, and treat unusually abrupt changes in a marginal under small modeling or computational changes as a reason for further investigation.

INLA and Markov chain Monte Carlo (MCMC) target the same posterior distributions under the same model and priors, but use different computational strategies. INLA approximates posterior marginals through latent-field and hyperparameter calculations; MCMC uses draws from a Markov chain to estimate posterior quantities. INLA can be efficient for latent Gaussian models when its approximations are adequate. MCMC can represent complex posterior dependence and non-Gaussian shapes more directly, but its results require careful assessment of mixing and Monte Carlo error. A comparison with MCMC can be informative when accuracy is uncertain, though MCMC is not automatically reliable without adequate computation and diagnostics.

Do not confuse a conditional marginal with an integrated marginal. A result at one selected \(\theta\) omits variation across hyperparameters.
Do not interpret separate marginals as a joint posterior. Marginal summaries do not reveal all dependence between coefficients, random effects, and association parameters.
Do not assume a normal marginal simply because a Gaussian approximation was used locally. Integration over hyperparameters can produce a skewed or mixed marginal.
Do not interpret the random-effect marginal as a population distribution. It describes a particular latent effect conditional on the data and model.

Key takeaway: INLA’s target for a latent-field component is its posterior marginal after accounting for hyperparameter uncertainty. Fixed effects and subject-specific random effects are commonly latent-field components; association parameters are commonly hyperparameters. The approximation and its adequacy must be assessed for the relevant marginal, not inferred from a point estimate alone.

Check Your Understanding

Answer each question, then check the brief solutions below.

Why does the marginal posterior of a latent-field component require integration over \(\theta\), rather than evaluation at a single hyperparameter estimate?
A random-effect component has conditional means that vary across hyperparameter configurations. Which term in the total-variance identity captures this variation?
In the usual setup described here, where is a fixed-effect coefficient represented, and what is its posterior target?
Why does a separate marginal for an association parameter not fully describe its dependence with a random-effect variance?
Name two posterior features that can make a local Gaussian approximation less credible.

Solutions. (1) The marginal averages the conditional posterior over \(p(\theta\mid y)\), including uncertainty in the hyperparameters. (2) The term is \(\operatorname{Var}_{\theta\mid y}\{E(x_i\mid\theta,y)\}\), the variance of the conditional means. (3) It is commonly a latent-field component \(x_i\), and the target is \(p(\beta_j\mid y)\), integrated over hyperparameters. (4) A marginal retains the distribution of one quantity but not the joint posterior dependence between it and another. (5) Examples include strong skewness, heavy tails, multiple modes, or a nearly singular Hessian.