Blocking Updates for Correlated Random Effects

Why the Update Block Matters

In a Bayesian GLMM, random effects for the same group may be correlated. The sampler must then move through a posterior distribution whose high-probability region can be tilted or elongated. Updating one coefficient at a time is valid when each update targets the correct conditional distribution, but it may make only small effective progress: changing one coefficient while holding its correlated partner fixed can repeatedly move the chain away from the high-probability region. Updating the coefficients together can let the chain move along that region instead.

The choice is not simply “joint is better.” A joint update may require a more demanding proposal or more computation, and a poorly chosen joint proposal can also mix badly. The useful question is which block structure produces adequate movement for the cost, while preserving the intended posterior distribution.

Posterior Target and Update Blocks

Consider a GLMM with groups \(j=1,\ldots,J\) and observations \(i=1,\ldots,n_j\). Let \(y_{ij}\) be the response, \(x_{ij}\) the fixed-effect design vector, and \(z_{ij}\) the random-effect design vector. Write the linear predictor as \(\eta_{ij}=x_{ij}^{\mathsf T}\beta+z_{ij}^{\mathsf T}b_j\), with the response distribution and link specified by the GLMM. Suppose the group-level vectors are conditionally independent given covariance parameters and satisfy \(b_j\sim N_q(0,\Sigma)\). A common decomposition is \(\Sigma=DRD\), where \(D\) contains random-effect standard deviations and \(R\) is a correlation matrix.

Let \(\theta\) collect fixed effects, covariance parameters, and any other model parameters. For observed data \(y\), the posterior target for the random effects and parameters is proportional to the conditional likelihood times the random-effects density and parameter priors:

Here \(\phi_q(\,\cdot\,;0,\Sigma)\) is the \(q\)-dimensional normal density. The grouping structure makes each group’s likelihood contribution local to \(b_j\), but the prior covariance couples the components of that vector. This tutorial assumes the model and priors define a proper posterior, \(\Sigma\) is positive definite wherever the posterior has support, and each sampling transition is constructed to leave the posterior invariant. For Metropolis–Hastings updates, the proposal and acceptance probability must account for the target density and proposal probabilities.

For correlated random effects, a natural middle ground is to update the \(q\) components of one group’s \(b_j\) as a block, rather than updating every random effect in the entire model simultaneously. A further option is a larger block containing random effects and some covariance parameters. Each design changes the dimension and geometry of the transition. The posterior target remains the same; what changes is how efficiently the chain explores it.

Why Strong Correlation Can Slow Componentwise Gibbs Updates

The effect of blocking can be seen exactly in a simple bivariate normal target. This calculation is a geometric illustration of posterior dependence, not a claim that a logistic GLMM’s conditional posterior is exactly normal. Suppose the target for two scalar effects \(U,V\) is standard bivariate normal with correlation \(r\), where \(-1<r<1\). Its conditional distributions are \(U\mid V=v\sim N(rv,1-r^2)\) and \(V\mid U=u\sim N(ru,1-r^2)\).

A systematic componentwise Gibbs sweep first draws \(U_{t+1}\) given \(V_t\), then draws \(V_{t+1}\) given \(U_{t+1}\). Write the independent standard normal innovations as \(\epsilon_{t+1}\) and \(\zeta_{t+1}\). The two conditional draws are

Substituting the first equation into the second gives \(V_{t+1}=r^2V_t+r\sqrt{1-r^2}\epsilon_{t+1}+\sqrt{1-r^2}\zeta_{t+1}\). The innovations are independent of the current state, so the lag-one correlation of the stationary \(V\) chain is \(r^2\). As \(|r|\) approaches one, this correlation approaches one: successive sweeps retain much of the previous state. A joint draw directly from the bivariate target, by contrast, produces independent successive draws in this special case.

That comparison has a qualification: an exact joint draw is available here because the target is a known normal distribution. In a GLMM, a joint update may instead require a Metropolis proposal, Hamiltonian Monte Carlo, or another transition. A joint transition is not automatically independent, and its efficiency depends on how well it follows the posterior geometry.

Designing a Joint Metropolis Update

When the conditional posterior is not available in a convenient closed form, a block can be updated using Metropolis–Hastings. Given the current state \(b\), propose \(b'\) from \(q(b'\mid b)\) and accept it with probability

For a symmetric proposal, \(q(b'\mid b)=q(b\mid b')\), so the proposal terms cancel. A useful joint proposal often accounts for the orientation and scale of the conditional posterior: proposals along a long, high-density direction can move farther than proposals that repeatedly cross a narrow direction. The target covariance is not necessarily the same as the prior covariance \(\Sigma\), because the likelihood also informs the random effects.

Choosing a Block and Checking Its Behaviour

In practice, block size is a balance between dependence and computational cost. Updating each group’s random-effect vector together can address within-group correlation without building one very large proposal for all groups. If the posterior dependence is weak, scalar updates may be simpler and can work adequately. If strong dependence involves both random effects and covariance parameters, a block that includes only the effects may leave important dependence untouched; a larger block can help, but is more demanding to design and compute.

A sensible comparison asks whether the chain makes movement in the posterior directions that matter, not just whether each update is accepted. Useful preliminary checks include plotting draws of correlated effects together, inspecting trace plots, and examining autocorrelation for representative coordinates or posterior summaries. Persistent traces, narrow bands of points in joint plots, or slow changes in quantities such as a random-effect contrast can signal that the chosen updates are not exploring efficiently. These signs do not by themselves identify the cause; initialization, parameterization, or other features of the posterior may also contribute.

The main alternatives have different trade-offs:

Regardless of block choice, correctness and efficiency are separate questions. A valid transition must preserve the posterior target and allow the chain to reach the relevant posterior regions under its support and transition structure. A transition can be correct but inefficient; a move that appears to improve mixing is not valid if its acceptance rule omits a required density or proposal term. Blocking also cannot repair a misspecified model or an inappropriate prior.

Common Pitfall

Wrong reasoning: “The random effects are correlated, so a joint update must always be better.” Correlation gives a reason to consider blocking, but not proof that a particular block or proposal will perform well. A large, poorly scaled joint proposal can be rejected often, while a componentwise scheme can be quite adequate when dependence is modest or its conditional updates are effective.

Correction: Treat blocking as an algorithmic choice to assess. State the target, use a transition that preserves it, and compare the movement and computational cost obtained from plausible block structures. Strong posterior dependence makes joint updating a useful candidate, not a universal rule.