What “Centered” Means for Group-Level Effects
A Bayesian GLMM represents differences among groups through latent effects drawn from a population distribution. The centered parameterization treats those effects themselves as model parameters: each group effect is expressed on the scale used in the linear predictor and is assigned its population distribution directly. This can make posterior sampling efficient when each group’s observations provide substantial information about its effect.
The word centered refers to how the latent effects are represented relative to their population distribution. It does not mean that the observed data have been centered by subtracting a sample mean. Nor does it mean that the group effects are fixed constants. They remain unknown random effects whose distribution is governed by variance and, where relevant, correlation parameters.
For group \(j\), let \(Y_{ij}\) be an outcome and let \(\mathbf x_{ij}\) and \(\mathbf z_{ij}\) be its fixed-effect and group-effect predictors. A standard GLMM specifies the conditional mean through \(g\{E(Y_{ij}\mid\mathbf b_j,\boldsymbol\beta)\}=\mathbf x_{ij}^{\mathsf T}\boldsymbol\beta+\mathbf z_{ij}^{\mathsf T}\mathbf b_j\), where \(g\) is the link function. Conditional on the group effects and model parameters, the observations are assumed independent and follow the stated response distribution. For \(q\)-dimensional effects, a common population model is \(\mathbf b_j\mid\boldsymbol\Sigma\sim N_q(\mathbf 0,\boldsymbol\Sigma)\), independently across groups conditional on \(\boldsymbol\Sigma\). Priors must also be specified for \(\boldsymbol\beta\) and the covariance parameters in \(\boldsymbol\Sigma\).
These assumptions are part of the model, not consequences of choosing a centered representation. In particular, the normal population distribution, conditional independence, and chosen priors must be scientifically and statistically defensible for the application. The centered form changes the coordinates in which the posterior is represented; it does not relax the GLMM’s modeling assumptions.
The Posterior Target in Centered Coordinates
Let \(\mathbf y\) denote all observed responses. Conditional independence gives a likelihood that factors over observations. Combining that likelihood with the group-effect distributions and the priors gives the joint posterior, up to a normalizing constant:
Here \(\phi_q(\cdot;\mathbf 0,\boldsymbol\Sigma)\) is the \(q\)-variate normal density. If \(\boldsymbol\Sigma\) is positive definite, each group-level density is
This expression makes the centered target explicit: the likelihood and the population distribution both involve the same \(\mathbf b_j\). Hamiltonian Monte Carlo can use the log of this target and its gradient to generate proposals in the joint parameter space. The prior contribution for each group penalizes effects that are large relative to the covariance structure, while the likelihood rewards effects compatible with that group’s observations.
A short derivation clarifies the balance. For a scalar random intercept \(b_j\sim N(0,\tau^2)\), its log prior, omitting constants, is \(-b_j^2/(2\tau^2)\). Adding its group log likelihood \(\ell_j(b_j)\) gives the conditional log posterior contribution \(\ell_j(b_j)-b_j^2/(2\tau^2)\). Differentiating twice with respect to \(b_j\) gives
Thus, local posterior curvature combines likelihood curvature with prior curvature \(1/\tau^2\). If the likelihood is informative, its curvature can dominate and constrain \(b_j\) fairly precisely. If it is weak, the posterior depends strongly on the prior scale, and changes in that scale can alter the shape of the joint posterior for \(b_j\) and \(\tau\).
Worked Example: Evaluating a Centered Normal Prior Density
Suppose two hypothetical scalar group effects are conditionally independent given a shared standard deviation \(\tau=0.30\), with \(b_j\mid\tau\sim N(0,\tau^2)\). Evaluate their prior densities at \(b_1=0.20\) and \(b_2=0.10\). These are density values, not probabilities at exact points.
For a normal variable with standard deviation \(0.30\), the density is \(f(b\mid0.30)=\{1/(0.30\sqrt{2\pi})\}\exp\{-b^2/(2(0.30)^2)\}\). At \(b_1=0.20\), the standardized squared value is \(0.20^2/0.30^2=0.04/0.09=0.4444\), so
At \(b_2=0.10\), the standardized squared value is \(0.01/0.09=0.1111\), giving \(f(0.10\mid0.30)=\{1/(0.30\sqrt{2\pi})\}\exp(-0.05556)\approx1.258\). Because the effects are independent conditional on \(\tau\), their joint density at these values is the product: \(1.0648267\times1.25794408\approx1.339492\).
The calculation illustrates the centered representation: the parameters are \(b_1\) and \(b_2\), and their prior density is evaluated directly at their model-scale values. A density can exceed 1 because it is not a probability; probabilities are obtained by integrating density over an interval or region.
When Direct Group Effects Can Be Efficient
Centered sampling is often effective when each group has enough data, and those data identify the group effect reasonably well. In that setting, the group likelihood narrows the plausible range of \(b_j\), so the posterior geometry is less dominated by the hierarchical prior. For HMC, this can permit useful movement in the group effects without requiring every group effect to follow the population scale almost exactly.
The following Gaussian example gives an exact calculation of this likelihood-versus-prior information. Assume independent observations \(Y_{ij}\mid b_j\sim N(\mu+b_j,\sigma^2)\), with known \(\mu\), \(\sigma\), and \(\tau\), and prior \(b_j\sim N(0,\tau^2)\). Define \(r_{ij}=Y_{ij}-\mu\). The conditional posterior is normal because the product of the normal likelihood and normal prior is proportional to a normal kernel. Completing the square yields posterior variance \(V_j\) and mean \(m_j\):
Worked Example: A Group Effect Well Informed by Gaussian Data
Consider one hypothetical group with \(n_j=8\), residual mean \(\bar r_j=0.60\) outcome units, known \(\sigma=0.40\) outcome units, and prior standard deviation \(\tau=0.50\) outcome units. The model assumptions are the Gaussian sampling model above and a normal prior; suppose the observations are conditionally independent. The likelihood precision is \(n_j/\sigma^2=8/0.16=50\), and the prior precision is \(1/\tau^2=1/0.25=4\), both in inverse squared outcome units.
The posterior variance is \(V_j=1/(50+4)=1/54=0.01852\) squared outcome units, so the posterior standard deviation is \(\sqrt{1/54}=0.1361\) outcome units. Since \(\sum_i r_{ij}=8(0.60)=4.80\) outcome units, the posterior mean is \(m_j=(1/54)(4.80/0.16)=(1/54)(30)=0.5556\) outcome units.
The likelihood precision of 50 is much larger than the prior precision of 4, and the posterior mean, 0.5556, is consequently close to the observed residual mean, 0.60. This is a setting in which directly sampling \(b_j\) can be well supported by the group’s observations. The calculation is conditional on the other parameters being known; in a full GLMM, they too are uncertain and sampled or otherwise accounted for.
In a GLMM with a non-Gaussian response, the conditional posterior for a group effect will generally not be exactly normal. The same information principle still applies locally: the likelihood’s curvature can be compared with the prior curvature. This comparison is a useful guide to the geometry, not a guarantee of good MCMC performance. Correlated effects, uncertainty in the covariance parameters, weakly identified fixed effects, and nonlinear likelihood shapes can all affect sampling.
Worked Example: Local Curvature in a Logistic Random-Intercept Model
Suppose a hypothetical group has 20 binary outcomes, of which 11 are successes. At a candidate random intercept \(b=0.11\), let the linear predictor be \(b\), so \(p=\operatorname{logit}^{-1}(0.11)\approx0.52747\). Assume a centered prior \(b\sim N(0,0.5^2)\). The observations are modeled as conditionally independent Bernoulli outcomes, and the prior standard deviation is \(0.5\) on the log-odds scale.
For a binomial log likelihood, the negative second derivative with respect to \(b\) is \(np(1-p)\). At the stated point, the likelihood curvature is \(20(0.52747)(1-0.52747)\approx4.985\) per squared log-odds unit. The prior curvature is \(1/\tau^2=1/0.25=4\) per squared log-odds unit. Therefore the local negative log-posterior curvature is approximately \(4.985+4=8.985\), corresponding to a local normal-approximation standard deviation \(1/\sqrt{8.985}\approx0.3336\) log-odds units.
At this point, the likelihood contributes more local curvature than the prior, though the prior contribution is still substantial. This calculation does not claim that the posterior is exactly normal or that \(b=0.11\) is its mode; it is a local assessment at the specified value. The data and prior together constrain the effect more than either curvature alone would suggest.
Comparison, Diagnostics, and Limitations
The centered representation is one way to target the posterior, not a different likelihood or prior model. One alternative is to integrate over the group effects and sample a marginal posterior for population parameters; this can remove the individual \(b_j\)'s from the sampled state but may require evaluating integrals that are difficult for non-Gaussian GLMMs. Another alternative uses different coordinates for the group effects. Its precise construction and comparison with the centered form are separate topics. The practical distinction is that centered sampling represents each effect directly, whereas alternative coordinates represent it indirectly through other quantities.
| Approach | What is represented in the sampling target? | Practical consideration |
|---|---|---|
| Centered | The group effects \(\mathbf b_j\) on the GLMM’s model scale | Can be effective when group data inform the effects; may have challenging dependence with population scales when data are weak |
| Marginalized | Population parameters after integrating out group effects | Can reduce the number of sampled latent effects, but marginal likelihood calculations may be demanding |
| Alternative effect coordinates | Group effects represented through a different set of coordinates | May change posterior dependence and sampling behavior; the coordinate definition matters |
Warning signs that a centered target is difficult include slow movement of group effects, strong posterior dependence between effects and their scale parameters, and poor HMC exploration such as repeated divergences. These symptoms do not identify the cause by themselves: inadequate data, a poorly scaled predictor, an unsuitable prior, and model misspecification can also contribute. Check that the likelihood, link, group structure, and covariance prior match the scientific design before attributing a sampling problem to parameterization alone.
Centered sampling has important limits. A group with few observations may provide little information about its effect, leaving the posterior strongly shaped by the population distribution. When a scale parameter is near zero, the group effects must also be near zero, which can create difficult dependence in the joint posterior. A centered parameterization does not fix an unrealistic normal random-effects assumption or resolve confounding between fixed and group-level terms. Nor does a successful run establish that the model is appropriate; posterior summaries still depend on the likelihood and prior assumptions.
A common mistake is to assume that “sample the effects directly” must always be the most natural and therefore the most efficient strategy. Direct representation is convenient for interpretation, but computational efficiency depends on the posterior geometry: the relationship between the likelihood information and the population distribution matters. Another mistake is to treat a random effect’s prior density as its posterior distribution. The posterior also includes the group’s likelihood and the priors on other unknown parameters.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- For \(b_j\mid\tau\sim N(0,\tau^2)\), what does the centered parameterization treat as the group-level parameter?
- In the Gaussian example, if \(n_j=4\), \(\sigma=1\), and \(\tau=1\), what are the likelihood precision and prior precision?
- For a logistic group with \(n=10\) and \(p=0.50\), calculate the likelihood curvature with respect to a scalar intercept.
- Why does a large prior density at one value of \(b_j\) not, by itself, establish that the posterior favors that value?
Solutions. (1) The parameter is \(b_j\) itself, expressed on the linear-predictor scale and assigned its population distribution directly. (2) The likelihood precision is \(n_j/\sigma^2=4/1=4\), and the prior precision is \(1/\tau^2=1\); both are in inverse squared outcome units for this Gaussian model. (3) The curvature is \(np(1-p)=10(0.50)(0.50)=2.50\) per squared log-odds unit. (4) The posterior combines the prior density with the likelihood and the priors on other unknown parameters; a prior density alone omits the information from the observed data.