Deriving the Centered and Noncentered Random-Effects Forms

Why Two Representations Describe the Same Model

A random-intercept model describes group effects through a distribution whose scale is unknown. It can be written either in terms of the group effects themselves or in terms of standardized latent variables that are multiplied by the unknown scale. These are different coordinates for the same model, not different assumptions about the data. The earlier material on centered parameterization for group-level effects introduced the centered form; here the transformation and its effect on the posterior density are derived explicitly.

The distinction matters in Bayesian computation because a posterior density is a density with respect to a particular set of coordinates. Rewriting the model changes that density by a Jacobian factor. Omitting the factor, or treating the standardized effects as if they had the same prior as the original effects, generally changes the probability model rather than merely changing its notation.

Model and Assumptions

Let \(j=1,\ldots,J\) index groups and \(i=1,\ldots,n_j\) index observations within group \(j\). Let \(Y_{ij}\) be the response and \(x_{ij}\) the covariate vector, including any fixed-effect design terms. For a generalized linear mixed model, assume the conditional distribution of each response belongs to an exponential family, with conditional mean \(\mu_{ij}\) related to the linear predictor by a link function \(g\):

g(\mu_{ij})=x_{ij}^{\mathsf T}\beta+b_j.

Here \(\beta\) is the vector of fixed effects and \(b_j\) is the random intercept for group \(j\). Conditional on \(\beta\) and all group effects, observations are independent, with likelihood \(p(y\mid\beta,b)\). The random-intercept model assumes

b_j\mid\tau\ \mathrel{\mathop{\sim}^{\mathrm{ind}}}\ N(0,\tau^2), \qquad j=1,\ldots,J,

where \(\tau>0\) is the group-level standard deviation. The model also requires a prior for the fixed effects and for \(\tau\); write their joint density as \(p(\beta,\tau)\). The derivation below assumes the same prior \(p(\beta,\tau)\) in both representations and that it has support on \(\tau>0\). No particular link function or response family is needed for the transformation.

Definition: The centered representation uses the group effects \(b_j\) directly, with \(b_j\mid\tau\sim N(0,\tau^2)\). The noncentered representation writes \(b_j=\tau z_j\), where the standardized effects satisfy \(z_j\mathrel{\mathop{\sim}^{\mathrm{ind}}}N(0,1)\). “Centered” and “noncentered” refer to the parameterization of the hierarchy, not to whether the group effects have mean zero.

Deriving the Transformation and Posterior

For any fixed \(\tau>0\), the transformation between the two sets of group-level coordinates is

b_j=\tau z_j,\qquad z_j=\frac{b_j}{\tau},\qquad j=1,\ldots,J.

Substituting \(b_j=\tau z_j\) into the linear predictor gives the noncentered model:

g(\mu_{ij})=x_{ij}^{\mathsf T}\beta+\tau z_j, \qquad z_j\mathrel{\mathop{\sim}^{\mathrm{ind}}}N(0,1).

The conditional mean, and therefore the likelihood for any fixed \((\beta,\tau,z)\), is the same as in the centered representation evaluated at \(b_j=\tau z_j\). But the prior density must also be transformed. For fixed \(\tau\), the derivative matrix of \(b=(b_1,\ldots,b_J)\) with respect to \(z=(z_1,\ldots,z_J)\) is \(\tau I_J\), so its determinant in absolute value is \(\tau^J\). The joint transformation from \((z,\tau)\) to \((b,\tau)\) has the same determinant: its derivative matrix is block triangular, with diagonal blocks \(\tau I_J\) and \(1\).

\left|\frac{\partial(b,\tau)}{\partial(z,\tau)}\right|=\tau^J.

In centered coordinates, the joint posterior density, up to its normalizing constant, is

p(\beta,\tau,b\mid y) \ \propto\ p(y\mid\beta,b)\,p(\beta,\tau) \prod_{j=1}^{J}\frac{1}{\tau}\, \phi\!\left(\frac{b_j}{\tau}\right),

where \(\phi\) is the standard normal density. Applying the change-of-variables formula gives the posterior density with respect to \(d\beta\,d\tau\,dz\):

\begin{aligned} p(\beta,\tau,z\mid y) &=p(\beta,\tau,b=\tau z\mid y)\, \left|\frac{\partial(b,\tau)}{\partial(z,\tau)}\right|\\ &\propto p(y\mid\beta,b=\tau z)\,p(\beta,\tau) \prod_{j=1}^{J}\frac{1}{\tau}\phi(z_j)\,\tau^J\\ &=p(y\mid\beta,b=\tau z)\,p(\beta,\tau) \prod_{j=1}^{J}\phi(z_j). \end{aligned}

The factors \(1/\tau\) in the \(J\) centered normal densities cancel the Jacobian \(\tau^J\). Thus the noncentered prior is \(z_j\) independent standard normal effects, independent of \((\beta,\tau)\) under the specified prior \(p(\beta,\tau)\); the likelihood still depends on \(\tau z_j\). This equality of transformed densities is the precise sense in which the posterior is preserved.

Theorem: For \(\tau>0\), the centered posterior and the noncentered posterior describe the same probability distribution under the one-to-one transformation \(b_j=\tau z_j\), provided the noncentered density includes the change-of-variables Jacobian. Posterior probabilities and summaries of corresponding quantities agree after transforming draws or events between coordinates.

Worked Examples

Worked Example: Transforming a Group Effect and Its Linear Predictor

Suppose a hypothetical binary-response GLMM uses the logit link. For one observation, let \(\beta_0=-1\), \(\beta_1=0.4\), covariate value \(x=1.5\), group standard deviation \(\tau=0.8\), and standardized group effect \(z_j=-0.5\). Calculate the centered effect and the conditional success probability in both forms.

The centered group effect is

b_j=\tau z_j=(0.8)(-0.5)=-0.4.

The centered linear predictor is \(\eta=-1+(0.4)(1.5)-0.4=-0.8\). The noncentered predictor gives the same value: \(\eta=-1+(0.4)(1.5)+(0.8)(-0.5)=-0.8\). Thus the conditional success probability is

P(Y_{ij}=1\mid\beta,\tau,z_j) =\frac{1}{1+\exp(-\eta)} =\frac{1}{1+\exp(0.8)} \approx 0.3100.

Both parameterizations therefore assign probability about \(0.3100\) to success for this observation, conditional on the stated parameters. The equality follows from substitution, not from an approximation to the link function.

Worked Example: The Likelihood Is Unchanged

Consider hypothetical binary data from two groups. Suppose the parameter values in a centered fit give conditional success probabilities \(0.4013\) for each observation in group 1 and \(0.2142\) for each observation in group 2. Group 1 has three successes and two failures; group 2 has one success and three failures. Calculate the conditional likelihood, then explain what happens under the noncentered transformation.

Under conditional independence, the likelihood is the product of the success probabilities for successes and their complements for failures:

\begin{aligned} L &=(0.4013)^3(1-0.4013)^2(0.2142)^1(1-0.2142)^3\\ &=(0.4013)^3(0.5987)^2(0.2142)(0.7858)^3\\ &\approx 0.002408. \end{aligned}

If the same group effects are written as \(b_j=\tau z_j\), each linear predictor is unchanged, so each conditional probability remains the same. The product likelihood therefore remains approximately \(0.002408\). This is a likelihood calculation at fixed corresponding parameter values; it does not by itself establish equality of posterior densities, which also requires transforming the prior density correctly.

Worked Example: Calculating the Jacobian Factor

Take \(J=2\), \(\tau=2\), and standardized effects \(z=(0.5,-1)\). Find the corresponding centered effects and compare the normal prior densities in the two coordinate systems.

The centered effects are \(b=(2(0.5),2(-1))=(1,-2)\). The absolute Jacobian determinant is

\left|\frac{\partial(b_1,b_2,\tau)} {\partial(z_1,z_2,\tau)}\right| =\tau^J=2^2=4.

The centered conditional prior density at \(b=(1,-2)\), given \(\tau=2\), is

\begin{aligned} p(b\mid\tau) &=\prod_{j=1}^{2}\frac{1}{2\sqrt{2\pi}} \exp\!\left(-\frac{b_j^2}{2(2^2)}\right)\\ &=\frac{1}{8\pi}\exp\!\left(-\frac{1^2+(-2)^2}{8}\right) \approx 0.02130. \end{aligned}

The corresponding density in \(z\)-coordinates is obtained from the exact centered density times 4. Equivalently, it is the product of standard normal densities:

\phi(0.5)\phi(-1) =\frac{1}{2\pi}\exp(-0.625) \approx 0.08519.

The displayed rounded centered density should not be multiplied to obtain a more precise answer: \(0.02130\) is rounded, whereas the transformed density rounds to \(0.08519\). The Jacobian changes the numerical density because the coordinate volume changes; it does not change probabilities of corresponding regions.

Why the Posterior Is Preserved

The posterior-preservation result is an application of the change-of-variables formula. For any posterior event \(A\) in centered coordinates, let \(A^\star\) be the corresponding set of noncentered coordinates, defined by \(b=\tau z\). Then integration of the transformed density gives

\begin{aligned} P\big((\beta,\tau,b)\in A\mid y\big) &=\int_A p(\beta,\tau,b\mid y)\,d\beta\,d\tau\,db\\ &=\int_{A^\star}p(\beta,\tau,\tau z\mid y)\, \tau^J\,d\beta\,d\tau\,dz\\ &=P\big((\beta,\tau,z)\in A^\star\mid y\big). \end{aligned}

The second equality substitutes \(db=\tau^J dz\), and the last uses the transformed posterior density. For example, a posterior interval for a particular group effect \(b_j\) corresponds to the event \(\tau z_j\) lying in that interval. It is not generally the same numerical interval for \(z_j\), because \(z_j=b_j/\tau\) depends on the unknown scale.

The marginal likelihood is preserved as well. Integrating the centered likelihood and prior over \(b\) is equivalent to integrating their transformed product over \(z\), because the Jacobian is included in the change of variables. The evidence for the model therefore does not depend on which coordinates are used.

Assumptions, Diagnostics, and Common Errors

The transformation is one-to-one and differentiable when \(\tau>0\), and the derivation assumes a continuous prior on this positive scale. At \(\tau=0\), every value of \(z\) maps to \(b=0\), so the transformation is no longer one-to-one and its Jacobian is zero. A model that assigns positive posterior probability to the boundary as a separate point mass needs that boundary component handled separately; it is not represented by simply applying the interior change-of-variables formula there.

A useful check in a derivation or implementation is to compare corresponding quantities at fixed values: calculate \(b_j=\tau z_j\), verify that the linear predictors agree, and verify that the likelihoods agree. Then check the prior density transformation, including the factor \(\tau^J\). These checks catch errors that a visually plausible posterior formula may conceal.

Dropping the Jacobian. The wrong reasoning is that changing coordinates only renames \(b_j\) as \(\tau z_j\), so the density stays unchanged. The correction is that densities are relative to coordinate volume; for \(J\) effects the factor is \(\tau^J\).
Giving \(b_j\) and \(z_j\) the same scale prior. The wrong reasoning is that both are group effects with mean zero. The correction is \(b_j\mid\tau\sim N(0,\tau^2)\), whereas \(z_j\sim N(0,1)\); the scale enters through \(b_j=\tau z_j\).
Confusing likelihood equality with posterior equality. Matching predictors ensures matching likelihoods, but posterior equality also requires compatible priors and the Jacobian.
Interpreting the Jacobian as an extra model assumption. It is not a new prior preference. It is the density adjustment required to describe the same probability measure in different coordinates.

The centered and noncentered forms are exact alternatives for expressing the same hierarchy when their priors are transformed consistently. They can produce different computational behavior for a sampling algorithm, but that question is separate from whether the underlying posterior distributions are mathematically equivalent. Parameterization also does not fix misspecified random-effect distributions, inappropriate covariates, or dependence structures omitted from the model.

Key takeaway: Write \(b_j=\tau z_j\), keep the likelihood at the corresponding linear predictor, and multiply the centered-coordinate density by \(\tau^J\) when expressing it in noncentered coordinates. With that Jacobian, the two forms preserve the same posterior probabilities.

Check Your Understanding

Answer each question, then compare with the solutions below.

For \(J=4\) groups and \(\tau=1.5\), what is the absolute Jacobian determinant for the transformation from \(z\) to \(b\), holding \(\tau\) as a coordinate?
If \(\tau=0.6\) and \(z_j=2\), calculate \(b_j\). Does the conditional likelihood change when the centered form uses this \(b_j\)?
Why is it incorrect to use the centered posterior density unchanged as a density with respect to \(d\beta\,d\tau\,dz\)?
What special issue arises at \(\tau=0\), and why does the interior Jacobian argument not apply there?

Solutions. (1) The determinant is \(\tau^J=(1.5)^4=5.0625\). (2) \(b_j=(0.6)(2)=1.2\); the conditional likelihood is unchanged because the corresponding linear predictor is unchanged. (3) The centered density is with respect to \(db\), not \(dz\); transforming coordinates requires multiplying by \(\tau^J\). Without that factor, the density generally integrates to the wrong total and describes a different distribution. (4) When \(\tau=0\), every \(z\) maps to \(b=0\), so the mapping is not one-to-one and its determinant is zero. A boundary point mass must be treated separately from the continuous transformation for \(\tau>0\).