Choosing Centered or Noncentered Sampling

Why Parameterization Is a Sampling Decision

A Bayesian GLMM can have the same statistical model but very different computational behavior depending on how its group effects are represented. The centered and noncentered forms describe the same probability model when transformed correctly. They do not, however, present the posterior to a sampler in the same coordinates. The useful choice depends largely on how strongly each group’s observations inform its random effect relative to the prior scale.

The introductory material on centered and noncentered random-effects forms establishes the two representations. Here the emphasis is on choosing between them: the relevant comparison, why it affects posterior geometry, and what to do when groups provide different amounts of information. The criteria below are guides, not universal thresholds; GLMM likelihoods are generally not exactly quadratic in the random effects.

Definition: In a centered parameterization, group effects \(b_j\) are sampled directly under their hierarchical distribution, such as \(b_j\mid\tau\sim N(0,\tau^2)\). In a noncentered parameterization, introduce standardized effects \(z_j\sim N(0,1)\) and set \(b_j=\tau z_j\). In a GLMM, \(b_j\) enters the group’s linear predictor, and the likelihood is evaluated using that predictor.

For illustration, let \(y_{ij}\) denote the response for observation \(i\) in group \(j\), let \(x_{ij}\) denote its covariates, and let \(b_j\) be a scalar random intercept. A typical GLMM specifies a conditional response distribution from an exponential-family model, with a link relating its conditional mean to \(\eta_{ij}=x_{ij}^{T}\beta+b_j\). Conditional on \(\beta\) and the group effects, observations are independent; the group effects are conditionally independent given their shared scale \(\tau>0\), with \(b_j\mid\tau\sim N(0,\tau^2)\). The same decision applies to vector-valued effects, though the information comparison then involves more than one direction.

Information Relative to the Prior Scale

A group with many informative observations can pin down its effect through the likelihood. A group with little information leaves its effect largely governed by the hierarchical prior. The balance can be summarized locally. Let \(\ell_j(b_j)\) be the group’s log likelihood as a function of its random effect, with other model parameters held at specified values. Near a relevant value of \(b_j\), the likelihood can be approximated by a quadratic, with local information \(I_j=-\partial^2\ell_j(b_j)/\partial b_j^2\). For a scalar effect, \(I_j\) is nonnegative when the local likelihood is concave. Compare \(I_j\) with the prior precision \(1/\tau^2\), using the dimensionless quantity \(I_j\tau^2\).

Practical guide: If \(I_j\tau^2\) is small, the likelihood contributes little curvature relative to the prior, so a noncentered form is often easier to sample. If \(I_j\tau^2\) is large, the data tightly constrain \(b_j\), and a centered form is often more efficient. Values near one, or groups spanning both regimes, call for empirical comparison or a mixed parameterization. This local ratio is a guide to geometry, not a formal cutoff or a guarantee of good sampling.

The reason is visible in a normal approximation. Suppose, for a single group, a summary \(Y_j\) has likelihood \(Y_j\mid b_j\sim N(b_j,1/I_j)\), and the prior is \(b_j\mid\tau\sim N(0,\tau^2)\). This is a local working model for understanding geometry, not an assertion that every GLMM has a normal likelihood. The joint density in centered coordinates is proportional to the likelihood times the scale-dependent prior for \(b_j\). When \(I_j\) is small, the data allow a wide range of \(b_j\); its prior spread changes with \(\tau\). Thus the joint posterior can form a funnel: at small \(\tau\), \(b_j\) must be near zero, while at larger \(\tau\), a much wider range is plausible. A sampler must explore both the narrow and wide regions.

In noncentered coordinates, the prior for \(z_j\) is a fixed standard normal distribution, and the likelihood depends on the product \(\tau z_j\). When the data are weak, much of the posterior structure comes from the stable prior for \(z_j\), rather than a narrow, scale-dependent distribution for \(b_j\). When the data are strong, however, the likelihood holds \(b_j=\tau z_j\) near a data-supported value. Changes in \(\tau\) then require compensating changes in \(z_j\), which can create strong posterior dependence in the noncentered coordinates. This is why strong information tends to favor centering.

A Short Derivation of the Information Comparison

The normal working model gives the comparison explicitly. Its log posterior for \(b_j\), conditional on \(\tau\) and apart from constants, is the sum of the log likelihood and log prior:

-\frac{I_j}{2}(Y_j-b_j)^2-\frac{1}{2\tau^2}b_j^2.

Collecting the terms involving \(b_j\) gives a quadratic with precision \(I_j+1/\tau^2\). Completing the square yields the conditional posterior:

b_j\mid Y_j,\tau\sim N\left( \frac{I_j}{I_j+1/\tau^2}Y_j,\, \frac{1}{I_j+1/\tau^2} \right).

If \(I_j\tau^2\ll 1\), the posterior variance is close to \(\tau^2\): the prior scale largely determines the spread of \(b_j\). If \(I_j\tau^2\gg 1\), the posterior variance is close to \(1/I_j\): the data largely determine its spread. The first regime explains the centered funnel; the second explains why a noncentered representation can couple \(z_j\) strongly to \(\tau\). The algebra justifies the comparison for this model. In a GLMM, use it as a local approximation because the likelihood curvature may change with \(b_j\), the fixed effects, and the observed responses.

Worked Example: Comparing Information in Logistic Groups

Suppose a hypothetical logistic random-intercept GLMM has \(b_j\) with prior standard deviation \(\tau=0.8\). For a group with an intercept-only contribution and fitted probabilities near \(0.5\), the logistic likelihood information for the intercept is approximately \(\sum_i p_{ij}(1-p_{ij})\), or about \(n_j(0.25)\). This calculation assumes the probabilities are indeed near \(0.5\); it is a local approximation, not a fixed property of the group.

For a group of 40 observations, the approximate information is \(I_j=40(0.25)=10\). The prior variance is \(\tau^2=0.8^2=0.64\), so \(I_j\tau^2=10(0.64)=6.4\). The likelihood is strong relative to this prior scale, suggesting that a centered form is worth trying for this group.

For a group of 3 observations under the same probability approximation, \(I_j=3(0.25)=0.75\), and \(I_j\tau^2=0.75(0.64)=0.48\). The prior scale is comparatively influential, so a noncentered form is a reasonable starting choice for this group.

These calculations check the assumptions behind the recommendation: the groups are treated as conditionally independent given their effects and shared parameters, and the stated probabilities are used only as local values for approximating curvature. If probabilities are near zero or one, the terms \(p_{ij}(1-p_{ij})\) are smaller, so the same group size can provide less intercept information. Group size alone does not determine the answer.

Applying the Guide to GLMMs

For a logistic random intercept, the conditional log likelihood has curvature \(\sum_i p_{ij}(1-p_{ij})\) with respect to \(b_j\), evaluated at the current or otherwise relevant fitted probabilities. Rare outcomes, strong imbalance, and covariate patterns that produce fitted probabilities near zero or one can therefore weaken effective information. A group with many observations may still be weakly informative about its random effect. For random slopes, information also depends on the within-group design: observations with little variation in a slope covariate may provide limited information about that slope even when the group is large.

For a vector of random effects, let \(\Sigma\) be their prior covariance matrix and \(H_j\) the local negative Hessian of the group log likelihood with respect to \(b_j\). The matrix \(\Sigma^{1/2}H_j\Sigma^{1/2}\) expresses likelihood curvature on prior-standardized scales. Its eigenvalues indicate how strongly the data inform different directions: small values correspond to directions that remain relatively prior-dominated, while large values correspond to directions strongly constrained by the likelihood. This is a local diagnostic concept, not a substitute for assessing the model or sampling behavior.

Worked Example: Choosing Coordinates Across Four Groups

Consider a hypothetical random-intercept model with shared prior scale \(\tau=0.5\), so \(\tau^2=0.25\). Suppose local information approximations for four groups are \(20\), \(12\), \(0.5\), and \(0.2\). The corresponding information-to-prior-scale quantities are:

20(0.25)=5,\qquad 12(0.25)=3,\qquad 0.5(0.25)=0.125,\qquad 0.2(0.25)=0.05.

The first two groups are relatively well informed, suggesting centered coordinates for their effects. The last two are weakly informed, suggesting noncentered coordinates. If the implementation allows group-specific choices, a mixed parameterization is a defensible starting point. It does not alter the model: each effect still has the same \(N(0,0.25)\) prior conditional on the shared scale, and the likelihood uses the same effect value.

The recommendation assumes that these are reasonable local information values and that the groups share the stated scale. It does not establish that the mixed form will outperform either all-centered or all-noncentered sampling in every implementation. A practical comparison should fit equivalent formulations and check whether posterior summaries agree, then compare sampling behavior under the same data, prior, and computational settings.

Choosing and Checking a Parameterization

Identify the hierarchy.
Write the conditional distribution of group effects and the way they enter the GLMM likelihood. Confirm that the centered and noncentered forms encode the same prior and likelihood.

Assess effective information.
Use likelihood curvature or a defensible local approximation, taking account of fitted probabilities, covariates, and the prior scale. For random slopes, assess the information in each direction rather than relying on group size.

Choose a starting representation.
Favor noncentering when effects are weakly informed, centering when they are strongly informed, and consider a mixed form when groups or effect directions differ substantially.

Compare equivalent fits.
Check that alternative parameterizations yield compatible posterior summaries for the same scientific quantities. Compare sampling behavior as well; agreement of summaries alone does not show that a chain has explored the posterior adequately.

The alternatives are not different scientific models. Centering samples the effects in their original hierarchical scale; noncentering samples standardized effects and maps them to that scale. A mixed representation applies the two coordinate choices to different groups or effect directions while preserving the same target distribution. In contrast, changing the likelihood, prior, or dependence assumptions changes the model rather than merely its parameterization.

When choosing, inspect whether the anticipated difficulty matches the data and whether equivalent fits give compatible results. Poor exploration, unstable estimates, or warnings from a sampler can prompt a parameterization comparison, but they do not by themselves prove which coordinates are responsible. A parameterization choice is one part of computation, not a remedy for an incorrect model, an unsuitable prior, or insufficiently informative data.

Worked Example: When Group Size Gives the Wrong Impression

Suppose two hypothetical groups each contain 30 binary observations and share a random-intercept prior with \(\tau=1\). In the first group, fitted probabilities are approximately \(0.5\), giving an approximate information of \(30(0.5)(0.5)=7.5\), and \(I_j\tau^2=7.5\). In the second group, fitted probabilities are approximately \(0.02\), giving \(30(0.02)(0.98)=0.588\), and \(I_j\tau^2=0.588\).

Although the group sizes match, the second group has much lower local information about its intercept under these fitted probabilities. The first group leans toward a centered starting point; the second leans toward a noncentered one. This comparison assumes the stated probabilities are sensible local approximations and uses the logistic intercept information formula. It does not mean that the second group necessarily has poor information about every model quantity, nor that its parameterization is determined by one calculation.

Limitations and Common Pitfalls

Treating a rule of thumb as a theorem. The value \(I_j\tau^2\) is a local comparison, not a threshold that guarantees efficient sampling. Curvature can vary across the posterior.
Using group size as the whole diagnosis. In a GLMM, outcome balance, covariate patterns, separation, and random-slope design all affect how much information observations contribute about a group effect.
Assuming that one parameterization is always best. Weak and strong information favor different geometries. A single model can contain both regimes.
Confusing a coordinate change with a prior change. In noncentered coordinates \(b_j=\tau z_j\), the transformation must be implemented consistently so the induced prior for \(b_j\) remains the intended one.
Reading a numerical approximation as a full posterior diagnosis. A local Hessian cannot capture every feature of a non-Gaussian, hierarchical posterior; use it to guide comparison, not to replace it.

The central practical conclusion is conditional: center effects when their likelihood information is strong relative to their prior scale, and noncenter them when the likelihood is weak relative to that scale. When this balance varies, compare a mixed form as well as the two uniform alternatives. The posterior model remains the same only if the transformation and prior are handled correctly.

Check Your Understanding

Answer each question, then compare with the brief solutions.

A scalar group effect has prior standard deviation \(\tau=0.6\) and local information \(I_j=0.5\). Calculate \(I_j\tau^2\) and give a parameterization starting point.
In a logistic random-intercept model, why can two groups of equal size provide different information about their intercepts?
What geometry motivates noncentering when a group is weakly informed?
For random slopes, why is group size alone insufficient to choose a parameterization?
Does changing from centered to noncentered coordinates change the scientific posterior distribution?

Solutions. (1) \(I_j\tau^2=0.5(0.6^2)=0.5(0.36)=0.18\), a relatively small value, so noncentering is a reasonable starting point, not a guarantee. (2) Logistic intercept information depends locally on \(\sum_i p_{ij}(1-p_{ij})\); fitted probabilities near zero or one contribute less than probabilities near \(0.5\). (3) Weak information leaves the centered effect’s spread strongly dependent on \(\tau\), producing a funnel-like geometry; standard-normal \(z_j\) coordinates can reduce that scale-dependent prior geometry. (4) Slope information depends on the within-group covariate design and can be weak in a particular direction even with many observations. (5) No, provided the transformation and prior are implemented correctly; the coordinates change, not the underlying model or posterior for the same scientific quantities.