Hierarchical Priors for Group-Level Scales

Why Model Several Scales Together?

A GLMM can contain several random-effect components: for example, a random-intercept scale and a random-slope scale, or scales for related outcomes. Each scale describes the spread of a different set of group-level effects. When the components are related, estimating every scale independently can leave weakly informed estimates highly variable. At the other extreme, forcing all components to have exactly the same scale can conceal genuine differences. A hierarchical prior offers an intermediate option: it allows the scales to differ, while relating them through a shared distribution.

The pooling here is among scale parameters. It is distinct from the conditional distribution of the group-level random effects given their scales. For instance, a random effect may be modeled as normal with a component-specific scale; the hierarchical prior then relates that scale to the scales of other components. The component scales are not assumed equal, and the group-level effects themselves are not being averaged together.

A Hierarchical Prior on Positive Scales

Let \(j=1,\ldots,J\) index related random-effect components, and let \(\tau_j>0\) be the scale for component \(j\). Because scales are positive, a convenient construction is to model their logarithms. If the scale has physical units, choose a common reference scale \(\tau_{\mathrm{ref}}\) in those units, so the logarithm is taken of a dimensionless ratio.

The implied prior for \(\tau_j\), conditional on \(\mu,\omega\), is lognormal, with density \(p(\tau_j\mid\mu,\omega)=\phi(\log(\tau_j/\tau_{\mathrm{ref}});\mu,\omega^2)/\tau_j\) for \(\tau_j>0\), where \(\phi(\,\cdot\,;\mu,\omega^2)\) is the normal density. The divisor \(\tau_j\) is the change-of-variables Jacobian. The conditional median scale is \(\tau_{\mathrm{ref}}\exp(\mu)\), not generally the conditional mean; the mean is \(\tau_{\mathrm{ref}}\exp(\mu+\omega^2/2)\).

In a GLMM, this prior is combined with the likelihood and the rest of the model. For example, if a component has group effects \(b_{gj}\), one might specify \(b_{gj}\mid\tau_j\sim N(0,\tau_j^2)\), with an appropriate conditional-independence structure. The prior above relates the \(\tau_j\); it does not by itself specify the entire GLMM. Its use assumes the components are sufficiently comparable for exchangeability to be plausible. They may differ, but the prior treats their labels symmetrically unless additional predictors or structure are introduced.

How Partial Pooling Arises

The basic shrinkage mechanism is clearest when the hyperparameters are fixed and the likelihood for a component's log scale is approximately normal. Write \(z_j\) for a data-based estimate of \(\theta_j\), and suppose \(z_j\mid\theta_j\sim N(\theta_j,s_j^2)\), where \(s_j\) is its standard error. This normal likelihood is a useful analytic benchmark; it is not a general claim that an estimated log scale from every GLMM has an exactly normal likelihood. The prior is \(\theta_j\mid\mu,\omega\sim N(\mu,\omega^2)\).

The result follows by multiplying the normal likelihood kernel by the normal prior kernel. Terms depending on \(\theta_j\) in the log posterior are \(-\tfrac12[(z_j-\theta_j)^2/s_j^2+(\theta_j-\mu)^2/\omega^2]\). Collecting the quadratic and linear terms gives \(-\tfrac12[(1/s_j^2+1/\omega^2)\theta_j^2-2(z_j/s_j^2+\mu/\omega^2)\theta_j]+\text{constant}\). Completing the square yields the stated variance and mean. Since \(0<B_j<1\), the posterior mean lies between the component estimate and the shared center. The data estimate receives more weight when it is precise (small \(s_j\)); the shared center receives more weight when the component is weakly informed or the prior scales are tightly clustered (small \(\omega\)).

This conditional result does not mean that \(\mu\) and \(\omega\) are known in a full hierarchical analysis. With hyperpriors, their posterior uncertainty must be integrated over to obtain the marginal posterior for each \(\theta_j\). The components then inform the hyperparameters, and those hyperparameters in turn affect the component scales. This mutual dependence induces posterior dependence among the scales. In general, the marginal posterior need not be normal, even though the conditional distribution in the benchmark calculation is normal.

Choosing Among Pooling Strategies

The hierarchical construction is one of several possible ways to relate the scales. The distinction is substantive, not merely computational.

A small posterior for \(\omega\) supports similarity among the component log scales; a larger value permits more separation. But \(\omega\) is itself uncertain, and a small number of components may provide limited information about it. Independent priors can be preferable when the scales are not meaningfully related; a common scale can be defensible when equality is scientifically justified. Hierarchical pooling is not automatically superior simply because it is more flexible.

Assumptions, Diagnostics, and Limitations

The exchangeability assumption deserves substantive scrutiny. The components should be comparable enough that their labels do not, by themselves, imply different prior distributions. A random-intercept scale and a random-slope scale may have different units or roles; using a common hierarchy requires a defensible scaling and reference-scale convention. If components have known systematic differences, a common exchangeable distribution may be inadequate. A richer model can relate their log scales to component-level predictors, but that requires additional structure and data.

Prior and posterior predictive checks can reveal implausible implications. Draw hyperparameters and scales from the prior, then inspect whether the resulting random-effect variation would produce plausible group-level patterns on the response scale. After fitting, inspect the posterior distributions of \(\omega\) and the \(\tau_j\), and compare implied variability across components. A posterior for \(\omega\) concentrated near zero may indicate that the data support near-equal scales, but it can also reflect a restrictive prior or weak information. Strong sensitivity to the prior on \(\omega\), highly skewed scale posteriors, or substantial posterior mass near very small scales warrants careful interpretation. Computational convergence checks remain necessary; a hierarchical prior does not guarantee reliable sampling.

A lognormal hierarchy assigns positive probability only to strictly positive scales. It does not place a point mass at exactly zero, so it is not by itself a model for a variance component that is absent. If exact zero is a scientific possibility, a model that explicitly includes a point mass is needed. Also, the normal-normal shrinkage equations are conditional analytic results, not a general closed-form solution for Bayesian GLMMs. In many GLMMs, the likelihood for random-effect scales is not normal in log scale, and the hyperparameters are unknown. Posterior computation must then use the full model rather than treating a noisy summary as an exact sufficient statistic.

Common Mistake: Treating the Shared Center as Known

A tempting but incomplete argument is: “The component estimate shrinks toward \(\mu\), so the posterior mean is just the weighted average in the formula.” That formula is correct only conditional on fixed \(\mu\) and \(\omega\) under the stated normal likelihood. In a hierarchical model, the hyperparameters are usually uncertain and estimated jointly with the component scales. Their uncertainty can widen or reshape a marginal posterior, and all the scales can influence the shared center. Report whether a calculation is conditional on fixed hyperparameters or has integrated over their posterior.