Prior Specification for Random-Intercept Variance

Why the Random-Intercept Prior Needs Context

In a generalized linear mixed model, a random intercept allows groups to differ after accounting for measured predictors. A prior for its variance therefore expresses prior beliefs about the amount of residual group-to-group heterogeneity. The numerical scale alone is not enough to judge whether that belief is sensible: the interpretation depends on the model’s link function and the outcome being studied.

For example, in a logistic model, a random intercept is measured in log odds. A standard deviation that looks small as a number can still imply appreciable differences in group odds. In a log-link model, the same numerical value corresponds to multiplicative differences in conditional means. Prior specification should begin with these implications, not with a default scale detached from the application.

Definition: For observations $i$ in group $j$, a random-intercept GLMM has linear predictor $$ \eta_{ij}=\mathbf{x}_{ij}^{\mathsf T}\boldsymbol{\beta}+b_j,\qquad b_j\mid \sigma_b\sim N(0,\sigma_b^2). $$ Here $g(E[Y_{ij}\mid b_j])=\eta_{ij}$, where $g$ is the link function; $\sigma_b\geq 0$ is the random-intercept standard deviation and $\sigma_b^2$ is its variance. The usual model assumes that the group effects are independent across groups and that responses are conditionally independent given the group effects and predictors, with the specified GLMM response distribution.

The scale $\sigma_b$ is measured in units of the linear predictor; the variance $\sigma_b^2$ is measured in squared units. In a logistic model those are log-odds units and squared log-odds units. In a Poisson log-link model they are log-rate units and squared log-rate units. The random intercept $b_j$ itself has mean zero in the population model; this does not mean that observed groups have identical intercepts.

Specify and Assess a Prior on the Scale

A common strategy is to put a prior directly on the nonnegative standard deviation, then interpret the resulting distribution of $\sigma_b$ and $\sigma_b^2$. For example, a half-normal prior with scale $A$ is the distribution of the absolute value of a $N(0,A^2)$ random variable. An exponential prior with rate $\lambda$ has density $\lambda e^{-\lambda\sigma_b}$ for $\sigma_b\geq 0$. A half-Student-$t$ prior is the distribution of the absolute value of a centered Student-$t$ variable with a specified degrees of freedom and scale.

These choices differ in their concentration near zero and their tail behavior. The parameter in the prior must be calibrated in the context of the link: ask what group-level contrasts are plausible, and what values of $\sigma_b$ would make those contrasts unusually large. Prior predictive checks can then reveal whether the model generates group outcomes that are credible before observing the data. The introductory material on sensitivity analysis for variance-component priors is useful when comparing more than one defensible choice.

Worked Example: A Half-Normal Prior in a Logistic Model

Suppose a hypothetical study records a binary outcome in several clinics and uses a logistic random-intercept model. Investigators judge that a one-standard-deviation shift in clinic log odds should usually correspond to no more than about a doubling or halving of the odds. Choose a half-normal prior scale that makes this judgment approximately its 95th-percentile boundary.

A log-odds shift of $\sigma_b$ multiplies the odds by $e^{\sigma_b}$. A doubling corresponds to $\sigma_b=\log(2)=0.6931$. For a half-normal prior with scale $A$, its 95th percentile is approximately $1.96A$, since the 95th percentile of the absolute value of a standard normal variable is approximately 1.96. Set that percentile equal to the proposed boundary: $$ 1.96A=0.6931 \quad\Longrightarrow\quad A=\frac{0.6931}{1.96}=0.3536. $$

Thus one possible specification is $\sigma_b\sim\text{HalfNormal}(0,0.3536^2)$. Its approximate 95th percentile is $1.96(0.3536)=0.6931$, and the corresponding variance percentile is $0.6931^2=0.4804$ squared log-odds units. At that scale, a one-standard-deviation shift multiplies the odds by $e^{0.6931}=2.00$. This is a calibration of the prior for the random-effect standard deviation, not a claim that 95% of clinic odds ratios lie between one-half and two.

The calculation assumes the logistic link and that the proposed doubling is meaningful for a one-standard-deviation shift in clinic effects. It does not determine whether such differences are plausible in the actual application; that judgment needs subject-matter knowledge and prior predictive assessment.

Transforming Between Standard Deviation and Variance

A prior described for $\sigma_b$ induces a prior for the variance $v_b=\sigma_b^2$. The induced density is important because a distribution that appears regular on the standard-deviation scale may behave differently near zero on the variance scale.

Write the transformation.
For $v_b>0$, $\sigma_b=\sqrt{v_b}$.

Compute the derivative.
$\left|d\sigma_b/dv_b\right|=1/(2\sqrt{v_b})$.

Transform the density.
The change-of-variables formula gives $p(v_b)=p(\sqrt{v_b})/(2\sqrt{v_b})$, where $p$ on the right is the density for $\sigma_b$.

For example, a half-normal density is finite and positive at $\sigma_b=0$. Consequently, its induced variance density is proportional to $v_b^{-1/2}$ near zero. This density becomes unbounded as $v_b$ approaches zero, but the singularity is integrable: it does not create a point mass at zero. A model that needs positive prior probability of exactly zero variance requires a separate point-mass component, not merely a continuous prior with density near zero.

Worked Example: Calibrating an Exponential Prior to a Rate Ratio

Consider a hypothetical Poisson GLMM with a log link, where $e^{b_j}$ multiplies group $j$’s conditional mean rate relative to a group with $b_j=0$, holding predictors fixed. Suppose the prior design goal is $P(\sigma_b>\log 2)=0.05$: only 5% of prior draws for the scale should exceed the value associated with a twofold one-standard-deviation rate multiplier.

For an exponential prior with rate $\lambda$, $P(\sigma_b>c)=e^{-\lambda c}$. Substituting $c=\log(2)$ and setting the tail probability to 0.05 gives $$ e^{-\lambda\log(2)}=0.05 \quad\Longrightarrow\quad \lambda=\frac{-\log(0.05)}{\log(2)} =\frac{2.9957}{0.6931}=4.322. $$

The resulting prior has median $\log(2)/\lambda=0.1604$, because the exponential median is $\log(2)/\lambda$. Its 95th percentile is $-\log(0.05)/\lambda=0.6931$. Thus, by construction, 95% of prior scale draws are at or below the scale at which a one-standard-deviation shift multiplies the conditional rate by two.

This statement concerns the random-effect standard deviation, not the probability that a particular pair of groups differs by more than a factor of two. Nor is $e^{\sigma_b}$ the multiplicative change in the population-average rate after integrating over normally distributed random effects. For a log link, that marginal mean includes the factor $E[e^{b_j}\mid\sigma_b]=e^{\sigma_b^2/2}$, which is a different quantity.

Tail Behavior and the Choice Among Priors

A prior with a relatively light tail can constrain very large group heterogeneity, while a heavier-tailed prior leaves more room for it. This distinction matters when the data contain few groups or limited information about between-group variation. A half-Student-$t$ prior can allow more large values than a half-normal with a comparable central scale; that flexibility is not automatically desirable if extremely large differences would be implausible.

The choice is not simply “weakly informative” versus “informative.” A proper prior can be weakly informative in the sense that it rules out implausible scales while retaining a broad range of plausible ones. A prior whose tail permits enormous effects can have strong consequences for prior predictive outcomes, even if much of its density is near zero. Conversely, a prior concentrated near zero may suppress heterogeneity that the application makes plausible.

Worked Example: Reading a Half-Student-t Prior in Odds Units

Suppose a hypothetical logistic GLMM uses $\sigma_b\sim\text{Half-}t_3(0,0.4)$, meaning the absolute value of a $t$ random variable with 3 degrees of freedom and scale 0.4. What does the tail imply about large random-intercept scales?

The event $\sigma_b>0.8$ is equivalent to a standard $t_3$ variable exceeding $0.8/0.4=2$ in absolute value. Because the prior is half-$t$, its upper-tail probability is the one-sided $t_3$ tail at 2, approximately 0.0697. Therefore, about 7% of prior draws place the standard deviation above 0.8 log-odds units. A shift of 0.8 log odds corresponds to an odds multiplier of $$ e^{0.8}=2.226. $$

This does not mean that 7% of groups have odds more than 2.226 times another group’s odds. It means that approximately 7% of prior draws for the common scale exceed 0.8. Conditional on a particular scale, individual group effects are still normally distributed as $N(0,\sigma_b^2)$. Since the half-$t$ has 3 degrees of freedom, its variance exists; the induced prior mean of $\sigma_b^2$ is $0.4^2(3/(3-2))=0.4800$ squared log-odds units.

The tail probability should be judged against the application. If odds shifts of this magnitude would imply implausible prior predictive event rates, the scale or tail choice needs reconsideration. If such shifts are possible and the group-level data are sparse, a heavier tail may be defensible, provided the resulting predictions remain reasonable.

Diagnostics, Comparisons, and Common Pitfalls

Assess a proposed prior by simulating $\sigma_b$, then drawing group effects and outcomes under the model’s assumed link and response distribution. Examine group contrasts, predicted probabilities or rates, and the range of group-level outcomes. The check is not whether every simulated dataset resembles the eventual data; it is whether the prior assigns substantial probability to scenarios that are scientifically implausible or nearly impossible under the study design.

The main alternatives are different prior families on the same scale, or a prior placed directly on the variance rather than the standard deviation. Half-normal priors offer a comparatively light tail; exponential priors permit a direct tail-probability calibration; half-Student-$t$ priors allow heavier tails. Direct variance priors can be convenient in some models, but their shape must be interpreted in variance units and their behavior near zero checked. None of these choices is universally best. Compare defensible alternatives using prior predictive implications and, when data are available, the prior sensitivity analysis.

Prior choice	Useful feature	Question to check
Half-normal on $\sigma_b$	Direct scale interpretation and a comparatively light tail	Does the chosen scale rule out plausible large heterogeneity?
Exponential on $\sigma_b$	A tail probability can be calibrated directly	Does the implied concentration near zero and tail match the application?
Half-Student-$t$ on $\sigma_b$	Allows more probability in the tail	Do tail draws produce implausibly extreme group predictions?
Prior directly on $v_b$	Works on the variance scale itself	Are the units and induced behavior near zero appropriate?

A common mistake is to treat a numerical prior scale as though it had the same meaning in every GLMM. A scale of 0.5 is not inherently “small”: in a logistic model it describes log-odds variation, while in a log-link model it describes log-rate variation. The correction is to translate candidate scales through the link function and inspect implied group-level differences and outcomes.

Confusing variance and standard deviation. A prior scale specified for $\sigma_b$ is not a prior scale for $\sigma_b^2$; squaring changes both the units and the distribution.
Reading a continuous prior as a test of zero variance. A continuous prior on $\sigma_b\geq0$ assigns probability zero to the exact value $\sigma_b=0$. It can favor small values without assigning positive probability to no heterogeneity.
Interpreting a scale quantile as a group-contrast quantile. A 95th percentile for $\sigma_b$ describes the scale parameter. Group effects and pairwise contrasts also vary conditional on that scale.
Checking only the center of a heavy-tailed prior. Prior predictive checks should include tail draws, because a small amount of prior probability can still generate extreme predicted outcomes.

Key takeaway: Specify the random-intercept prior in a way that can be translated into plausible group heterogeneity on the model’s link scale. Check both the variance-scale implications and the prior predictive outcomes; a familiar prior family does not guarantee a suitable calibration.

Check Your Understanding

Answer each question, then compare with the brief solutions below.

In a logistic random-intercept model, what odds multiplier corresponds to a log-odds shift of 0.4?
For an exponential prior with rate $\lambda=2$ on $\sigma_b$, calculate $P(\sigma_b>0.5)$.
A continuous prior on $\sigma_b\geq0$ has a density that increases without bound as $\sigma_b$ approaches zero. Does it assign positive probability to exactly zero variance?
In a Poisson log-link model, distinguish the multiplier $e^{\sigma_b}$ from the marginal mean multiplier $e^{\sigma_b^2/2}$.

Solutions. (1) $e^{0.4}=1.492$, so a 0.4 log-odds shift multiplies the odds by approximately 1.49. (2) The exponential survival function gives $P(\sigma_b>0.5)=e^{-2(0.5)}=e^{-1}=0.3679$. (3) No. A continuous prior still assigns probability zero to the single point $\sigma_b=0$, even if its density is unbounded near zero. (4) $e^{\sigma_b}$ is the conditional mean-rate multiplier for a group effect equal to one standard deviation. The marginal mean multiplier $e^{\sigma_b^2/2}$ results from averaging $e^{b_j}$ over $b_j\sim N(0,\sigma_b^2)$; the two describe different quantities.

Prior choice	Useful feature	Question to check
Half-normal on \(\sigma_b\)	Direct scale interpretation and a comparatively light tail	Does the chosen scale rule out plausible large heterogeneity?
Exponential on \(\sigma_b\)	A tail probability can be calibrated directly	Does the implied concentration near zero and tail match the application?
Half-Student-\(t\) on \(\sigma_b\)	Allows more probability in the tail	Do tail draws produce implausibly extreme group predictions?
Prior directly on \(v_b\)	Works on the variance scale itself	Are the units and induced behavior near zero appropriate?