Calibrating Prior Scales from Plausible Group Heterogeneity

From Plausible Heterogeneity to a Prior Scale

A prior for a group-level standard deviation should be expressed in terms of variation that matters for the application. A scale such as “2” is not informative by itself: it may mean two log-odds units, two days, or two units of a transformed predictor. Calibration starts with a scientific statement about plausible group differences, translates that statement to the model’s scale, and then chooses a prior whose probability statements match the intended belief.

Consider a GLMM with a scalar random intercept. Conditional on group \(j\)’s random effect, the response distribution has mean \(\mu_{ij}\), and \(\eta_{ij}=g(\mu_{ij})=\mathbf{x}_{ij}^{T}\boldsymbol{\beta}+b_j\). Assume \(b_j\mid\sigma\sim N(0,\sigma^2)\), where \(\sigma\geq 0\) is the group-level standard deviation. The fixed effects and link function determine the baseline predictor; \(b_j\) shifts that predictor for group \(j\). In this tutorial, calibration concerns the prior for \(\sigma\), before observing the responses.

This statement is about the standard deviation itself. It does not say that a random effect has probability \(\alpha\) of exceeding \(U\), nor that the response has probability \(\alpha\) of changing by a particular amount. To connect \(U\) to group deviations, one must also use the conditional random-effect distribution and the link function.

Deriving the Exponential-Rate Calibration

For the exponential prior, the survival probability above \(U\) is the integral of the density from \(U\) to infinity:

Setting this probability equal to \(\alpha\) gives \(e^{-\lambda U}=\alpha\). Taking logarithms yields \(-\lambda U=\log(\alpha)\), and hence \(\lambda=-\log(\alpha)/U\). Because \(\lambda\) has units reciprocal to \(\sigma\), \(U\) must be expressed in the same units as the random effect. A smaller \(\alpha\) puts less prior probability above the chosen threshold; a larger \(U\), with \(\alpha\) fixed, gives a smaller rate and a more dispersed prior.

A convenient way to choose \(U\) is to describe a conditional range of random effects at that scale. Since \(b_j\mid\sigma=U\sim N(0,U^2)\), its central 95% interval is approximately \([-1.96U,1.96U]\). Thus, if a plausible statement is that a group’s random-intercept contribution would be about \(\pm\Delta\) on the linear-predictor scale at the threshold scale, set \(U=\Delta/1.96\). This is a calibration convention, not a claim that 95% of groups must fall in that range after averaging over the prior for \(\sigma\).

Calibrating on the Response Scale

For a nonlinear link, a proposed change on the response scale must first be translated at a stated baseline. In a binary model with logit link, the predictor corresponding to probability \(p\) is \(\operatorname{logit}(p)=\log\{p/(1-p)\}\). A change from baseline \(p_0\) to \(p_1\) requires the predictor shift \(\Delta=\operatorname{logit}(p_1)-\operatorname{logit}(p_0)\). The inverse-logit then maps shifted predictors back to probabilities. Because this transformation is nonlinear, the same \(\Delta\) need not represent the same percentage-point change at different baseline probabilities.

For a Poisson GLMM with log link, the mean is \(\mu=\exp(\eta)\). A predictor shift \(\Delta\) multiplies the mean by \(\exp(\Delta)\), regardless of the baseline mean. Thus a multiplicative rate benchmark \(R\) corresponds to \(\Delta=\log(R)\). For an identity link, the predictor shift is an additive change in the mean in the response’s units. In every case, state whether the threshold describes a single group’s deviation from the population predictor or a contrast between two groups.

Deviations, Group Contrasts, and Other Priors

A random intercept \(b_j\) is the deviation of group \(j\) from the population-level predictor. It is not the difference between two independently sampled groups. Under the stated model, if groups \(j\) and \(k\) have independent random intercepts with variance \(\sigma^2\), then \(b_j-b_k\mid\sigma\sim N(0,2\sigma^2)\). A central 95% range for the group-to-group contrast is therefore approximately \(\pm1.96\sqrt{2}\sigma\). Confusing these two quantities makes a prior appear more or less concentrated than intended.

The calibration can be used with other prior families, but the probability statement and its solution change. A half-normal or half-Student-\(t\) prior does not have the exponential survival function derived above; its scale parameter must be chosen from its own tail probability. The useful general practice is to state a threshold in meaningful units and solve the prior’s actual equation for the parameter that achieves the desired probability. The earlier material on exponential and penalized-complexity priors for group-level scales provides the prior forms; calibration supplies an application-specific choice of their scales.

Assumptions, Diagnostics, and Limitations

These calculations rely on the stated random-effect model and the calibration target. In particular, the normal conditional distribution is what justifies using \(1.96\sigma\) for a central 95% range. If the random-effect distribution differs, or if the relevant quantity is a contrast with a different variance, the multiplier must be changed accordingly. The response-scale translation also relies on the specified link and baseline covariate pattern. For a random slope, a predictor contribution depends on the covariate value, so a single scale may imply very different changes at different covariate values.

Before using a calibrated scale, check the arithmetic and the interpretation:

• Confirm that \(\lambda\) has reciprocal units and that \(\lambda U=-\log(\alpha)\).
• Check the sign and direction of the predictor shift when translating to a response value.
• For a nonlinear link, calculate the response at the stated baseline rather than treating predictor changes as percentage-point changes.
• Distinguish a group deviation from a group-to-group contrast, including the contrast’s standard deviation.
• State clearly that \(P(\sigma>U)=\alpha\) is a prior probability about the scale, not a probability statement about a parameter after seeing data.

A common mistake is to choose \(U\) from a plausible predictor shift and then claim that the same percentage-point change is plausible in every group. This fails when the link is nonlinear: a symmetric predictor shift produces asymmetric response changes, and the response changes depend on the baseline. The correction is to specify a baseline, transform the shifted predictor through the inverse link, and report the resulting response-scale values. Calibration also cannot make a poorly chosen scientific benchmark appropriate; its role is to express a defensible benchmark transparently in the prior.