Why Calibrate the Implied Variability?
In a multilevel GLMM, the prior on a hyperparameter is not itself the prior distribution of the group-level standard deviation. For example, a hyperprior may place probability on a scale parameter \(A\), while the standard deviation \(\tau\) is drawn conditionally on \(A\). The distribution of \(\tau\) across repeated prior draws combines both sources of uncertainty. Calibration therefore asks what values of \(\tau\) the full hierarchy makes plausible, not merely whether the hyperprior on \(A\) looks reasonable by itself.
This matters because \(\tau\) controls the amount of group-to-group variation on the scale of the model's linear predictor. A prior that looks modest for \(A\) can still imply a substantial chance of very large \(\tau\), or can concentrate too much mass near zero. The appropriate check is to draw repeatedly from the entire prior hierarchy and examine the induced distribution of the standard deviation in context.
For a scalar random-effect component, write \(b_j\) for the effect in group \(j\), and consider
Here \(\operatorname{HalfNormal}(A)\) is the distribution of the absolute value of a \(N(0,A^2)\) random variable. Thus \(A>0\) is a hyperparameter controlling the conditional scale of \(\tau\); it is not the group-level standard deviation itself. The model assumes the stated conditional distributions, with the hyperprior for \(A\) specified separately. In a GLMM, the units of \(\tau\) are the units of the random effect on the linear-predictor scale. A random-slope standard deviation, for example, is measured per unit of its predictor.
From a Hyperprior to a Marginal Distribution
The law of total probability gives the marginal tail probability of \(\tau\):
Under the half-normal conditional prior, \(P(\tau>t\mid A=a)=2\Phi(-t/a)\) for \(t\geq0\), where \(\Phi\) is the standard normal cumulative distribution function. Consequently,
If the hyperprior is a discrete mixture, this integral is a weighted sum. If it is continuous, numerical integration or repeated prior simulation can approximate it. In either case, the target is the distribution of \(\tau\) after \(A\) has also been drawn, rather than a conditional distribution evaluated at one convenient value of \(A\).
Worked Example: Calculating Marginal Tail Probabilities
Suppose \(A\) has a two-point hyperprior: \(P(A=0.25)=0.5\) and \(P(A=0.75)=0.5\). Conditional on \(A\), let \(\tau\sim\operatorname{HalfNormal}(A)\). Calculate the prior probabilities that \(\tau\) exceeds 0.5 and 1.0.
For \(t=0.5\), average the two conditional tail probabilities:
The first conditional probability is \(2\Phi(-2)\), approximately 0.04550, and the second is \(2\Phi(-2/3)\), approximately 0.5050. The reported marginal probability is obtained using the unrounded tail probabilities and rounding only the final result.
For \(t=1.0\), the calculation is
Thus, under this hypothetical prior, about 27.52% of prior draws of the group-level standard deviation exceed 0.5, and about 9.124% exceed 1.0, in the units of the random effect. These are prior probabilities, not posterior conclusions about any data.
A Calibration Calculation and Its Interpretation
A useful calibration statement is often an upper-tail requirement: for example, choose the hyperprior so that the prior probability of a group-level standard deviation exceeding a contextually large value is acceptably small. The threshold must be meaningful in the model's units, and the probability should be evaluated under the whole hierarchy. A scale transformation can make comparisons between candidate hyperpriors particularly transparent.
Worked Example: Rescaling a Hyperprior to Meet a Tail Target
Use the two-point distribution above as a baseline \(A_0\), and define a candidate hyperprior \(A=cA_0\), where \(c>0\). Under the same half-normal conditional specification, \(\tau\) also scales by \(c\). Suppose the design goal is to make \(P(\tau>1.0)\) approximately 0.275. Find a simple value of \(c\) that meets this goal.
Let \(\tau_0\) denote a draw under the baseline hyperprior. The scaled hierarchy gives \(\tau=c\tau_0\), so
Taking \(c=2\) makes the last threshold \(1.0/2=0.5\). The first worked calculation therefore gives
So doubling both possible values of \(A\) meets the approximate 0.275 tail target at threshold 1.0. This is a calibration illustration, not a universal recommendation: the threshold and acceptable tail probability must come from the scientific scale of the random effect. Other features, such as the lower tail near zero, should also be checked.
A concise derivation explains why the scaling works. If \(Z\sim N(0,1)\), then a conditional half-normal draw can be represented as \(\tau=A|Z|\). Replacing \(A\) with \(cA\) gives \(cA|Z|=c\tau\). This representation also provides a direct way to generate prior draws: draw \(A\) from its hyperprior, draw an independent standard normal \(Z\), and set \(\tau=A|Z|\).
Repeated Prior Draws as a Calibration Tool
For a continuous or complicated hyperprior, repeated simulation often gives the clearest view of the marginal distribution. Each repetition must draw every layer in order: first the hyperparameters, then the group-level scale conditional on those hyperparameters. Drawing \(\tau\) repeatedly at one fixed value of \(A\) only checks a conditional prior and does not assess the hierarchical prior as specified.
Write down the conditional prior for \(\tau\), the hyperprior for its parameters, and the model scale on which \(\tau\) operates.
For each repetition, draw the hyperparameters and then draw \(\tau\) conditionally. Retain the resulting \(\tau\), not just the hyperparameter values.
Report quantiles and exceedance proportions at thresholds that have a substantive interpretation in the model's units.
Consider whether the implied variability is too large, too concentrated near zero, or otherwise inconsistent with plausible group differences; revise and repeat if needed.
Worked Example: Reading a Prior Simulation Summary
Suppose 12 hypothetical draws of \(\tau\), produced by drawing through the full hierarchy, are already sorted as follows:
The empirical proportion exceeding 0.5 is \(4/12=0.3333\), since the four values 0.53, 0.61, 0.78, and 1.04 exceed that threshold. The empirical median for these 12 draws is the average of the sixth and seventh ordered values:
These summaries say that one third of this small set of prior draws exceed 0.5 and that its median is 0.375. They do not establish that the exact prior tail probability is 0.3333: with only 12 draws, the empirical proportion is noisy. For a final calibration, use many more draws and check that the reported summaries are stable across larger simulation runs. The values and summaries here are hypothetical.
Diagnostics, Alternatives, and Limitations
A prior-draw summary should be read as a diagnostic, not as a mechanical pass-or-fail test. Extremely large \(\tau\) draws may indicate that the hyperprior gives too much weight to very large group differences. Conversely, almost all draws close to zero may encode strong prior concentration on near-equal groups. Whether either pattern is problematic depends on the model scale and the substantive application. For a GLMM, one should also consider what such random-effect values imply for the linear predictor and, where relevant, for response probabilities; calibration of \(\tau\) alone does not guarantee sensible predictions.
Check that each standard deviation is expressed in the right units. For example, a random-intercept standard deviation is on the linear-predictor scale, while a random-slope standard deviation is tied to the unit used for its predictor. Rescaling a predictor changes the numerical scale of its random slope, so a threshold appropriate before rescaling may be inappropriate afterward. With several random-effect scales, inspect the marginal distribution of each scale and consider their joint draws as well; plausible individual marginals do not ensure that every combination is plausible.
The main alternatives differ in what they specify directly. A fixed scale sets \(\tau\) at one value and removes hyperprior uncertainty. A direct prior on \(\tau\) specifies its marginal distribution without an additional layer. A hierarchical prior lets the scale vary through hyperparameters, which can be useful but makes marginal calibration essential. These approaches are not interchangeable by appearance: two priors with similar conditional scales can imply very different marginal tails.
Prior simulation also has limitations. Its summaries are Monte Carlo estimates, and rare-tail probabilities need enough draws to estimate reliably. A simulation cannot decide which variability is scientifically plausible; that judgment requires the meaning and units of the random effect. Finally, prior calibration concerns the prior before observing outcomes. It does not establish that the likelihood, model structure, or posterior behavior will be adequate.
Check Your Understanding
Use the definitions and calculations above. Solutions are included after the questions.
- If \(A=0.5\) is fixed and \(\tau\mid A\sim\operatorname{HalfNormal}(A)\), express \(P(\tau>0.5)\) using \(\Phi\), and give its approximate value.
- For the two-point hyperprior in the first worked example, explain why \(P(\tau>1.0)\) is not the conditional probability evaluated at the average value of \(A\).
- If every value of \(A\) in a hierarchy is multiplied by 3, what happens to \(\tau\) under a half-normal conditional prior?
- A prior simulation produces 600 exceedances of a threshold among 2,000 complete hierarchy draws. What is the empirical exceedance proportion, and what does it estimate?
Solutions. (1) \(P(\tau>0.5)=2\Phi(-0.5/0.5)=2\Phi(-1)\approx0.3173\). (2) The marginal probability is the weighted average \(0.5\{2\Phi(-1.0/0.25)\}+0.5\{2\Phi(-1.0/0.75)\}\approx0.09124\); evaluating at the average \(A\) does not generally equal that average because the tail probability is a nonlinear function of \(A\). (3) The resulting \(\tau\) is multiplied by 3 in every corresponding draw, since \(\tau=A|Z|\). (4) The empirical proportion is \(600/2000=0.3000\). It estimates the marginal prior probability that \(\tau\) exceeds the threshold under the specified full hierarchy.