Why Several Random-Effect Scales Need Joint Attention
A GLMM can contain more than one random-effect component: for example, a random intercept and a random slope, or random effects for two distinct grouping factors. Each component has its own scale, but the prior choice is not merely a collection of separate decisions. The scales jointly determine how much group-to-group variation the model can express, and a prior that treats them independently has different implications from one that links them through a shared scale.
The focus here is the prior distribution on multiple standard deviations and the dependence those priors imply. This is distinct from the correlation among random-effect coefficients within a multivariate random effect. A prior can make two standard deviations dependent even when the random-effect correlation matrix is fixed to the identity; conversely, a correlation prior does not by itself specify how the marginal scales depend on one another.
Scales, Variances, and Covariance Structure
Suppose a grouping unit has a vector of \(q\) random effects \(b_j=(b_{j1},\ldots,b_{jq})^\mathsf{T}\). A common model is \(b_j\mid \Sigma\sim N_q(0,\Sigma)\), where \(\Sigma\) is the covariance matrix. Write the standard deviation of component \(k\) as \(\tau_k\), and collect the scales in \(D=\operatorname{diag}(\tau_1,\ldots,\tau_q)\). When the scales are positive, write \(\Sigma=D\Omega D\), where \(\Omega\) is a correlation matrix. Thus \(\Sigma_{kk}=\tau_k^2\), and \(\Sigma_{k\ell}=\tau_k\tau_\ell\Omega_{k\ell}\).
Positive scales are required for each component's correlation to be defined in the usual way. A zero scale is allowed as a boundary case for the covariance matrix: \(D\Omega D\) remains positive semidefinite when \(D\) has zero diagonal entries and \(\Omega\) is a valid correlation matrix. But a component with zero variance is degenerate, so correlations involving it are not identified as correlations of nondegenerate random effects.
For separate random-effect blocks, such as effects associated with different grouping factors, each block can have its own scale vector and covariance structure. The joint prior question remains: are the scales across blocks or components independent, or does their prior specification link them? The appropriate answer depends on the scientific scale of plausible heterogeneity and on the model's intended flexibility.
Independent Priors: Clear Marginals, No Prior Link
One direct choice assigns each scale a familiar distribution on the nonnegative line, such as a half-normal, half-Student-\(t\), or exponential distribution. The distribution and its scale parameter should be calibrated in the units of the corresponding random effect. Earlier material on half-Student-\(t\) priors, exponential and penalized-complexity priors, and prior scale calibration gives the single-component constructions; here the key point is what happens when several are assigned together.
If the priors are independent, then prior probabilities for joint events factor. For instance, if the scale priors for two components are independent, \(P(\tau_1\leq c_1,\tau_2\leq c_2)=P(\tau_1\leq c_1)P(\tau_2\leq c_2)\). Independence does not mean the posterior scales will be independent: the likelihood can couple them through the data and model. It means only that before observing the data, the prior contains no dependence between those scales.
Worked Example: Independent Exponential Priors
Consider a hypothetical logistic GLMM with a random intercept and a random slope for a predictor measured in tens of units. Suppose the intercept is on the log-odds scale and the slope is the change in log odds per ten predictor units. Assign independent priors \(\tau_0\sim\operatorname{Exponential}(1)\) and \(\tau_1\sim\operatorname{Exponential}(2)\), where the numbers are rates per corresponding scale unit. For an exponential distribution with rate \(\lambda\), \(P(\tau>c)=e^{-\lambda c}\), so \(P(\tau\leq c)=1-e^{-\lambda c}\).
For the prior probability that both scales are at most \(0.5\) in their respective units, independence gives \(P(\tau_0\leq 0.5,\tau_1\leq 0.5)=(1-e^{-1(0.5)})(1-e^{-2(0.5)})\). The two factors are \(1-e^{-0.5}\approx0.3935\) and \(1-e^{-1}\approx0.6321\); their product is approximately \(0.2488\).
Thus about 24.9% of prior draws place both scales at or below their stated thresholds. This calculation is a property of the specified prior, not a probability about the unknown scales after seeing data. The thresholds and rates are only a hypothetical specification: their adequacy must be judged against plausible log-odds variation in the application, including the slope's predictor units.
Joint Alternatives: Linking Scales Deliberately
A dependent prior can express a prior belief that several components tend to be simultaneously small or simultaneously large. One simple construction uses a positive shared multiplier \(A\) and component-specific multipliers \(\lambda_k\): \(\tau_k=A\lambda_k\). If \(A\) is random and the \(\lambda_k\)'s are fixed positive constants, all scales move together with \(A\). More flexible constructions give the \(\lambda_k\)'s their own distributions. Such models are hierarchical because a common latent scale links the component scales; they require calibration of both the shared and component-specific parts.
The existence of a shared multiplier does not automatically make every pair positively dependent under every possible construction. The simple case below makes the assumptions explicit. Let \(A>0\) be random, let \(\lambda_1,\lambda_2>0\) be fixed, and suppose \(E(A^4)<\infty\). Then the covariance of the squared scales is well-defined and follows directly from their definitions.
The fourth-moment condition ensures the terms in this covariance calculation are finite. If \(A^2\) is nonconstant, its variance is positive, so the squared scales have positive covariance. This describes dependence in the prior on the variances; it is not a statement that the random-effect coefficients themselves have positive correlation. If a construction instead allows \(A=0\), it can put scales on the boundary, where correlations for zero-variance components are not identified.
Worked Example: Dependence From a Shared Scale
For a hypothetical two-component model, let \(\lambda_1=0.4\) and \(\lambda_2=2\), in the respective random-effect units. Let \(A\) equal \(0.5\) or \(1.5\), each with probability \(1/2\). This finite distribution has \(E(A^4)<\infty\), as required for the covariance calculation. The squared values of \(A\) are \(0.25\) and \(2.25\), so their mean is \(1.25\) and their variance is \(\{(0.25-1.25)^2+(2.25-1.25)^2\}/2=1\).
The component variances are \(\tau_1^2=0.16A^2\) and \(\tau_2^2=4A^2\). Therefore \(\operatorname{Cov}(\tau_1^2,\tau_2^2)=0.16(4)(1)=0.64\), in the product of the two variance units. Directly, the two possible variance pairs are \((0.04,1)\) and \((0.36,9)\), each with probability \(1/2\). Their means are \(0.20\) and \(5\); the mean product is \((0.04+3.24)/2=1.64\). Subtracting the product of means, \(1.64-(0.20)(5)=0.64\), confirms the result.
This construction forces the two scales to rise and fall together; it is intentionally simple rather than a universal recommendation. In an applied model, a shared-scale prior is sensible only if such joint behavior is scientifically defensible and its marginal implications are acceptable.
Calibration and Prior Predictive Checks
For several components, checking each marginal prior separately is not enough. A joint prior can assign substantial probability to combinations that are individually plausible but jointly extreme, or it can make large values occur together more often than intended. Prior predictive checks examine the consequences of draws from the full prior, including the fixed effects, random effects, and observation model, before conditioning on observed outcomes.
For a GLMM, draw parameter values from the proposed joint prior, draw random effects from their specified distribution, and then simulate outcomes using the chosen response distribution and link. Examine both the scales and quantities on the response scale: for example, predicted probabilities, counts, or rates across groups. This reveals whether the joint scale prior implies implausibly extreme group variation, near-identical groups, or an unreasonable pattern of simultaneous variation across components. The link function matters: a seemingly moderate spread on the linear-predictor scale can imply probabilities near zero or one in a logistic model.
Worked Example: Calibrating Two Half-Normal Priors
Suppose, hypothetically, two independent random-effect scales each receive a half-normal prior with scale parameter \(s=0.5\). For a half-normal variable \(\tau=|Z|\), where \(Z\sim N(0,s^2)\), \(P(\tau>c)=2\{1-\Phi(c/s)\}\). At the threshold \(c=1\), \(c/s=2\), and \(\Phi(2)\approx0.97725\). Thus each prior has \(P(\tau>1)\approx2(1-0.97725)=0.0455\).
Under independence, the probability that at least one of the two scales exceeds 1 is \(1-P(\tau_1\leq1,\tau_2\leq1)=1-(1-0.0455)^2\approx0.0889\). So about 8.89% of prior draws have at least one scale above 1, in its own component units. This marginal-and-joint calculation helps assess the specified prior, but it does not establish that \(s=0.5\) is appropriate. The component units and plausible effect sizes must be checked, and simulated outcomes should also be inspected under the GLMM response model.
Comparison, Diagnostics, and Limitations
| Prior structure | What it implies | Useful check |
|---|---|---|
| Independent half-distributions | Each marginal can be calibrated separately; there is no prior dependence between scales. | Check both marginal tails and probabilities of joint events. |
| Shared-scale or other dependent prior | The joint prior links scales; the particular construction determines the dependence pattern. | Plot joint prior draws and inspect simultaneous large or small values. |
| Prior on a full covariance matrix | Scales and correlations may have joint implications depending on parameterization and prior choice. | Inspect implied marginal scales, correlations, and covariance draws. |
These approaches answer related but different questions. Independent priors are transparent and straightforward to calibrate marginally, but they can be a poor representation when prior knowledge links components. A shared-scale construction can express such a link compactly, but it may impose strong co-movement and needs careful calibration. A prior directly on a covariance matrix is another route, but its implied scale and correlation behavior must be understood rather than inferred from its name alone. No choice is automatically robust to weak data or to a poorly scaled predictor.
Common warning signs include prior draws dominated by huge group effects, response predictions concentrated at implausible extremes, and a joint prior that almost never allows one component to be large without another. Also investigate whether a scale prior puts meaningful probability near zero: that can be appropriate, but a continuous prior does not assign positive probability to an exact zero. If the scientific question requires a point mass at zero, that is a different prior specification. Posterior behavior can be sensitive to these choices, especially when the data provide limited information about one or more components.
A Practical Specification Sequence
Identify the random-effect term, its units, and the meaning of its standard deviation. Rescale predictors when necessary so slope scales have interpretable units.
Select a proper distribution for each nonnegative scale and calibrate its central range and upper tail to plausible heterogeneity.
Use independent priors unless a dependence structure is intended; if using a shared or other joint construction, state what relationships it induces.
Summarize marginal and joint prior draws, then simulate outcomes from the full GLMM to detect implausible group variation.
Check Your Understanding
Answer each question, then compare with the brief solutions below.
- If two scale priors are independent and their marginal probabilities of exceeding specified thresholds are \(0.10\) and \(0.20\), what is the probability both exceed their thresholds?
- In the shared-scale construction \(\tau_k=A\lambda_k\) with fixed positive \(\lambda_k\), what condition makes the covariance calculation for squared scales well-defined?
- Does positive prior covariance between two squared scales imply positive correlation between the corresponding random-effect coefficients?
- Why can separately plausible marginal priors still produce an unsatisfactory joint prior?
Solutions. (1) Independence gives \(0.10(0.20)=0.020\), or 2%. (2) A sufficient condition is \(E(A^4)<\infty\), so the fourth-moment terms in the covariance are finite. (3) No. It describes dependence between prior variance scales, not correlation between random-effect coefficients. (4) The marginals do not determine whether joint combinations are plausible; for example, they do not show how often both scales are large together. Inspect joint scale draws and prior predictive outcomes.