Joint Priors for Random-Effect Scales and Correlations

Why Scales and Correlations Need Separate Attention

A random-effects covariance matrix describes both the variability of each random effect and how the effects vary together. These are different features. For example, in a random-intercept and random-slope model, the intercept scale describes variation in group baselines, while the slope scale describes variation in group-specific changes. Their correlation describes whether groups with higher baselines tend to have higher or lower slopes.

A prior that specifies plausible values for each scale does not, by itself, say what correlations are plausible. Conversely, a correlation prior does not determine how large the random effects tend to be. A complete prior specification must address their joint distribution, including whether the scales and correlations are independent a priori.

Definition: For \(q\) random effects, let \(\tau_k>0\) be the standard deviation of random effect \(k\), and let \(R\) be their \(q\times q\) correlation matrix. Write \(D=\operatorname{diag}(\tau_1,\ldots,\tau_q)\). The random-effects covariance matrix, conditional on these parameters, is \(\Sigma=D R D\). A joint prior specifies a probability distribution for \((\tau_1,\ldots,\tau_q,R)\); it therefore induces a prior distribution on \(\Sigma\).

In a GLMM, a typical group-level model is \(b_j\mid \tau,R\sim N_q(0,\Sigma)\), where \(b_j\) is the vector of random effects for group \(j\). This tutorial concerns the prior on \(\tau\) and \(R\), not the likelihood or the full posterior calculation. It assumes the random effects are conditionally mean-zero multivariate normal, the scales are positive, and \(R\) is a valid correlation matrix. The prior must also be supported on parameter values allowed by the model. In Bayesian analysis, these parameters are then updated through the likelihood; prior independence need not persist in the posterior.

What the Joint Specification Implies

The diagonal entries of \(\Sigma\) are the random-effect variances, and its off-diagonal entries are covariances. For indices \(k\) and \(l\), the scale-correlation representation gives

\Sigma_{kk}=\tau_k^2,\qquad \Sigma_{kl}=\tau_k\tau_l\rho_{kl}\quad(k\ne l),

where \(\rho_{kl}\) is the corresponding entry of \(R\). Thus, the prior on \(\Sigma\) is determined by the joint distribution of the scales and correlations, not just by their separate marginal distributions. Multiplying the scale parameters by correlations also means that the induced covariance distribution can be quite different from the correlation distribution itself.

Key distinction: Marginal priors \(p(\tau_k)\) and \(p(R)\) describe distributions considered one at a time. A joint prior \(p(\tau,R)\) also specifies dependence across scales and between scales and correlations. The common choice \(p(\tau,R)=p(\tau)p(R)\) asserts prior independence between the scale vector and the correlation matrix; it does not make the entries of \(\Sigma\) independent.

A short derivation explains the covariance formula. For fixed \(\tau\) and \(R\), write \(b=Dz\), with \(z\sim N_q(0,R)\). Then

\operatorname{Cov}(b\mid\tau,R) =\operatorname{Cov}(Dz\mid\tau,R) =D\operatorname{Cov}(z)D^\mathsf{T} =DRD,

because \(D\) is diagonal and symmetric. The \((k,l)\) entry is therefore \(\tau_k R_{kl}\tau_l\); for \(k=l\), \(R_{kk}=1\), giving \(\tau_k^2\). This calculation is conditional on the hyperparameters. The prior on \(\Sigma\) is the distribution obtained by drawing \(\tau\) and \(R\) from their joint prior and applying this mapping.

Worked Example: Independent Half-Normal Scales and a Uniform Correlation

Consider two random effects, an intercept and a slope. Suppose their standard deviations have independent half-normal priors with scale parameters \(0.5\) and \(0.2\), respectively, and their correlation \(\rho\) has a uniform prior on \((-1,1)\). Assume the scale vector and \(\rho\) are also independent. These choices are proper, the scales are positive, and every \(\rho\) in the prior support gives a valid two-effect correlation matrix.

For a half-normal prior with scale parameter \(s\), \(E(\tau^2)=s^2\). Hence the prior means of the two conditional variances are \(0.5^2=0.25\) and \(0.2^2=0.04\), in squared units of their respective random effects. The correlation prior has \(E(\rho)=0\) and \(E(\rho^2)=1/3\). The induced off-diagonal covariance is \(\Sigma_{01}=\tau_0\tau_1\rho\), so prior independence gives

E(\Sigma_{01}) =E(\tau_0)E(\tau_1)E(\rho)=0.

A mean covariance of zero does not mean that the covariance is usually zero: the prior puts probability zero on exactly \(\rho=0\). Its magnitude can be summarized here by the second moment. Since the scales are independent and \(E(\tau_0^2)=0.25\), \(E(\tau_1^2)=0.04\),

E(\Sigma_{01}^2) =E(\tau_0^2)E(\tau_1^2)E(\rho^2) =(0.25)(0.04)\left(\frac{1}{3}\right) =0.003333.

The prior standard deviation of this off-diagonal covariance is therefore \(\sqrt{0.003333}=0.05774\), in the product of the intercept and slope units. Also, \(P(|\rho|>0.8)=(1-0.8)=0.2\): one fifth of the prior probability allows a correlation magnitude greater than 0.8. Whether that is suitable depends on context; it cannot be decided by inspecting the scale priors alone.

Identical Marginals Can Give Different Covariance Priors

Specifying the same marginal distribution for every scale does not uniquely determine the joint scale distribution. This matters because the covariance between two random effects contains the product \(\tau_k\tau_l\). If large scales tend to occur together, the prior may put more mass on large absolute covariances than it would under independent scales, even when each scale has exactly the same marginal prior.

Worked Example: Same Scale Marginals, Different Dependence

For illustration, let each of two random-effect scales have the prior \(P(\tau=1)=P(\tau=3)=0.5\), and let \(\rho\sim\operatorname{Uniform}(-1,1)\) independently of both scales. Compare two joint scale priors. Under the first, \(\tau_0\) and \(\tau_1\) are independent. Under the second, they match exactly: with probability 0.5 both equal 1, and with probability 0.5 both equal 3. Each scale has the same marginal distribution in both specifications, and the correlation prior is unchanged.

Under independent scales, the products \(\tau_0\tau_1\) are \(1,3,3,9\), each with probability \(0.25\), so \(E(\tau_0\tau_1)=(1+3+3+9)/4=4\). Under matched scales, the product is 1 or 9 with equal probability, so \(E(\tau_0\tau_1)=(1+9)/2=5\). For a uniform correlation on \((-1,1)\), \(E|\rho|=0.5\). Consequently,

E(|\Sigma_{01}|) =E(\tau_0\tau_1)E(|\rho|) =\begin{cases} 4(0.5)=2, & \text{independent scales},\\ 5(0.5)=2.5, & \text{matched scales}. \end{cases}

The second joint prior therefore gives a larger prior mean absolute covariance, despite having the same marginal scale priors and the same correlation prior. This discrete example is a demonstration of the principle, not a default recommendation for continuous scale priors.

Prior independence is a substantive modeling choice rather than an automatic consequence of listing separate marginal priors. In a continuous model, one could specify the scales jointly through a shared latent quantity or another dependence structure while preserving their individual marginal distributions. Such a choice should have a defensible interpretation and be assessed through the implied covariance distribution. If the intended assumption is independence, state the product prior explicitly, for example \(p(\tau,R)=\{\prod_k p(\tau_k)\}p(R)\), when the scales are also mutually independent.

Checking an Implied Covariance Draw

For two random effects, the correlation matrix must have the form \(\begin{pmatrix}1&\rho\\ \rho&1\end{pmatrix}\) with \(|\rho|<1\). Combined with positive scales, this produces a positive-definite covariance matrix. A prior draw can be checked directly by computing its diagonal, off-diagonal, and determinant.

Worked Example: From a Scale and Correlation Draw to a Covariance Matrix

Suppose a hypothetical prior draw for a random intercept and a random slope is \(\tau_0=0.4\), \(\tau_1=0.3\), and \(\rho=-0.6\). These are parameter values, not observations. The correlation is valid because \(|-0.6|<1\). Applying the scale-correlation formula gives

\Sigma= \begin{pmatrix} 0.4^2 & (0.4)(0.3)(-0.6)\\ (0.4)(0.3)(-0.6) & 0.3^2 \end{pmatrix} = \begin{pmatrix} 0.16 & -0.072\\ -0.072 & 0.09 \end{pmatrix}.

The determinant is \((0.16)(0.09)-(-0.072)^2=0.0144-0.005184=0.009216>0\); both diagonal entries are positive. Thus the matrix is positive definite. The negative covariance is consistent with the negative correlation: groups with higher intercept effects tend, under this draw, to have lower slope effects. The covariance’s units are the product of the intercept-effect and slope-effect units; the correlation is dimensionless.

Diagnostics, Alternatives, and Limitations

A reliable check examines joint prior draws, not only separate histograms of the scales and correlations. For each draw, compute the implied covariance entries and inspect whether their magnitudes, signs, and combinations are plausible. In a GLMM, continue the prior-predictive simulation by drawing random effects and outcomes from the specified model. This can reveal, for example, that a seemingly moderate scale prior combined with a correlation prior and the predictor range produces implausibly large changes across groups.

Check marginal scales in context. A scale’s units depend on the associated random effect. For a random slope, those units also depend on the predictor’s units and coding. A scale prior that seems small numerically may not be small for the predictor range in use.
Inspect the correlation prior directly. Check its concentration near zero and near extreme correlations, rather than inferring correlation behavior from scale summaries.
Inspect joint draws. Look for dependence between scales or between scales and correlations that was intended, or that arises from the chosen joint construction.
Check admissible support. For more than two random effects, \(R\) must be a valid correlation matrix, not merely a collection of individually allowable pairwise correlations.

A scale-correlation prior is often attractive because scales and correlations have direct interpretations. An alternative is to assign a prior directly to a covariance matrix. That approach can be convenient, but its implied marginal scale and correlation priors may be less transparent or may be coupled in ways that are not intended. Either approach should be evaluated by the distributions it induces on quantities of interest. In particular, independent priors on covariance entries are generally not a safe shortcut: arbitrary combinations can fail to produce a valid covariance matrix.

This parameterization also has limits. It does not make prior selection automatic: each scale prior, the correlation prior, and any dependence assumptions still require justification. For multiple random effects, the valid-matrix constraint makes correlation specification more involved than choosing each pairwise correlation separately. Finally, a prior that looks reasonable for the covariance parameters may still generate implausible random effects or outcomes, so prior-predictive checks remain important.

Common mistake: “I specified a prior for every random-effect standard deviation, so the covariance prior is complete.” This omits the correlation prior and any dependence assumptions. The correction is to specify a joint prior on the scale vector and correlation matrix, state whether they are independent, and examine the covariance matrices that this joint prior implies.

Check Your Understanding

Answer each question, then compare with the solutions below.

For two random effects with scales \(0.5\) and \(0.2\), and correlation \(0.4\), what is their covariance?
Does a prior mean correlation of zero imply that the covariance is zero in every prior draw?
Two priors have identical marginal scale distributions and the same correlation prior. Must they induce the same covariance prior?
For three or more random effects, why is it not sufficient to choose any allowable value separately for each pairwise correlation?

Solutions. (1) The covariance is \((0.5)(0.2)(0.4)=0.04\), in the product of the random effects’ units. (2) No. A mean correlation of zero can result from positive and negative draws; it does not require any draw to equal zero. (3) No. Different dependence between scales, or between scales and correlations, can change the joint distribution of covariance entries while preserving the stated marginals. (4) The full correlation matrix must be valid; pairwise values chosen separately can combine into a matrix that is not positive definite.