Wishart Priors and Covariance-Scale Dependence

Why Matrix Priors Need Careful Interpretation

A random-effects covariance matrix must be positive definite, so a prior on its entries cannot be chosen as if each entry varied independently over the real line. Wishart and inverse-Wishart distributions provide distributions supported on positive-definite matrices. Their convenience comes with an important cost: a prior that looks simple in matrix form can imply non-obvious marginal distributions for standard deviations and correlations. In particular, the scales and correlations need not be prior-independent.

For a GLMM with a \(q\)-dimensional random effect, write its covariance matrix as \(\Sigma\). Its diagonal entries are random-effect variances, and its off-diagonal entries describe covariances. The corresponding correlations are obtained by dividing each covariance by the product of the two associated standard deviations. The introductory material on constructing covariance matrices from scales and correlations establishes this decomposition; here the focus is how matrix-distribution priors behave under it.

Wishart and Inverse-Wishart Conventions

Both distributions require a positive-definite matrix parameter. The following parameterizations fix what “scale matrix” means, since other references sometimes use different parameter conventions. Let \(|A|\) denote the determinant of a matrix \(A\), and let \(\operatorname{tr}(A)\) denote its trace. The multivariate gamma function is \(\Gamma_p(a)=\pi^{p(p-1)/4}\prod_{j=1}^{p}\Gamma(a+(1-j)/2)\).

For \(S\sim W_p(\nu,V)\), the mean exists and is \(E(S)=\nu V\). When \(\nu\) is a positive integer, this distribution can be represented as the sum of \(\nu\) outer products of independent \(N_p(0,V)\) vectors; the density also defines it for real \(\nu>p-1\). A Wishart prior is often placed on a precision matrix \(\Omega=\Sigma^{-1}\), rather than directly on a covariance matrix.

If \(\Sigma\sim IW_p(\nu,\Psi)\), then \(\Sigma^{-1}\sim W_p(\nu,\Psi^{-1})\). The inverse-Wishart distribution is proper when \(\nu>p-1\), but its mean exists only when \(\nu>p+1\), in which case \(E(\Sigma)=\Psi/(\nu-p-1)\). These conditions are distinct: propriety does not guarantee a finite covariance mean. The entries of \(\Psi\) have the units of covariance entries, but \(\Psi\) is not generally the prior mean.

How the Inverse-Wishart Couples Scales and Correlations

The coupling can be seen directly in dimension two. Write the covariance matrix in terms of variances \(v_1,v_2>0\) and correlation \(-1<\rho<1\):

Positive definiteness is equivalent here to \(v_1>0\), \(v_2>0\), and \(|\rho|<1\). The absolute Jacobian determinant for transforming from \((v_1,v_2,\rho)\) to the three distinct entries of \(\Sigma\) is \(\sqrt{v_1v_2}\). Substitution into the inverse-Wishart density, including this Jacobian, gives the joint density on variances and correlation, up to a constant independent of those variables:

This expression is not generally a product of a function of \(v_1,v_2\) and a function of \(\rho\). Even when \(\Psi\) is diagonal, the exponent contains the scale-dependent quantity \(\Psi_{11}/v_1+\Psi_{22}/v_2\), divided by \(1-\rho^2\). Consequently, the conditional distribution of \(\rho\) changes with the variances. The coupling is a feature of the induced prior, not a dependence introduced by observing data.

Wishart Priors on Precision and Their Covariance Implications

A Wishart prior is often used for a precision matrix. This choice should be interpreted on the covariance scale before it is adopted for random effects. If \(\Omega\sim W_p(\nu,V)\) and \(\Sigma=\Omega^{-1}\), then the induced covariance prior is \(IW_p(\nu,V^{-1})\). When its mean exists,

Thus \(V\) is the Wishart scale for precision, not a covariance target. Inverting the mean of \(\Omega\) is not the same operation as taking the mean of \(\Omega^{-1}\).

Assumptions, Diagnostics, and Alternatives

These distributions are mathematically defined on positive-definite matrices, but mathematical validity alone does not establish that a prior is appropriate for a GLMM. The model must identify which random effects the matrix describes, and their units and scaling must be clear. For random slopes, changing the units or centering of a predictor changes the interpretation of the covariance entries and correlations. A prior specified before such a transformation need not express the same substantive beliefs afterward.

Useful prior checks include drawing covariance matrices from the prior and examining the implied standard deviations, correlations, and resulting random-effect values on the model’s linear-predictor scale. Check the probability of near-zero or very large scales, the frequency of correlations near \(-1\) or \(1\), and whether plausible combinations of scales and correlations occur together. If the inverse-Wishart mean does not exist, do not summarize its covariance prior using a finite mean; inspect quantiles or prior draws instead.

Common warning signs include a scale parameter interpreted as a mean when it is not, a Wishart prior placed on covariance when precision was intended, and apparently reasonable marginal scale summaries paired with undesirable joint behavior. A prior-predictive check should preserve the joint draws: independently recombining sampled scales and correlations would erase the dependence being assessed.

Separate scale and correlation priors, including the LKJ family for correlation matrices, can make substantive calibration more transparent. That flexibility does not automatically make a prior suitable: the selected scale distributions, correlation concentration, predictor coding, and joint prior predictions still require assessment. A matrix prior may remain useful when its dependence structure is intended or when its algebraic convenience is valuable, but convenience is not a substitute for examining its induced behavior.