Why the correlation prior needs its own specification
In a random-effects model with multiple coefficients, the correlations among those coefficients are unknown parameters. A prior for their marginal scales does not by itself determine a prior for their correlation matrix. The LKJ family provides a way to specify the overall shape of a prior on a valid correlation matrix, using one positive parameter, \(\eta\). Its central feature is simple: the density is a power of the matrix determinant.
The shape parameter is not a correlation, nor does it specify a separate prior for every pair of effects. It controls how the prior weights entire matrices. Interpreting it well therefore requires attention to the matrix dimension, the positive-definiteness constraint, and the difference between a density and a probability.
The domain restriction matters: the density is supported only on positive-definite correlation matrices. A symmetric matrix with ones on the diagonal is not automatically a valid correlation matrix. The LKJ density is invariant to relabeling the variables, so it does not prefer one pair of variables over another before other model information is considered.
What the shape parameter does
A correlation matrix has determinant between 0 and 1. The identity matrix has determinant 1, while matrices approaching singularity have determinants approaching 0. Consequently, when \(\eta>1\), the factor \(|R|^{\eta-1}\) downweights matrices with small determinant relative to matrices with larger determinant. This generally concentrates prior weight toward less extreme correlation configurations, including configurations nearer the identity. When \(0<\eta<1\), the exponent is negative, so lower-determinant matrices receive greater density; the density can grow without bound as the positive-definite boundary is approached. The prior remains proper for every \(\eta>0\): a density that diverges at a boundary need not assign positive probability to that boundary.
At \(\eta=1\), the determinant factor is 1. The density is constant over the space of valid correlation matrices, with respect to the specified measure. This is a uniform density on that constrained space, not a claim that every entry is independently uniform. In two dimensions, the space is just the interval from -1 to 1, so the marginal correlation is uniform. In higher dimensions, the constraints couple entries, and a constant matrix-space density does not make all entries independent or necessarily uniform marginally.
Worked Example: Comparing LKJ Priors for One Correlation
Suppose there are two random effects, so \(R\) contains one unknown correlation \(\rho\). Then \(R=\begin{pmatrix}1&\rho\\ \rho&1\end{pmatrix}\), and positive definiteness requires \(-1<\rho<1\). Its determinant is \(1-\rho^2\), so the LKJ density for \(\rho\) is proportional to \((1-\rho^2)^{\eta-1}\).
For this one-dimensional case, the normalized density is $$ p(\rho\mid\eta)=\frac{\Gamma(\eta+\tfrac12)}{\sqrt{\pi}\,\Gamma(\eta)} (1-\rho^2)^{\eta-1},\qquad -1<\rho<1. $$
For \(\eta=1\), the constant is \(1/2\), so the density is \(0.5\) throughout the interval. Thus \(P(|\rho|<0.5)=0.5\).
For \(\eta=2\), the constant is \(3/4\), giving \(p(\rho\mid 2)=0.75(1-\rho^2)\). The probability of being within 0.5 of zero is $$ P(|\rho|<0.5\mid\eta=2)=\int_{-0.5}^{0.5}0.75(1-\rho^2)\,d\rho =0.75\left(1-\frac{1}{12}\right)=\frac{11}{16}=0.6875. $$ This prior places more probability near zero than the uniform case.
For \(\eta=0.5\), the density is \(1/(\pi\sqrt{1-\rho^2})\). Its density increases toward both endpoints, but the interval probability is still finite: $$ P(|\rho|<0.5\mid\eta=0.5)=\frac{2}{\pi}\arcsin(0.5)=\frac{1}{3}. $$ The divergence at \(\rho=\pm1\) does not create point masses there; those endpoints are excluded from the positive-definite parameter space and have probability zero.
Positive definiteness constrains correlations jointly
With three variables, write the pairwise correlations as \(a\), \(b\), and \(c\): $$ R=\begin{pmatrix} 1&a&c\\ a&1&b\\ c&b&1 \end{pmatrix}. $$ For positive definiteness, each pairwise correlation must lie between -1 and 1, and the determinant must be positive. Expanding the determinant gives $$ |R|=1+2abc-a^2-b^2-c^2 =(1-a^2)(1-b^2)-(c-ab)^2. $$ For fixed \(a\) and \(b\) with \(|a|<1\) and \(|b|<1\), the determinant is positive exactly when $$ ab-\sqrt{(1-a^2)(1-b^2)}<c<ab+\sqrt{(1-a^2)(1-b^2)}. $$ These bounds are strict: at either endpoint, the determinant is zero, so the matrix is singular rather than positive definite. The endpoints can be included when describing the closure of the allowed region or as integration limits; they have measure zero.
This constraint is one reason to use a prior defined on the whole matrix rather than assigning unrelated priors to each pairwise correlation. Independently drawn pairwise values can produce an invalid matrix.
Worked Example: Checking a Three-Effect Correlation Matrix
Consider hypothetical pairwise correlations \(a=0.4\) and \(b=-0.2\). To find the allowed values of \(c\), calculate $$ ab=-0.08,\qquad \sqrt{(1-a^2)(1-b^2)} =\sqrt{(0.84)(0.96)} =\sqrt{0.8064}\approx0.8980. $$
Therefore \(c\) must satisfy $$ -0.9780<c<0.8180. $$ A proposed value \(c=0.30\) is within this strict interval. Directly checking the determinant gives $$ |R|=0.8064-(0.30-(-0.08))^2 =0.8064-(0.38)^2 =0.6620>0. $$ Together with \(|a|<1\) and \(|b|<1\), this confirms positive definiteness. By contrast, choosing \(c=0.82\) violates the upper bound and gives a nonpositive determinant, so that set of three correlations cannot be a positive-definite correlation matrix.
Choosing and checking \(\eta\)
Choosing \(\eta\) is a substantive prior decision. A value above 1 expresses preference for matrices with larger determinants; a value below 1 gives greater density to matrices near the boundary; and 1 gives constant density over valid matrices. These descriptions do not mean that \(\eta\) fixes each pairwise correlation at a particular value. Even with a symmetric LKJ prior, correlations can be positive or negative, and their joint distribution respects the matrix constraints.
Dimension also matters. The same \(\eta\) defines a density proportional to the same determinant power for any \(q\), but the valid matrix space changes with \(q\). Therefore, a statement such as “\(\eta=2\) concentrates correlations near zero” should be checked for the actual number of random effects and the prior summaries relevant to the application. A bivariate calculation does not fully characterize a higher-dimensional prior.
A practical prior check is to draw correlation matrices from the proposed LKJ prior before fitting the model. Inspect the distribution of each pairwise correlation, the frequency of large absolute correlations, and how often sampled matrices have determinants close to zero. These checks assess the induced prior behavior in the intended dimension. They do not establish that the prior is scientifically appropriate; that requires comparison with plausible group-level relationships.
Count the random-effect coefficients whose correlations are being modeled. The prior is on a matrix of that specific dimension.
Use \(\eta=1\) for constant density over valid correlation matrices, values greater than 1 to favor larger determinants, or values between 0 and 1 to favor lower-determinant regions. Relate the choice to plausible dependence, not just a default convention.
Simulate prior matrices in the chosen dimension and examine pairwise correlations and near-singular draws. Revise the choice if those draws conflict with the intended prior information.
Common mistakes, alternatives, and limitations
- Reading \(\eta\) as a target correlation. It is a shape parameter in a matrix density, not the prior mean of every pairwise correlation. Symmetry implies no preference for positive versus negative signs, but the matrix entries are dependent.
- Calling the \(\eta=1\) prior independent uniform correlations. It is constant over valid matrices. The allowed values of one correlation depend on the others, so a product of independent uniform distributions does not generally describe it.
- Including singular endpoints in the support. Correlation matrices in this model are positive definite. A zero determinant marks the boundary, not an allowed positive-definite value.
- Assuming a boundary-diverging density creates boundary probability. For \(0<\eta<1\), the density can diverge near singularity while the prior remains proper and assigns zero probability to the singular boundary itself.
- Using the same verbal interpretation at every dimension. The LKJ family has a common determinant-based definition, but the correlation-matrix space and induced marginal behavior depend on \(q\). Check prior draws in the dimension of the model.
The main alternative is to encode stronger, specific prior information directly, rather than relying on one LKJ shape parameter to describe overall concentration. Whatever approach is used, the joint values must respect positive definiteness. LKJ priors are convenient and symmetric, but one scalar shape parameter cannot express every possible pattern of prior beliefs about particular pairs of random effects.
Check Your Understanding
Answer each question, then compare with the solutions below.
- For a two-variable correlation matrix, what does \(\eta=1\) imply about the density of \(\rho\)?
- For \(\eta=2\) in the bivariate case, calculate \(P(|\rho|<0.5)\).
- For a three-variable matrix with \(a=0.4\) and \(b=-0.2\), is \(c=0.82\) admissible? Explain using the determinant condition.
- Why is it incorrect to describe \(\eta=1\) as independent uniform priors on all pairwise correlations when \(q>2\)?
Solutions. (1) The density is constant at \(1/2\) on \(-1<\rho<1\), so \(\rho\) is uniform in this bivariate case. (2) The probability is \(\int_{-0.5}^{0.5}0.75(1-\rho^2)\,d\rho=11/16=0.6875\). (3) No. The strict upper bound is approximately \(0.8180\), and \(0.82\) exceeds it; the determinant is not positive. (4) Pairwise correlations are constrained jointly by positive definiteness. A constant density over valid matrices does not make the entries independent, and independent draws can produce invalid matrices.