Why Correlation Tests Need a Covariance Check
A random-effect correlation can describe a useful feature of a GLMM: for example, whether clusters with higher random intercepts also tend to have steeper random slopes. But the estimate and its test can become hard to interpret when the fitted covariance matrix is close to singular. A large estimated correlation is not, by itself, evidence of a precisely estimated relationship. It may instead reflect a direction in which the data provide little information, or a variance component that is close to zero.
The key distinction is between a well-defined null hypothesis and a weakly informed parameter. Testing a correlation of zero is a conventional one-parameter test when both random-effect variances are positive, the covariance matrix is safely inside the positive-definite region, and the model is identifiable. If a variance is zero, its correlation with another effect is undefined. If the covariance matrix is nearly singular, the correlation may be formally defined but poorly informed, and the usual large-sample reference distribution may not approximate the test statistic well.
Model and Meaning of the Correlation
For cluster \(i\), suppose the conditional response model has linear predictor \(\eta_{ij}=x_{ij}^{T}\beta+z_{ij}^{T}b_i\), with responses conditionally independent given the random effects. For a random intercept and random slope, write \(b_i=(b_{0i},b_{1i})^{T}\), with a bivariate normal distribution having mean zero and covariance matrix \(\Sigma\). The response distribution, link, fixed-effects structure, and random-effects distribution must be correctly specified for likelihood-based inference to have its intended meaning; clusters are assumed independent.
A common target is \(H_0:\rho=0\) against \(H_a:\rho\ne 0\), while retaining both random-effect variances in the model. This tests whether the two random effects have zero correlation, not whether either variance is zero and not whether the random-effects block can be omitted. For a regular likelihood-ratio comparison, fit both models to the same observations with the same fixed effects and response specification; the constrained model sets \(\sigma_{01}=0\), while the alternative estimates it. The usual large-sample reference is \(\chi^2\) with \(df=1\) only when the null is an identifiable interior restriction and the other parameters satisfy the regularity conditions for likelihood-ratio inference.
Why Singularity Changes the Interpretation
For the two-dimensional covariance matrix, its determinant is \(\sigma_0^2\sigma_1^2-\sigma_{01}^2\). A zero determinant means at least one eigenvalue is zero: a linear combination of the random effects has no variation. When both variances are positive, this occurs at \(|\rho|=1\). Near that boundary, small changes in the data or numerical optimization can produce large changes in the fitted correlation.
There is a different problem when a variance approaches zero. Since \(\sigma_{01}=\rho\sigma_0\sigma_1\), if \(\sigma_0\) is zero then the covariance must also be zero for a valid covariance matrix, no matter what value is assigned to \(\rho\). Thus \(\rho\) cannot be recovered from the covariance matrix at that null point. As \(\sigma_0\) becomes small, many possible correlations correspond to very small covariances. The likelihood can then change little as \(\rho\) changes: the parameter is weakly identified, even if software prints a numerical estimate.
These cases should not be conflated. Near \(|\rho|=1\), the covariance matrix is close to singular along a linear combination of effects. Near a zero variance, the correlation itself is becoming undefined. Either can make standard asymptotic inference unreliable, but for different reasons. A well-defined null requires that the two variances are positive and that the correlation is an identifiable interior parameter under \(H_0\); being close to a boundary can still make a finite-sample approximation poor.
Worked Covariance Checks
Worked Example: A Well-Conditioned Covariance Estimate
Suppose a hypothetical GLMM fit gives the random-intercept and random-slope covariance estimate, with the slope measured per one-unit increase in a centered predictor:
The estimated correlation is \(0.0240/\sqrt{0.160(0.0900)}=0.0240/0.120=0.200\). To check for a nearly singular covariance direction, calculate the eigenvalues. For a symmetric two-by-two matrix, they are half the trace plus or minus half the square root of the squared diagonal difference plus four times the squared off-diagonal entry.
Both eigenvalues are positive and neither is comparably tiny to the other; their ratio is about \(2.03\). The determinant is \(0.160(0.0900)-0.0240^2=0.013824\), also positive. These checks support treating the fitted covariance matrix as well-conditioned. They do not establish that the correlation test is valid by themselves: the data and likelihood must also identify the parameters, and the model assumptions must be reasonable.
Worked Example: A Correlation Near the Singular Boundary
Consider a hypothetical fitted covariance matrix with equal random-effect variances:
The estimated correlation is \(0.0396/\sqrt{0.0400(0.0400)}=0.0396/0.0400=0.990\). The eigenvalues are \(0.0400+0.0396=0.0796\) and \(0.0400-0.0396=0.000400\). The determinant is \(0.0400^2-0.0396^2=0.00003184\), and the eigenvalue ratio is \(0.0796/0.000400=199\). The matrix is positive definite, but one direction has very little estimated variance relative to the other.
This matrix does not prove that the test of \(H_0:\rho=0\) is invalid, nor does it show that the null is itself singular. It is a warning that the fitted alternative is close to the boundary \(|\rho|=1\), where the likelihood may be sensitive and the usual approximation may be inaccurate. A nominal \(p\)-value based on \(\chi^2_1\) should therefore be reported as conditional on the asymptotic approximation, not treated as definitive evidence. Inspect the likelihood profile and fitting diagnostics; if inference materially depends on this near-boundary fit, seek calibration appropriate to the setting rather than interpreting the printed \(p\)-value mechanically.
Worked Example: A Small Variance Makes the Correlation Hard to Interpret
In another hypothetical fit, suppose the estimated variances are \(0.000400\) for the random intercept and \(0.250\) for the random slope, and the estimated covariance is \(0.00600\). The formal correlation is \(0.00600/\sqrt{0.000400(0.250)}=0.00600/0.0100=0.600\). This value appears moderate, but the intercept variance is very small.
The covariance matrix is positive definite: its determinant is \(0.000400(0.250)-0.00600^2=0.000100-0.0000360=0.0000640\). Its eigenvalues are approximately \(0.250144\) and \(0.000256\), both positive. Yet the small intercept variance means the fitted intercept deviations contribute little variation, and changes in the correlation can correspond to small changes in covariance.
Therefore, \(0.600\) is an algebraically defined estimate in this fitted model, but its practical and inferential meaning requires caution. If the intercept variance were exactly zero, the covariance would have to be zero and the correlation would be undefined; a test of that correlation would not have a well-defined null parameter. Here the variance is positive, so the correlation is formally defined, but one should assess how well the likelihood profile supports its value instead of interpreting the estimate alone.
Test Conditions, Diagnostics, and Alternatives
For the standard likelihood-ratio test of a zero correlation, the null and alternative must be nested, fit to the same data, and differ only in the correlation restriction. Under \(H_0\), both random-effect variances should be positive and identifiable; the covariance structure should be away from a singular boundary; and the likelihood should be sufficiently regular for its large-sample approximation. When these conditions hold, twice the log-likelihood difference is compared with \(\chi^2_1\). The model must also converge to a genuine optimum rather than merely stop at a numerical tolerance.
Useful checks include the estimated variances and covariance, eigenvalues (or a singularity diagnostic based on them), the determinant, convergence messages, and sensitivity to reasonable optimizer settings. A profile likelihood for \(\rho\) can show whether the likelihood has a clear maximum and how rapidly it falls away from that maximum. A flat or irregular profile, a correlation estimate very near \(1\) or \(-1\), a near-zero variance, or unstable estimates across fits signals that the usual interpretation deserves scrutiny. The predictor scale matters for the numerical slope variance and covariance, so center or rescale predictors sensibly when appropriate; such changes do not eliminate a genuine lack of information.
| Question | Correlation test | Alternative target |
|---|---|---|
| Are the two random effects associated? | Test \(H_0:\rho=0\), when both variances are positive and \(\rho\) is identifiable. | Examine the profile likelihood for \(\rho\) and its uncertainty. |
| Is one random effect needed? | A correlation test does not answer this. | Test or estimate the relevant variance component using methods for boundary parameters. |
| Is the whole covariance structure needed? | A one-parameter correlation test does not compare all possible structures. | Compare clearly specified covariance models, using an appropriate reference calibration. |
The main alternative is not to substitute a different ordinary test automatically. If the scientific question concerns whether a random slope varies at all, test its variance rather than its correlation with the intercept. If the concern is the covariance structure as a whole, compare the intended structures and account for any boundary restrictions. Boundary likelihood-ratio tests have nonstandard reference behavior; their calibration depends on the null geometry and assumptions. The earlier material on variance components and boundary tests provides that context. Finite-sample calibration is a separate issue from deciding whether the correlation null is identifiable.
Common Mistakes and Limits
- Reading a large correlation as strong evidence. A high estimate can occur when a covariance matrix is close to singular. Check the eigenvalues and likelihood profile, not just the correlation number.
- Calling correlation zero when a variance is zero. Zero variance forces the associated covariance to zero, but the correlation becomes undefined. It is not a well-defined zero-correlation null.
- Using a variance-component test to answer a correlation question. A test that a variance is zero asks whether an effect varies across clusters; it does not test whether two nonzero random effects are correlated.
- Treating positive definiteness as proof of reliable inference. A positive but tiny eigenvalue can still indicate near-singularity and weak information. Numerical validity is not the same as inferential adequacy.
- Reporting the ordinary chi-square p-value without qualification. The \(\chi^2_1\) reference requires regularity and a well-identified interior null; near-singular fits can undermine that approximation.
These checks cannot repair a misspecified response distribution, an inadequate random-effects distribution, or a design with too little information to distinguish random effects. They also do not turn a near-singular estimate into a precise scientific conclusion. Report the covariance estimates and diagnostics alongside any test, and state clearly when the asymptotic reference is questionable.
Check Your Understanding
Answer each question, then compare with the brief solutions.
- If the random-intercept variance is exactly zero, can its correlation with a random slope be tested as \(H_0:\rho=0\)? Why?
- A fitted covariance matrix has eigenvalues \(0.12\) and \(0.0006\). What does this suggest, and what does it not establish?
- For variances \(0.09\) and \(0.16\), and covariance \(0.060\), calculate the correlation and determine whether the matrix is positive definite.
- What is the usual large-sample reference distribution for a likelihood-ratio test of \(H_0:\rho=0\) under regular interior conditions?
- A model reports a correlation of \(0.85\), but one random-effect variance is nearly zero. Give one diagnostic and one interpretive caution.
Solutions. (1) No. Positive semidefiniteness forces the covariance to zero, while the correlation denominator is zero, so the null parameter is undefined. (2) The matrix is positive definite if both eigenvalues are positive, but the ratio \(0.12/0.0006=200\) indicates a near-singular direction. This warns about sensitivity; it does not alone prove a particular test is invalid. (3) The correlation is \(0.060/\sqrt{0.09(0.16)}=0.060/0.120=0.500\). The determinant is \(0.09(0.16)-0.060^2=0.0108>0\), and both diagonal entries are positive, so the matrix is positive definite. (4) \(\chi^2_1\). (5) Inspect eigenvalues or the profile likelihood; the correlation may be weakly identified because many correlation values imply small covariance when one standard deviation is near zero.