Why a Random-Slope Fit Can Be Singular
A random slope allows the relationship between a predictor and the outcome to vary across groups. That flexibility is useful only when the data contain information about the different ways those group-specific coefficients can vary. In a generalized linear mixed model (GLMM), the fitted random-effects covariance matrix can lie on, or close to, the boundary of the set of positive-semidefinite matrices. The resulting fit is called singular when that matrix has less than full rank. It may indicate that the data do not support all the modeled random-effect directions; it is not, by itself, proof that the optimizer failed.
The diagnosis requires more than checking whether a reported random-slope variance is small. A slope and an intercept can be highly correlated, so a particular linear combination of them may have almost no estimated variation even when neither marginal variance is close to zero. Covariance eigenvalues reveal such combinations. The predictor values within groups then help assess whether the data could distinguish group-specific slopes from intercept differences.
Model and Meaning of Singularity
Suppose \(Y_{ij}\) is the response for observation \(i\) in group \(j\), and \(x_{ij}\) is a predictor with a random slope. A random-intercept–random-slope GLMM can be written as
Here \(g\) is the link function, \(\boldsymbol{\beta}\) is the fixed-effect coefficient vector, and the group-level random effects are assumed independent across groups and normally distributed with mean zero and covariance matrix
The model also assumes that the conditional response distribution and its mean-variance relationship are appropriately specified, and that observations are conditionally independent given the random effects and covariates. The covariance matrix must be positive semidefinite: its variances are nonnegative and its determinant is nonnegative. These conditions allow a zero eigenvalue, which is the boundary case of interest.
The eigenvalues describe variance along orthogonal directions in the two-dimensional random-effects space. If \(\mathbf{G}=\mathbf{Q}\boldsymbol{\Lambda}\mathbf{Q}^{\mathsf T}\), where \(\boldsymbol{\Lambda}\) contains eigenvalues \(\lambda_1,\lambda_2\), then a random-effects direction with eigenvalue zero has no variation under the fitted model. An eigenvalue near zero means that variation in that direction is estimated to be very small.
Reading Eigenvalues and the Near-Zero Direction
The following calculation shows why marginal standard deviations alone can miss a singular fit. Suppose a fitted covariance matrix, with random-intercept variance in the first row and random-slope variance in the second, is
Worked Example: Finding a Near-Zero Covariance Eigenvalue
Calculate the eigenvalues and assess whether the fit is near singular.
For a symmetric \(2\times2\) matrix, the eigenvalues are \(\{\operatorname{tr}(\hat{\mathbf{G}})\pm\sqrt{(G_{00}-G_{11})^2+4G_{01}^2}\}/2\). Here, the trace is \(4.25+0.04=4.29\), and the discriminant is
Therefore,
The eigenvalues are approximately \(4.2877\) and \(0.002332\). The determinant check gives \(4.25(0.04)-(-0.40)^2=0.17-0.16=0.01\), consistent with their product. The smaller eigenvalue is tiny compared with the larger one, so this is strong evidence of a near-singular covariance estimate. The fitted correlation is \(-0.40/\sqrt{4.25(0.04)}\approx-0.9701\), also indicating that the two effects are estimated to be strongly related.
The near-zero direction is approximately the eigenvector associated with the smaller eigenvalue. Solving \((4.25-0.002332)a-0.40b=0\) gives \(b\approx10.62a\). Thus, in these coordinates, a direction roughly proportional to \((1,10.62)\) has very little estimated variance. This is a statement about a combination of intercept and slope, not a claim that either marginal variance is zero.
Eigenvalues need contextual interpretation. The intercept and slope have different units, and changing the predictor’s origin or scale changes the numerical entries and eigenvalues of the covariance matrix. Exact rank deficiency is unchanged by a nonsingular change of coordinates, but the size of a near-zero eigenvalue relative to the others is not invariant. Check the predictor’s units and parameterization before treating a numerical tolerance as a universal definition of practical singularity.
Check Within-Group Predictor Variation
A random slope describes group-to-group variation in the predictor’s effect. Within-group changes in the predictor help distinguish that effect from a group-specific intercept. If \(x_{ij}\) is constant within a group, the contribution for that group is \(b_{0j}+b_{1j}x_j\), a single group-specific quantity. The observations in that group alone cannot separate its random intercept from its random slope. Groups with varying predictor values provide more direct information about slope behavior.
A useful descriptive quantity is the within-group sum of squares \(Q_j=\sum_i(x_{ij}-\bar{x}_j)^2\). This does not test whether a random slope is supported, and no single cutoff establishes identifiability. It does expose groups with no within-group variation and helps reveal whether slope information is concentrated in only a few groups.
Worked Example: Checking Predictor Variation Across Groups
A hypothetical data set has these predictor values in three groups: A has \(0,1,2\); B has \(4,4,4\); and C has \(8,9,10\). Calculate \(Q_j\) for each group and explain the diagnostic.
For A, \(\bar{x}_A=1\), so \(Q_A=(0-1)^2+(1-1)^2+(2-1)^2=2\). For B, \(\bar{x}_B=4\), so \(Q_B=0^2+0^2+0^2=0\). For C, \(\bar{x}_C=9\), so \(Q_C=(8-9)^2+(9-9)^2+(10-9)^2=2\).
Two groups show within-group predictor variation, while group B does not. Consequently, observations in B cannot separately reveal that group’s intercept and slope contributions. The predictor structure is not uniformly informative about slopes. This is a warning to inspect the fitted covariance and uncertainty alongside the design; it does not prove that the entire random-slope model is unidentifiable, because information also depends on the outcome data, number of groups, and the full design.
Plotting or tabulating predictor ranges by group can complement \(Q_j\). Look for many groups with a single predictor value, very narrow ranges, or slope information dominated by one or two groups. Also check whether the predictor varies mainly between groups rather than within them. Between-group differences may help estimate fixed associations, but they do not substitute automatically for within-group evidence about group-specific slopes.
Centering, Scaling, and Covariance Restrictions
Centering a predictor changes the meaning of the random intercept: it becomes the group deviation at the new zero of the predictor. Scaling changes the units of the random slope. With an unrestricted intercept–slope covariance, an affine change of predictor coordinates is a reparameterization of the same linear predictor, provided all corresponding coefficients and covariance terms are transformed.
Worked Example: Re-expressing a Covariance Matrix by Centering
Use the covariance matrix from the eigenvalue example and define \(x^*=x-10\). Since \(b_0+b_1x=(b_0+10b_1)+b_1x^*\), the new random effects are \(b_0^*=b_0+10b_1\) and \(b_1^*=b_1\). Compute their covariance matrix.
The transformed intercept variance is \(4.25+10^2(0.04)+2(10)(-0.40)=4.25+4-8=0.25\). Its covariance with the slope is \(-0.40+10(0.04)=0\), while the slope variance remains \(0.04\). Therefore,
The transformed matrix is diagonal and has determinant \(0.25(0.04)=0.01\), the same as before. The transformation has changed the numerical eigenvalues and removed the covariance in these coordinates; it has not changed the rank or created new information in the observations. In an actual refit, centering can also improve numerical conditioning and make the intercept interpretable at a useful predictor value. It cannot turn an exactly rank-deficient covariance into a full-rank one.
Scaling \(x\) can similarly make slope units more interpretable and improve numerical behavior, especially when predictor values are very large or very small. A full covariance model remains equivalent under a nonsingular linear re-expression if its covariance is transformed consistently. In contrast, fitting a diagonal covariance matrix by setting \(\tau_{01}=0\) is generally a model restriction, not merely a change of coordinates: it asserts that intercept and slope deviations are independent under their joint normal distribution. That may be scientifically reasonable, but should not be used as an automatic numerical repair.
How to Distinguish Diagnostics and Choose a Response
A singularity warning should be interpreted with the fitted covariance, predictor design, and the purpose of the model in view. Compare the marginal random-effect standard deviations and correlation with the covariance eigenvalues. Then inspect within-group predictor ranges and group sizes, and check whether centering or scaling materially improves numerical behavior. Re-expressing the same model is different from removing the random slope or imposing a zero covariance.
| Finding | What it suggests | Important qualification |
|---|---|---|
| One covariance eigenvalue is effectively zero | A random-effects direction has little or no estimated variation | Numerical thresholds depend on scale and software |
| Large intercept–slope correlation | The marginal effects may be difficult to distinguish in the chosen coordinates | Centering changes the intercept reference and correlation |
| Little within-group predictor variation | Groups provide weak direct information about slope differences | Variation is only one part of the information in the full model |
| Warnings change after rescaling or centering | Numerical conditioning or coordinate choice may matter | Compare equivalent full-covariance fits, not a restricted model as if equivalent |
A common mistake is to see nonzero intercept and slope variances and conclude that the fit cannot be singular. The eigenvalue calculation shows why this is wrong: a linear combination can have nearly zero variance even when both diagonal entries are positive. The correction is to examine the covariance matrix as a whole, including its eigenvalues and correlation, and then interpret that result in light of predictor scaling.
Another mistake is to treat a singularity warning as proof of failed optimization and repeatedly change optimizer settings. A fit can converge successfully to a boundary estimate because the likelihood supports little variation in one direction. Conversely, a warning alongside unstable estimates or failed convergence may indicate additional numerical problems. Eigenvalues diagnose the covariance structure; they do not by themselves certify convergence or identify the best scientific model.
Simplifying to a random-intercept model removes slope variation and changes the assumptions about group-specific associations. It can be a defensible alternative when the design and scientific question support it, but it should be compared and reported as a different model. Likewise, a diagonal covariance fit rules out intercept–slope covariance and is not a neutral substitute for a full covariance model. Reparameterization can improve computation and interpretation; it cannot supply information that the study design does not contain.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- For a positive-semidefinite random-intercept–random-slope covariance matrix, what does a zero smallest eigenvalue mean?
- If a group has predictor values \(2,2,2,2\), what is its within-group sum of squares \(Q_j\), and what limitation does this create?
- Does centering a predictor make an exactly rank-deficient covariance matrix full rank? Explain.
- Why is imposing \(\tau_{01}=0\) generally not just a numerical reparameterization of the unrestricted model?
Solutions. (1) The covariance matrix has rank less than two, so there is a random-effects direction with zero fitted variance. (2) The group mean is \(2\), so \(Q_j=\sum_i(2-2)^2=0\). Within that group, the intercept and slope contributions cannot be separated using changes in the predictor. (3) No. A nonsingular coordinate transformation preserves rank, so exact rank deficiency remains, even though the numerical eigenvalues may change. (4) The restriction sets the intercept–slope covariance to zero and excludes covariance structures allowed by the unrestricted model; it changes the model’s assumptions rather than simply changing coordinates.