Zero Variance Versus Singular Covariance

Two Different Questions About Random-Effects Variation

A GLMM can estimate no variation in one random effect, or it can estimate a multivariate random-effects distribution whose variation is confined to a lower-dimensional space. These situations can both put a covariance estimate on the boundary of the allowed parameter space, but they answer different questions. The first concerns the size of one variance component. The second concerns the rank of a whole covariance matrix.

This distinction matters when interpreting a fit. A variance estimate of zero says that one modeled random-effect component has no estimated marginal spread. A singular covariance matrix says that at least one linear combination of the modeled random effects has no estimated spread. That combination need not be any one component by itself.

Definition: Let \(b_i\) be a vector of random effects for group \(i\), with mean zero and covariance matrix \(G\). A scalar variance estimate is zero when a one-dimensional variance component is estimated as \(0\). A covariance matrix \(G\) is singular (rank deficient) when its rank is less than its dimension, equivalently when it has at least one zero eigenvalue. For a valid covariance matrix, \(G\) must be positive semidefinite: \(a^\mathsf{T}Ga \geq 0\) for every vector \(a\).

In the usual Gaussian random-effects formulation, \(b_i\) has a multivariate normal distribution with covariance \(G\). The response distribution and link are specified conditionally on the random effects, and groups are modeled using the stated random-effects structure. The distinction discussed here concerns the fitted covariance \(G\), not whether the conditional response model is appropriate or whether the optimizer converged.

For a single random effect, \(G\) is a one-by-one matrix, \([\,\tau^2\,]\). If \(\hat{\tau}^2=0\), that matrix has rank zero and is singular. Thus a zero scalar variance is a special, one-dimensional case of covariance rank deficiency. In a multivariate model, however, singularity does not require any individual variance to be zero.

Why Rank Deficiency Means a Direction Has No Spread

The key calculation is the variance of a linear combination of random effects. For any vector \(a\), the variance of \(a^\mathsf{T}b_i\) is \(a^\mathsf{T}Ga\). If \(G\) is singular and positive semidefinite, there is a nonzero vector \(a\) in its null space, so \(Ga=0\). It follows that the corresponding linear combination has zero variance:

\operatorname{Var}(a^\mathsf{T}b_i) = a^\mathsf{T}Ga = 0.

Conversely, the eigenvalue decomposition of a positive semidefinite matrix writes \(G=Q\Lambda Q^\mathsf{T}\), where the diagonal entries of \(\Lambda\) are nonnegative eigenvalues. If one eigenvalue is zero, its eigenvector gives a nonzero direction with zero variance. Thus singularity means that some direction in random-effects space has no spread. It does not, by itself, say which component or combination is responsible.

A useful two-dimensional special case makes the distinction clear. If the variances are \(\tau_0^2\) and \(\tau_1^2\), and their correlation is \(\rho\), then

G = \begin{pmatrix} \tau_0^2 & \rho\tau_0\tau_1 \\ \rho\tau_0\tau_1 & \tau_1^2 \end{pmatrix}, \qquad \det(G)=\tau_0^2\tau_1^2(1-\rho^2).

When both variances are positive, this matrix is singular if \(|\rho|=1\). Neither marginal variance has to be zero: instead, the two random effects are perfectly linearly related in the fitted distribution. A zero scalar variance is therefore not the general definition of multivariate singularity.

Worked Example: A Single Variance Estimated at Zero

Suppose a hypothetical binomial GLMM models treatment response for patients in 18 clinics, with a clinic-specific random intercept \(b_i\) and \(b_i\sim N(0,\tau^2)\). A fit to hypothetical clinic data returns \(\hat{\tau}^2=0.000\) on the log-odds scale. What does this estimate say about the random-intercept covariance?

For a single random intercept, the covariance matrix consists of that one variance:

\hat{G}=[\hat{\tau}^2]=[0.000], \qquad \operatorname{rank}(\hat{G})=0.

The estimated standard deviation is \(\sqrt{0.000}=0.000\) log-odds units. Under the fitted model, the clinic random intercepts have no estimated between-clinic spread. This is a boundary estimate: variances cannot be negative, and zero is the lower boundary of the parameter space.

This calculation does not prove that clinic responses are identical, nor does it show that the fixed treatment effect is zero. It says only that this fitted model estimates no variation for the clinic random intercept. Sampling uncertainty, the data, and the model specification still matter.

Worked Example: Positive Variances but a Singular Matrix

Consider a hypothetical model with a group-specific random intercept and random slope. Suppose the fitted standard deviations are \(2.00\) log-odds units for the intercept and \(1.00\) log-odds per unit of a centered predictor for the slope, with estimated correlation \(1.00\). The fitted covariance matrix is

\hat{G} = \begin{pmatrix} 2.00^2 & (1.00)(2.00)(1.00) \\ (1.00)(2.00)(1.00) & 1.00^2 \end{pmatrix} = \begin{pmatrix} 4.00 & 2.00 \\ 2.00 & 1.00 \end{pmatrix}.

Its determinant is \(4.00(1.00)-(2.00)^2=0.00\). Its trace is \(5.00\), so its eigenvalues are \(5.00\) and \(0.00\). Therefore \(\hat{G}\) has rank one, although both marginal variances, \(4.00\) and \(1.00\), are positive.

To identify the zero-variance direction, let \(b_{0i}\) and \(b_{1i}\) denote the intercept and slope. The combination \(b_{1i}-0.500b_{0i}\) has variance

\operatorname{Var}(b_{1i}-0.500b_{0i}) =1.00+0.500^2(4.00)-2(0.500)(2.00) =1.00+1.00-2.00 =0.00.

The fitted effects therefore vary along one dimension, with \(b_{1i}=0.500b_{0i}\) in the degenerate fitted distribution. Neither the intercept variance nor the slope variance is estimated as zero. The absent variation is in their contrast, not in either component on its own.

Rank Deficiency Can Involve More Than Two Effects

In a model with three or more random effects, a singular matrix can have all positive diagonal entries without any pairwise correlation being exactly \(1\) or \(-1\). The zero-variance direction may involve several components together. Checking only marginal variances, or only looking for an extreme pairwise correlation, can therefore miss the distinction.

Worked Example: Three Positive Marginal Variances with Rank Two

Suppose a hypothetical GLMM includes three group-level random effects, \(b_{1i}, b_{2i}, b_{3i}\), with the following fitted covariance matrix. The effects are expressed in their respective model units.

\hat{G} = \begin{pmatrix} 1 & 0 & 1 \\ 0 & 1 & 1 \\ 1 & 1 & 2 \end{pmatrix}.

All three estimated marginal variances are positive: \(1\), \(1\), and \(2\). But the third row is the sum of the first two, so the matrix has rank at most two. Its upper-left \(2\times2\) block has determinant \(1\), so the rank is at least two; hence its rank is exactly two. Its determinant is \(1(2)-1(1)=1\) if computed using the wrong two-by-two shortcut, but the full determinant is

\det(\hat{G}) =1(2-1)-0(0-1)+1(0-1) =1-1 =0.

The vector \(a=(-1,-1,1)^\mathsf{T}\) satisfies \(\hat{G}a=0\). Thus

\operatorname{Var}(-b_{1i}-b_{2i}+b_{3i}) =a^\mathsf{T}\hat{G}a =0.

In this hypothetical fit, \(b_{3i}=b_{1i}+b_{2i}\) with no variation around that relationship. The example illustrates why the eigenstructure or rank of the full matrix can reveal a singular direction that the marginal variances alone do not show.

How to Diagnose and Describe the Difference

A practical assessment separates the scalar and matrix questions. First inspect each estimated variance on its own scale. Then, for a multivariate random-effects term, examine the full covariance matrix or its equivalent correlation-and-standard-deviation representation. The eigenvalues describe variation along orthogonal directions: a zero eigenvalue indicates exact rank deficiency, while a very small eigenvalue relative to the others indicates a nearly singular matrix.

Numerical software uses finite precision, so a displayed zero or a singular-fit flag may depend on tolerances and parameterization. A small positive eigenvalue is not mathematically zero, though it can signal that the fitted covariance is close to a lower-rank boundary. Compare eigenvalues relative to the overall scale of the matrix; an absolute threshold alone can be misleading when random effects have different units. The earlier material on eigenvalues and ill-conditioned fits, singular random-effects covariance matrices, and random-effects correlations near plus or minus one provides related diagnostic context.

A clear report states which case was found. For example, “the clinic random-intercept variance was estimated at zero” is more informative than “the covariance was singular” when there is only one random effect. For a multivariate term, report whether a marginal variance is near zero, whether a linear combination has little estimated variance, or both. Do not describe positive marginal variances as zero just because the matrix is rank deficient.

Fitted feature	What it establishes	What it does not establish by itself
One estimated variance equals zero	No marginal spread is estimated for that scalar component.	That all random effects have zero variation, or that the fixed effects are zero.
A multivariate covariance has a zero eigenvalue	At least one linear combination has zero fitted variance.	That any particular marginal variance is zero.
A multivariate covariance has a very small eigenvalue	The fitted matrix may be close to rank deficient on its scale.	Exact singularity, or a definitive explanation for why it occurs.

These distinctions are descriptive of the fitted covariance, not stand-alone tests of whether a random-effects term belongs in the model. A boundary estimate can arise from limited information, the realized data, or a mismatch between the assumed structure and the data. Likewise, a singular direction identifies a lack of estimated spread in a combination, but does not alone identify its cause. Convergence diagnostics and substantive model considerations remain relevant; a covariance estimate should not be interpreted in isolation.

Common Pitfall

Common mistake: “The covariance matrix is singular, so one random-effect variance must be zero.” This is false for a multivariate matrix. Positive marginal variances can still yield a zero eigenvalue because a linear combination has no spread. Check the diagonal entries and the eigenvalues or rank separately; then describe the zero-variance direction if it can be identified.

The converse wording also needs care. A scalar variance estimated as zero makes its one-dimensional covariance matrix singular, but it does not imply that every covariance matrix in a larger model is singular for that same reason. State the scope of the claim: which component or which covariance block is being discussed.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A fitted random-intercept model has \(\hat{\tau}^2=0.36\). Is its one-dimensional covariance matrix singular? What is the estimated standard deviation?
A two-dimensional fitted covariance matrix has positive variances \(9\) and \(4\), with covariance \(6\). Compute its determinant and decide whether it is singular.
A covariance matrix has eigenvalues \(4.00\), \(1.20\), and \(0.00\), with all diagonal entries positive. What does the zero eigenvalue imply?
Why is a correlation near \(1\) not the same statement as one marginal variance being zero?

Solutions. (1) No: the one-dimensional matrix is \([0.36]\), which has rank one. The estimated standard deviation is \(\sqrt{0.36}=0.600\), in the random-intercept units. (2) The matrix is \(\begin{pmatrix}9&6\\6&4\end{pmatrix}\), so its determinant is \(9(4)-6^2=36-36=0\). It is singular despite both marginal variances being positive. (3) There is a nonzero linear combination of the three random effects with zero fitted variance. Positive diagonal entries do not rule out this rank deficiency. (4) A marginal variance concerns the spread of one component; a correlation describes how two components vary together relative to their standard deviations. With positive variances and correlation \(1\), a contrast of the two effects can have zero variance even though neither marginal variance is zero.