Variance Components and the Parameter-Space Boundary

Why Zero Is a Special Value for a Variance

A variance component describes how much a random effect varies across the units to which it applies, such as clusters, hospitals, or subjects. Unlike a fixed-effect coefficient, it cannot take any real value: a variance must be nonnegative. This restriction makes zero a qualitatively different value. It is not an ordinary point inside the allowable range; it lies at the edge of that range.

That distinction matters for inference. Results derived for a parameter that can move a little in either direction around its null value may not apply when the parameter is constrained to move in only one direction. The key issue here is the geometry of the model's parameter space, rather than a particular test calculation.

Definition: A parameter value is an interior point of a parameter space if some open interval around it lies entirely within that space. It is a boundary point if every open interval around it contains values in the space and values outside it. For a variance component \(v\in[0,\infty)\), every \(v>0\) is interior, while \(v=0\) is a boundary point.

For a usual fixed-effect coefficient \(\beta\in\mathbb{R}\), the null value \(\beta=0\) is interior: small positive and negative departures are both allowed. For a variance component \(v\), the null value \(v=0\) is on the boundary: positive departures are allowed, but negative departures are not valid variances. The fact that zero is also a familiar null value does not make the two situations equivalent.

Where the Constraint Comes From

In a GLMM, a random effect is commonly modeled as a mean-zero random variable or vector with a covariance matrix. A covariance matrix must be positive semidefinite: for every vector \(a\), the variance of the corresponding linear combination of random effects cannot be negative. This condition is the source of the allowable parameter restrictions.

\operatorname{Var}(a^\mathsf{T}b)=a^\mathsf{T}Ga\geq 0 \quad\text{for every }a,

where \(b\) is a vector of random effects and \(G\) is its covariance matrix. In the simplest scalar case, \(G\) contains just one variance \(v\), so the requirement is \(v\geq 0\). At \(v=0\), the random effect has no variation: if its mean is zero, it equals zero with probability one. A negative value would make its variance mathematically invalid, not merely unusual.

This geometry also explains why a variance component is not just a signed coefficient with a different label. A coefficient can be perturbed locally in either direction while remaining in the model. At a zero variance, a small negative perturbation leaves the model altogether. The parameter space has a one-sided edge at the null.

Worked Example: Checking Whether Zero Is a Boundary

Suppose a hypothetical random-intercept model has variance component \(v\) restricted to \(v\geq 0\). Check the boundary definition at \(v=0\), using an interval radius of \(0.02\).

The interval of radius \(0.02\) centered at zero is \((-0.02,0.02)\). It contains allowable values such as \(0.01\), but it also contains values such as \(-0.01\), which are not in the parameter space \([0,\infty)\). Thus, this neighborhood contains both a value in the space and a value outside it.

The same reasoning works for every positive radius \(\epsilon\): the interval \((-\epsilon,\epsilon)\) contains \(\epsilon/2\geq 0\) and \(-\epsilon/2<0\). Hence zero is a boundary point. By contrast, if \(v=0.04\), the interval \((0.03,0.05)\) is entirely allowable, so \(0.04\) is an interior point.

What a Zero Variance Means in a GLMM

Consider a binary-outcome random-intercept GLMM with clusters indexed by \(j\) and observations indexed by \(i\) within each cluster. Let \(Y_{ij}\) be the outcome, \(x_{ij}\) a predictor, and \(b_j\) the cluster-specific random intercept. A standard conditional model is

Y_{ij}\mid b_j\sim\operatorname{Bernoulli}(p_{ij}),\qquad \operatorname{logit}(p_{ij})=\beta_0+\beta_1x_{ij}+b_j,\qquad b_j\sim N(0,v),\quad v\geq 0.

This specification assumes that, conditional on the random effects, the outcomes follow the stated Bernoulli model; the random intercepts have mean zero and common variance \(v\); and the random effects are independent across clusters. At \(v=0\), the distribution \(N(0,v)\) becomes degenerate at zero: every \(b_j=0\) with probability one. The conditional model then has no cluster-to-cluster variation in its intercept. This says nothing by itself about possible dependence from other sources or about whether the fixed effects are correctly specified.

Worked Example: Interpreting a Random-Intercept Variance

Consider a hypothetical binary GLMM with \(\beta_0=-1\), \(\beta_1=0.7\), and a binary predictor value \(x=1\). The cluster random intercept \(b_j\) is measured on the log-odds scale and has distribution \(N(0,v)\). Compare the conditional probability when \(v=0\) with probabilities at two illustrative random-intercept values when \(v=0.25\).

At \(v=0\), \(b_j=0\) with probability one. The linear predictor is \(-1+0.7(1)+0=-0.3\), so \(p=\operatorname{logit}^{-1}(-0.3)=1/(1+e^{0.3})\approx 0.4256\).

At \(v=0.25\), the random-effect standard deviation is \(\sqrt{0.25}=0.5\) log-odds units. For the illustrative value \(b_j=0.5\), the predictor is \(-1+0.7+0.5=0.2\), giving \(p=1/(1+e^{-0.2})\approx 0.5498\). For \(b_j=-0.5\), the predictor is \(-0.8\), giving \(p=1/(1+e^{0.8})\approx 0.3100\).

These calculations show how positive variance permits cluster-specific conditional probabilities to vary, whereas zero variance removes this random-intercept variation. The two values \(b_j=\pm0.5\) are illustrative, not an exhaustive description of the distribution. In particular, the probability at \(b_j=0\) is not generally the marginal probability obtained by averaging over a positive-variance random-effect distribution.

Covariance Matrices Have Boundaries Too

With multiple random effects, the parameter space is not simply one nonnegative number. The whole covariance matrix must be positive semidefinite. A covariance matrix with all eigenvalues strictly positive is positive definite and lies in the interior of the positive-semidefinite cone. If it has an eigenvalue equal to zero, it is singular and lies on the boundary of that cone. Variance components on the diagonal are themselves constrained to be nonnegative.

Worked Example: A Zero Component in a Covariance Matrix

For a hypothetical model with two random effects, consider the diagonal covariance matrix \(G=\begin{pmatrix}0&0\\0&0.04\end{pmatrix}\). Its eigenvalues are its diagonal entries, \(0\) and \(0.04\).

Both eigenvalues are nonnegative, so \(G\) is positive semidefinite and is an allowable covariance matrix. It is singular because its determinant is \(0(0.04)-0(0)=0\). The first random effect has variance zero, while the second has variance \(0.04\). Since one eigenvalue is zero, \(G\) is on the boundary of the positive-semidefinite cone.

Changing the first diagonal entry from \(0\) to \(-0.01\) would give a negative eigenvalue and an invalid covariance matrix. Changing it to \(0.01\) would make both eigenvalues positive and move this diagonal example into the interior. This illustrates how the nonnegative-variance constraint becomes a matrix constraint when several random effects are modeled.

Why the Boundary Changes Inferential Reasoning

Many familiar large-sample likelihood results are derived under regularity conditions that include an identifiable null parameter in the interior of the parameter space. For an interior scalar null, local departures can be positive or negative. Near a variance null of zero, only nonnegative departures are permitted. The local parameter space is therefore one-sided rather than a full line, so an argument that assumes unrestricted two-sided movement does not automatically apply.

This is the relevant connection to Wilks' theorem for regular fixed-effect tests: its usual reference result relies on regularity conditions, including an interior null. A fixed-effect null such as \(\beta_1=0\) may satisfy the interior condition when \(\beta_1\) is unrestricted. A variance-component null such as \(v=0\) does not. Recognizing this distinction does not, by itself, determine the correct reference distribution for a particular test; that requires additional analysis of the model and test.

State the allowable parameter space.
For a scalar variance, write \(v\geq 0\), rather than treating \(v\) as an unrestricted real number.

Locate the null within that space.
If \(H_0\) sets \(v=0\), the null is at the edge; if it sets \(v>0\), it is an interior value of the scalar space.

Check the assumptions behind the inferential result.
A result derived for an interior null cannot be applied automatically to a boundary null. The boundary must be accounted for when selecting and calibrating inference.

The constraint also affects numerical fitting. An optimizer that respects \(v\geq0\) may return an estimate exactly equal to zero. This can be a legitimate maximum on the allowed edge; it is not necessarily evidence that the fitting algorithm failed. Conversely, repeated warnings, sensitivity to starting values, or estimates that change markedly under small model changes may indicate a fitting or identification problem that deserves investigation.

Common Mistakes and Practical Checks

Treating a variance null like an unrestricted coefficient null. The incorrect reasoning is that both null values equal zero, so the same interior-parameter arguments must apply. The correction is to inspect the parameter space: \(\beta=0\) is usually interior when \(\beta\in\mathbb{R}\), whereas \(v=0\) is a boundary when \(v\geq0\).
Allowing a negative variance in an unconstrained calculation. A negative value is not an alternative model value for a variance component. Algebra that extends a local approximation to negative values must not be mistaken for an allowable fitted model.
Reading a zero estimate as proof that the population variance is exactly zero. A boundary estimate is the result of fitting finite data under a constraint. It does not establish the true value with certainty.
Assuming every random effect disappears when one variance is zero. A zero variance removes variation in that component. Other positive random-effect variances, fixed effects, or other sources of dependence may remain.
Applying an interior-parameter reference result without checking its assumptions. A familiar test statistic does not make the null interior. The reference distribution must be justified for the boundary setting.

A useful comparison is between an unrestricted fixed-effect parameter and a constrained variance component. Ordinary Wald reasoning for a fixed effect is built around local variation in both directions when the null is interior. A variance estimate constrained to be nonnegative can pile up at zero, so symmetric approximations centered at a positive estimate may not describe behavior at the null. Likelihood-based methods can accommodate parameter constraints, but their calibration still depends on the boundary geometry; an unconstrained calculation is not a substitute for that justification. These points explain why boundary-aware methods are needed without prescribing a particular test.

Key takeaway: A variance component belongs to \([0,\infty)\), so zero is a boundary value, not an interior null. The constraint follows from the requirement that variances and covariance matrices be nonnegative in the appropriate sense. Any inferential method used at that null must account for the one-sided parameter space.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

For \(v\geq0\), is \(v=0.2\) an interior point or a boundary point? Why?
A random-intercept GLMM has variance \(v=0\). What does this imply about the random intercepts, and what does it not imply?
Why is \(\beta=0\) ordinarily an interior null when \(\beta\in\mathbb{R}\), while \(v=0\) is not when \(v\geq0\)?
Is \(G=\begin{pmatrix}0&0\\0&0.09\end{pmatrix}\) positive semidefinite? Is it on the boundary of the positive-semidefinite cone?
A fitting algorithm returns a variance estimate of zero. Does this alone establish that the true variance is zero or that the algorithm failed?

Solutions. (1) It is interior: for example, the interval \((0.1,0.3)\) around \(0.2\) lies entirely in \([0,\infty)\). (2) A mean-zero random intercept with zero variance equals zero with probability one, so it contributes no cluster-to-cluster intercept variation. This does not rule out other random effects or other dependence. (3) Small positive and negative values of \(\beta\) are both allowed, but negative values of \(v\) are not; therefore only the variance null is on the edge of its space. (4) Its eigenvalues are \(0\) and \(0.09\), so it is positive semidefinite and singular; the zero eigenvalue places it on the cone's boundary. (5) No. Zero can be a legitimate constrained maximum, but an estimate alone neither proves the true variance is exactly zero nor diagnoses whether fitting was successful; the model fit and diagnostics also matter.