Distinguishing Zero Variance from Weakly Identified Variance

Why a Zero Estimate Needs More Interpretation

A fitted generalized linear mixed model can estimate a random-effect variance as exactly zero. This is a boundary value: variances cannot be negative, so the likelihood is maximized at the edge of the parameter space. The estimate alone does not say whether the data tightly constrain the variance to be small or whether they contain too little information to distinguish small from substantial variation.

The distinction matters for conclusions. A narrow range of plausible variances can support the statement that between-group variation is small on the model's scale. A broad range can leave that question unresolved, even when the reported estimate is zero. The central diagnostic is the profile likelihood: how much fit is lost as the variance is fixed at progressively larger values and the other model parameters are re-estimated.

Model and Profile Likelihood

Consider a GLMM with observations $Y_{ij}$ in groups $i=1,\ldots,m$. Conditional on a group-specific random effect $b_i$, suppose the response follows the stated GLMM response distribution and link:

g\{E(Y_{ij}\mid b_i)\}=x_{ij}^{\mathsf T}\beta+z_{ij}^{\mathsf T}b_i,\qquad b_i\sim N(0,G).

Assume the random effects are independent across groups, the conditional response model is appropriately specified, and the design provides observations that can inform the variance component of interest. Let $\tau^2$ denote that component, with $\tau^2\geq 0$. Other parameters—including fixed effects, residual or dispersion parameters when present, and other covariance parameters—are nuisance parameters for this particular profile.

Definition: The profile log-likelihood for $\tau^2$, written $\ell_p(\tau^2)$, is the largest log-likelihood attainable when $\tau^2$ is held fixed at that value and all nuisance parameters are re-estimated. The profile likelihood-ratio deviance from the maximum is $$ D(\tau^2)=2\{\ell_p(\widehat{\tau}^2)-\ell_p(\tau^2)\}, $$ where $\widehat{\tau}^2$ maximizes the profile likelihood over $\tau^2\geq 0$. Thus $D(\tau^2)\geq 0$, and smaller values indicate less loss of fit relative to the best-fitting variance.

If $\widehat{\tau}^2=0$, then $D(0)=0$. This is true by construction: the fitted model's best variance is zero. It does not tell us how quickly $D(\tau^2)$ rises for positive values. That rise determines whether the profile excludes variances that would matter scientifically.

From a Profile to an Uncertainty Statement

A likelihood-based confidence set retains parameter values whose profile likelihood is not too far below its maximum. For an approximate one-sided 95% upper bound when the maximum is at zero, a common profile-likelihood rule retains values satisfying $D(\tau^2)\leq 2.706$, the 90th percentile of a chi-square distribution with one degree of freedom. The upper bound is the value where the profile first reaches the cutoff. This is an approximation; with few groups or other nonstandard features, calibration may need to reflect the design and fitting procedure.

This upper-bound rule is not the same calculation as testing the null hypothesis $H_0:\tau^2=0$. Under standard large-sample conditions for a single variance component, the likelihood-ratio statistic for that boundary null has an approximate mixture distribution: one-half point mass at zero and one-half chi-square with one degree of freedom. When the estimate is zero, the observed likelihood-ratio statistic for testing zero against the fitted alternative is zero. That boundary test alone cannot reveal whether the likelihood is sharply peaked at zero or nearly flat over a wide range.

The following short argument shows why the profile shape adds information beyond the boundary test. For any candidate $\tau^2$, compare its profiled log-likelihood with the best-fitting one. Then the relative likelihood is

\frac{\exp\{\ell_p(\tau^2)\}}{\exp\{\ell_p(\widehat{\tau}^2)\}} =\exp\left[-\frac{D(\tau^2)}{2}\right].

At the boundary maximum, the comparison at zero gives $D(0)=0$, so the relative likelihood there is 1. At positive values, however, $D(\tau^2)$ may rise steeply or remain small. For example, $D=2.706$ corresponds to relative likelihood $\exp(-2.706/2)\approx 0.258$. The likelihood at that candidate is about 26% of the maximum, which is the stated cutoff for the approximate upper bound.

Worked Example: A Tight Profile for a Clinic-Level Variance

Suppose a hypothetical logistic random-intercept GLMM is fitted to binary outcomes from 24 clinics, with 12 patients observed per clinic. The clinic random intercept has variance $\tau^2$ on the log-odds scale. Assume the conditional response model and the independence of clinic effects are reasonable, and that the model converges to the same optimum from multiple starting values. The fitted variance is $\widehat{\tau}^2=0$. The following are hypothetical profile deviances from refitting nuisance parameters at each fixed variance:

Fixed variance $\tau^2$ (log-odds squared)	Profile deviance $D(\tau^2)$
0.000	0.00
0.010	1.60
0.015	3.20
0.040	8.40

The 2.706 cutoff lies between 0.010 and 0.015. Using linear interpolation over this short interval gives

\widehat{U} =0.010+\frac{2.706-1.60}{3.20-1.60}(0.015-0.010) =0.010+\frac{1.106}{1.60}(0.005) \approx 0.0135.

The approximate one-sided 95% upper bound is therefore 0.0135 log-odds squared; its square root is about 0.116 log-odds. The profile also puts $D(0.040)=8.40$, well above the cutoff, so variances as large as 0.040 are not supported by this approximate profile set. This is evidence that a substantial clinic-level variance is inconsistent with these data under the fitted model. It is not proof that the true variance is exactly zero.

When the Estimate Is Zero but Information Is Weak

A flat profile is a different result. If increasing $\tau^2$ causes little reduction in profile likelihood, the data do not sharply separate zero from larger values. Common causes include few independent groups, sparse outcomes within groups, or a design in which the random effect is difficult to distinguish from other model features. Repeated observations within a group do not automatically substitute for having many independent groups when estimating between-group variation.

Worked Example: A Flat Profile Across School Variances

In a hypothetical logistic random-intercept model, 8 schools each contribute 15 students. Suppose the outcome model, covariates, and independence of school effects are plausible, but there are few independent schools. The random-intercept variance estimate is zero. A numerical profile calculation gives $D(0.10)=0.10$, $D(0.50)=0.45$, $D(1.00)=0.90$, and $D(2.00)=1.40$, with variances measured in log-odds squared.

Even at $\tau^2=2.00$, the deviance is below 2.706. Thus the profile has not reached the approximate 95% upper-bound cutoff anywhere in the displayed range. The correct conclusion is not that the variance is zero, nor that its upper bound equals 2.00. Rather, these calculations provide no finite upper bound within the range examined; the profile range and numerical fit should be checked, and the data appear weakly informative about the variance. The limited number of schools is a plausible explanation, but the profile alone does not identify the cause.

This example has the same zero estimate as the clinic example, but a different uncertainty pattern. In the first case, the profile quickly excluded larger variances. Here, several much larger values remain compatible with the approximate likelihood criterion. A report should describe this uncertainty rather than treating the point estimate as a precise finding.

Put Variance on a Useful Scale

The variance's numerical size has no universal meaning independent of the response model and link. For a logistic random intercept, $\tau^2$ is measured on the log-odds scale, not as a variance in outcome probabilities. For a Gaussian random intercept, it is measured in squared outcome units. Reporting the standard deviation $\tau=\sqrt{\tau^2}$, as well as the variance, may make the scale easier to interpret, but it does not change the likelihood evidence.

Worked Example: Assessing a Practically Important Batch Effect

Suppose a hypothetical Gaussian random-intercept model describes a continuous measurement in 30 production batches, with 10 measurements per batch. The batch effects are assumed independent and normally distributed, the residual model is plausible, and the profile fit is stable. The estimated batch variance is zero. The scientific team considers a batch standard deviation of 1.0 measurement unit important, corresponding to $\tau^2=1.00$ squared units. The hypothetical profile gives $D(0.25)=2.20$ and $D(1.00)=6.00$.

The cutoff 2.706 lies between 0.25 and 1.00. Linear interpolation gives

\widehat{U} =0.25+\frac{2.706-2.20}{6.00-2.20}(1.00-0.25) =0.25+\frac{0.506}{3.80}(0.75) \approx 0.350.

The approximate upper bound is 0.350 squared units, corresponding to a batch standard deviation of $\sqrt{0.350}\approx 0.592$ measurement units. Since the variance 1.00 is beyond this upper bound and has profile deviance 6.00, the data constrain the batch variation to below the team's stated important scale under this model and approximation. The conclusion is about the plausible magnitude, not a demonstration that the variance is exactly zero.

Diagnostics, Alternatives, and Limitations

A profile is only as useful as the model fit behind it. At each fixed variance, nuisance parameters must be re-optimized; holding them at their zero-variance estimates can distort the profile. Check that the optimizer converges across the variance range, that profiles do not contain unexplained jumps or numerical irregularities, and that extending the range does not reveal an unexamined cutoff crossing. A profile with multiple shoulders or discontinuities deserves investigation rather than mechanical interpolation.

Also assess whether the response distribution, link, random-effects structure, and independence assumptions are credible. Sparse binary outcomes, separation, influential groups, or a poorly identified covariance structure can make a variance profile unreliable. A narrow profile is conditional on the model being adequate; it does not protect against misspecification. With few groups, large-sample likelihood cutoffs may be inaccurate, so use a calibration appropriate to the design when needed.

Approach	What it contributes	Main caution near zero
Profile likelihood	Shows how fit changes over candidate variances and supports an upper bound.	Requires stable re-optimization and an appropriate cutoff.
Boundary likelihood-ratio test	Tests the zero-variance null against the fitted alternative using boundary calibration.	A zero statistic at a zero estimate does not reveal how broad the uncertainty is.
Wald standard error	Summarizes local curvature under a regular, approximately quadratic likelihood.	Can be misleading at a boundary, where the likelihood may be asymmetric or non-quadratic.

A Wald interval formed by estimate plus or minus a standard error can extend below zero or imply a symmetric uncertainty shape that the variance parameter cannot have. The profile approach respects the nonnegative parameter space and displays asymmetry, but its approximate cutoff remains an approximation. No single interval or test establishes literal absence of variation. A defensible conclusion combines the profile, the variance's practical scale, the number and structure of groups, and model diagnostics.

Common Pitfall

Correction: “The variance estimate is zero, so there is no group variation.” The estimate only identifies the likelihood maximum. If the profile is flat, larger variances may remain plausible, indicating weak information. If the profile rises sharply and its upper bound is small relative to a meaningful scale, the data support little variation under the model—but still do not prove an exactly zero population variance.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

If $\widehat{\tau}^2=0$, what additional profile feature distinguishes a tight estimate from weak identification?
A profile deviance is $D=0.80$ at a positive candidate variance. Is that value inside the approximate one-sided 95% profile set described here?
A profile never reaches 2.706 up to the largest variance examined. What should be reported?
Why can a Wald interval be a poor summary of uncertainty in a variance estimated at zero?

Solutions. (1) The rate at which $D(\tau^2)$ rises as variance increases: a steep rise yields a small upper bound, while a flat profile leaves larger values plausible. (2) Yes. Since $0.80<2.706$, that candidate is retained by the stated approximate profile rule. (3) Report that the profile has not produced an upper bound within the examined range; extend and check the numerical profile rather than calling the largest examined value the bound. (4) The parameter is constrained to be nonnegative and the likelihood near a boundary may be asymmetric or non-quadratic, unlike the symmetric local approximation underlying a Wald interval.