Why a Zero Variance Needs Diagnosis
A random-effects variance estimated as zero is not automatically evidence of a failed fit. It may indicate that the observations provide little support for between-individual variation in that model component. It may also arise because the likelihood is poorly identified, the covariance structure is unstable, or the optimizer has stopped at a boundary for numerical reasons. The estimate alone cannot distinguish these possibilities.
The diagnostic question is therefore not simply, “Did the optimizer return zero?” It is: does the observed data support a variance near zero, and do the likelihood, individual data, covariance diagnostics, and predictions tell a consistent story? This worked case uses a random-intercept model to make the reasoning explicit, then applies the same checks to a more general random-effects covariance.
Model and Assumptions
Suppose individual \(i\) has \(m_i\) repeated measurements, and the model for measurement \(j\) is
Here \(\mu\) is the population mean, \(\eta_i\) is the individual random intercept, and \(\epsilon_{ij}\) is residual error. For the calculations below, assume that random effects are independent across individuals, residuals are independent across observations, and random effects are independent of residuals. Also assume the stated normal distributions and a common residual variance \(\sigma^2\). These assumptions define this simple diagnostic model; a real pharmacometric model may have a nonlinear prediction function and a more complex residual structure.
For an individual with \(m\) measurements, the average is \(\bar{Y}_i=\mu+\eta_i+\bar{\epsilon}_i\). Independence of \(\eta_i\) and the residuals makes their covariance zero. Since the average of \(m\) independent residuals, each with variance \(\sigma^2\), has variance \(\sigma^2/m\), it follows that
This identity explains the basic data-support check: variation among individual averages reflects both between-individual variation and measurement noise. If the observed variation among averages is no larger than the amount expected from residual noise, the data may not support a positive \(\omega^2\).
A Quick Data-Support Check
In a balanced design with \(m\) observations per individual, a method-of-moments estimate suggested by the variance identity is the sample variance of the individual averages minus the estimated residual variance divided by \(m\). Because a variance cannot be negative, a negative raw estimate is a warning that the data do not show enough between-individual spread to separate it from residual noise; it is not a permissible negative variance estimate. This calculation is a diagnostic, not a substitute for fitting the specified likelihood.
Worked Example: Repeated Measurements Provide Little Between-Individual Support
Consider four hypothetical individuals, each measured twice. The measurements are 9 and 11; 9.5 and 11.5; 10 and 12; and 10.5 and 12.5 units. Assume the random-intercept model above, with independent random effects and residuals, and use the data to make a moment-based support check.
The individual averages are 10, 10.5, 11, and 11.5 units. Their mean is 10.75 units, so their sample variance is
Within each individual, the two observations are one unit below and one unit above that individual's average. The residual sum of squares is therefore \(4(1^2+1^2)=8\ \text{units}^2\), with \(8-4=4\) residual degrees of freedom. The pooled residual variance estimate is \(s^2=8/4=2.000\ \text{units}^2\). The moment-based variance component is
A negative variance is impossible. The calculation therefore gives no positive between-individual component after accounting for the estimated residual variation; a boundary value of zero is plausible. This small-data calculation does not establish that the population variance is zero. Nor does it show that a likelihood optimizer has found the correct solution. It identifies why the individual averages, by themselves, offer weak evidence for a positive component.
Profile the Likelihood Near Zero
A likelihood profile checks whether the boundary is actually favored after the other model parameters have been allowed to adjust. Let \(\ell(\omega^2,\psi)\) be the log likelihood, where \(\psi\) collects the remaining parameters, such as fixed effects, residual variance, and other random-effects parameters. The profile log likelihood is
At each fixed value of \(\omega^2\), the other parameters are re-estimated. A proper profile therefore differs from changing the variance while holding every other parameter at its estimate from the original fit. Compare the profiled objective at zero with values at a sequence of small positive variances. Inspect the shape, not just the single reported minimum: a clear increase away from zero supports the boundary, whereas a flat or erratic profile indicates weak information or unreliable optimization.
Worked Example: A Profile That Supports the Boundary
Suppose a hypothetical fit reports the following profiled values of \(-2\ell_p\), with smaller values indicating better fit. The nuisance parameters were re-optimized at every listed variance. The variance is in squared units of the random-intercept scale.
| \(\omega^2\) | Profiled \(-2\ell_p(\omega^2)\) | Increase from the minimum |
|---|---|---|
| 0 | 120.00 | 0.00 |
| 0.02 | 120.07 | 0.07 |
| 0.05 | 120.40 | 0.40 |
| 0.10 | 121.25 | 1.25 |
| 0.20 | 124.10 | 4.10 |
The minimum in this grid is at zero. For example, the profile change at \(\omega^2=0.10\) is \(121.25-120.00=1.25\) units of \(-2\ell\). The steady increase as the variance moves away from zero is consistent with a data-supported boundary, although the shallow change at very small values shows that the exact variance near zero is not sharply determined.
This table is a hypothetical optimizer output, not a universal likelihood-ratio cutoff. A profile can be irregular if nuisance parameters are not re-optimized reliably or if the parameterization creates numerical problems. Moreover, testing a zero variance is a boundary problem, so ordinary interior-parameter Wald reasoning and standard likelihood-ratio reference distributions should not be used automatically. The profile is useful here as a diagnostic of fit and information; formal inference requires a method appropriate to the boundary and model.
Check the Covariance Structure and Predictions
For multiple random effects, inspect the full estimated covariance matrix \(\Omega\), not only its diagonal. A valid covariance matrix is positive semidefinite: its variances are nonnegative, and its eigenvalues are nonnegative. A variance near zero can force associated covariances toward zero as well. For two effects, the covariance must satisfy \(|\omega_{12}|\leq\sqrt{\omega_1^2\omega_2^2}\). When a variance is exactly zero, the corresponding correlation is undefined because its denominator contains that zero variance. A reported extreme correlation near the boundary is therefore not, by itself, evidence of a strong scientific relationship.
Predictions provide a complementary check. Compare predictions under the boundary fit with those under a nearby positive-variance fit, using the same observations and covariates. Determine whether individual predictions materially change and whether residual patterns reveal systematic individual differences left unexplained by the boundary model. A small change in predictions is consistent with a practically negligible component, but is not conclusive: data may be too sparse to reveal an important effect. Conversely, a large, structured change warrants investigation of the model and data support before interpreting the positive-variance fit.
Worked Example: A Small Variance, an Unstable Correlation, and Similar Predictions
Suppose a hypothetical two-effect fit has an intercept variance of \(0.0400\), a slope variance of \(0.000400\), and an intercept-slope covariance of \(0.00380\). The covariance matrix is
The implied correlation is \(0.00380/\sqrt{0.0400(0.000400)}=0.00380/0.00400=0.950\). The determinant is \(0.0400(0.000400)-(0.00380)^2=0.00000156\), which is positive; this matrix is positive definite, though close to a singular structure because one variance is small. If the slope variance were exactly zero, a nonzero intercept-slope covariance would violate the covariance constraint, and the correlation would be undefined.
Suppose the slope effect is a random effect on log clearance and its estimated variance is \(0.000400\). Its standard deviation is \(\sqrt{0.000400}=0.0200\) log units. A one-standard-deviation multiplicative change is \(e^{0.0200}=1.0202\), about a 2.02% change in clearance. If fitted individual predictions under the zero-variance model and this nearby positive-variance model differ by about this scale, and residual checks show no systematic individual pattern, the slope component may have little practical impact in this data set. The calculation does not prove that the component is absent in the population; it quantifies the fitted scale and helps interpret the boundary.
Distinguishing Data Support from Fitting Failure
A boundary estimate is more credible as a data-supported result when several checks agree: the profile has its minimum at zero and rises smoothly for positive values; individual data provide little evidence of between-individual spread beyond residual noise; the covariance matrix remains valid and the fit is reproducible; and predictions do not reveal systematic patterns that the removed random effect should explain. These checks are complementary. No single diagnostic establishes the cause.
Treat the result as potentially numerical or structurally unstable when the profile changes erratically across nearby values, optimizer outcomes depend strongly on starting values, gradients or objective values are unreliable, the covariance matrix is not positive semidefinite, or individual likelihood contributions and predictions behave inconsistently. A nearly flat profile can reflect weak data support rather than optimizer failure, so inspect the model and data as well as the numerical output. The earlier material on replicate fits and covariance failures is useful context for these checks.
Check that the objective and parameter estimates are finite, the optimizer terminated as intended, and repeated fits do not produce incompatible solutions.
Fix the variance at zero and at nearby positive values, re-optimize nuisance parameters, and examine whether the profile is smooth and consistently favors the boundary.
Ask whether the number and timing of observations can distinguish individual variation from residual noise, and inspect individual-level residual and prediction patterns.
Confirm covariance validity, treat correlations near zero variances cautiously, and compare predictions with and without the component.
The main alternatives answer different questions. A likelihood profile examines the fitted objective over the variance while accounting for nuisance-parameter re-estimation. A moment-based calculation offers a quick view of between- versus within-individual variation but depends on its simplified assumptions and is not the fitted likelihood result. A Wald standard error relies on local, approximately symmetric behavior around an interior estimate; that approximation is poorly suited to a parameter estimated at zero. Finally, removing the random effect and refitting can show practical consequences, but it does not by itself establish whether the original zero was caused by limited data or a numerical problem.
Common Pitfall and Limitations
- Calling zero proof of no heterogeneity. A boundary estimate is a result under the fitted model and available data, not proof that the population variance is exactly zero.
- Reading a near-zero variance in isolation. Check the profile, individual information, covariance matrix, and prediction consequences together.
- Interpreting a near-boundary correlation literally. Correlation is poorly defined as either variance approaches zero and is undefined when a variance is exactly zero.
- Using a negative moment estimate as a variance. A negative value is a diagnostic indication that the observed spread does not support a positive component under that calculation; variance estimates remain nonnegative.
- Assuming every boundary fit is numerical failure. A smooth profile with a minimum at zero and consistent data and prediction diagnostics can indicate a genuine lack of support for that component in the current data.
These conclusions are limited by the model assumptions. Sparse or poorly timed observations, misspecified residual error, omitted predictors, or a random-effects structure that is too ambitious can all affect the apparent variance. A boundary diagnosis should therefore be reported in context: state which component reached zero, describe the profile and supporting checks, and avoid claiming more than the data and model establish.
Check Your Understanding
Answer each question, then compare with the solutions below.
- In the random-intercept model, why does \(\operatorname{Var}(\bar{Y}_i)\) include no covariance term?
- A profile has \(-2\ell_p(0)=80.0\) and \(-2\ell_p(0.05)=80.8\). What is the profile increase, and what does it establish?
- Why should a correlation estimate be treated cautiously when one associated variance is near zero?
- What is one reason that a zero variance estimate may be data-supported and one reason it may reflect a fitting problem?
Solutions. (1) The random effect is assumed independent of the residuals, so \(\operatorname{Cov}(\eta_i,\bar{\epsilon}_i)=0\); without that assumption, a covariance term would generally remain. (2) The increase is \(80.8-80.0=0.8\) units of \(-2\ell\). It indicates that zero has the better profiled objective among these two points, but two points alone do not establish the full profile shape or a formal inference. (3) The correlation divides the covariance by the product of standard deviations; as a variance approaches zero, that denominator approaches zero, making the correlation unstable, and at exactly zero it is undefined. (4) A smooth profile that rises away from zero and little individual-level spread beyond residual noise supports a boundary interpretation. Erratic profile values or strong dependence on starting values may instead signal numerical instability.