Why Fit Warnings Need Classification
A GLMM fitting routine can return estimates alongside warnings, boundary indicators, or notes about the random-effects covariance. These messages do not all mean the same thing. An optimizer may have failed to locate a stable optimum; an optimum may exist but be poorly determined in some parameter direction; a variance estimate may lie at its allowed lower bound; or a fitted covariance matrix may have deficient rank. Each pattern has different implications for what can be reported and what should be checked next.
A warning is evidence to investigate, not a diagnosis by itself. Conversely, a software message that says a fit converged does not establish that every parameter is well determined or that the model is appropriate. The aim here is to classify the pattern visible in output before choosing a remedy.
Model and Four Distinct Failure Patterns
For orientation, let \(Y_{ij}\) be the response for observation \(i\) in cluster \(j\), and let \(b_j\) be that cluster's random-effects vector. A GLMM specifies a conditional response distribution, a link, and a linear predictor such as \(\eta_{ij}=x_{ij}^{\mathsf T}\beta+z_{ij}^{\mathsf T}b_j\). Conditional on the random effects and modeled covariates, responses are assumed to follow the specified family and be independent as required by the model; the random effects follow a specified distribution, commonly a mean-zero multivariate normal distribution with covariance matrix \(G\). The likelihood fit integrates over the random effects and estimates parameters subject to their constraints, including \(G\) being positive semidefinite.
The labels below describe numerical or parameter-space patterns, not four mutually exclusive causes. For example, an optimizer can stop at a boundary, and a boundary covariance can also be singular. The output should be read as a collection of clues.
These patterns are distinct. Nonconvergence concerns whether the numerical search has adequately located a solution. Curvature concerns how sharply the objective identifies parameters near a solution. A boundary estimate concerns where an estimate lies relative to its constraints. Singularity concerns the rank of a particular covariance matrix. One can occur without the others: a fit may converge to a well-defined interior solution, or converge to a singular covariance estimate.
Reading the Output in a Deliberate Order
Begin by recording the model fitted, the software's termination status, and any warnings. Then inspect the parameter estimates and covariance-related output. Do not infer a cause from one line alone: a successful termination flag addresses the algorithm's stopping rule, while a boundary or singularity note addresses the estimated parameter values.
A maximum-iteration message, failed line search, or explicit convergence warning supports classifying the run as a possible optimizer nonconvergence. It does not by itself show whether the model is unidentified.
Warnings about a non-positive-definite Hessian, a nearly singular information matrix, or unstable standard errors suggest ill-conditioned curvature. Interpret these as evidence of weak local determination, not as proof of one particular cause.
Look for a variance estimated as zero or nearly zero, a correlation close to its permitted extreme, or an explicit boundary note. These indicate a boundary pattern; the boundary may be scientifically plausible or may expose weak information.
When available, inspect its eigenvalues, rank, or a software singularity flag. One small eigenvalue indicates a weakly represented linear combination of random effects, even if no individual variance looks especially small.
Worked Output Patterns
Worked Example: An Optimizer Stops Before Meeting Its Criterion
In a hypothetical logistic random-intercept GLMM for 48 clinics, the optimizer reports “maximum iterations reached; convergence not achieved.” The negative log-likelihood is 218.47 at the last iteration, compared with 219.02 at the initial iteration. The reported random-intercept variance is 0.31.
The objective has decreased by \(219.02-218.47=0.55\), but that decrease alone does not establish that the last point is a solution. The explicit stopping message says the optimizer did not meet its convergence criterion within the iteration limit. Classify this run first as optimizer nonconvergence. The variance value 0.31 is neither zero nor enough to diagnose a boundary or singular fit.
The data and output do not show that another optimizer would necessarily succeed, or that the model is necessarily wrong. A responsible report would label the estimate provisional, retain the warning, and check whether the objective and estimates stabilize under suitable reruns and diagnostics. If the estimates change materially, conclusions based on this run are not reliable.
Worked Example: Convergence With a Nearly Flat Direction
Suppose a hypothetical model reports successful termination, but the local curvature matrix for two parameters has eigenvalues 80 and 0.0005. The larger-to-smaller eigenvalue ratio is
This very large ratio indicates that curvature is much weaker in one direction than in the other. If these are eigenvalues of the Hessian of the negative log-likelihood, the local quadratic approximation has a direction with little curvature. The fit can therefore be classified as having ill-conditioned curvature, despite the success message. That classification is not the same as optimizer nonconvergence: the algorithm may have met its numerical stopping rule while the data still provide weak local determination.
The ratio is a diagnostic clue, not a universal cutoff. Its interpretation depends on parameter scaling and the software's parameterization. Check whether standard errors, estimates, or substantive conclusions are unusually sensitive to small changes in the fit. Do not report the success flag as evidence that this weak direction is well estimated.
Worked Example: A Boundary Variance and a Singular Covariance
Consider hypothetical output for a random intercept and random slope. The estimated covariance matrix is
The random-slope variance is estimated as zero, the lower boundary for a variance. This is a boundary estimate. The eigenvalues of this diagonal matrix are its diagonal entries, 0.16 and 0. Its determinant is \(0.16(0)-0^2=0\), so its rank is one rather than two: the estimated covariance is also singular. These are two descriptions of the same fitted pattern, one referring to a parameter at its constraint and the other to the rank of the matrix.
Now consider a second hypothetical fit,
Both variances are positive, but the determinant is \(0.16(0.09)-0.12^2=0.0144-0.0144=0\). Thus the matrix is singular even though neither variance is zero. Its correlation is \(0.12/(\sqrt{0.16}\sqrt{0.09})=0.12/(0.4\cdot0.3)=1\), so the two random effects are perfectly linearly dependent in this fitted covariance. In finite-precision output, a very small rather than exactly zero eigenvalue may lead software to flag the same near-singular pattern.
Neither pattern alone proves that the corresponding population variability is exactly zero or that the model is useless. It says that the fitted covariance lies on, or very near, a lower-dimensional boundary. The information available for estimating all the specified random-effect variation may be limited.
Why Curvature and Covariance Rank Matter
A local quadratic approximation explains why weak curvature is a concern. Let \(\theta\) collect the model parameters and let \(L(\theta)\) be the negative log-likelihood. Near a candidate estimate \(\hat\theta\), the second-order approximation is
where \(H\) is the Hessian matrix of second derivatives. At a well-located interior minimum, the linear term is small and positive curvature gives a local increase in \(L\) as parameters move away from \(\hat\theta\). If \(H\) has a very small eigenvalue \(\lambda\) in direction \(v\), then for a displacement \(\delta=cv\), the quadratic increase is approximately \(\frac{1}{2}c^2\lambda\). When \(\lambda\) is small, appreciable movement in that direction changes the objective very little. This justifies treating weak curvature as an uncertainty and stability concern rather than as proof that the optimizer did not stop correctly.
A covariance matrix has a related but different geometry. A positive-semidefinite \(G\) has nonnegative eigenvalues; a zero eigenvalue means some linear combination of random effects has zero estimated variance. Therefore the estimated random-effect distribution occupies a lower-dimensional subspace. This is why checking each variance separately cannot always detect singularity.
Choosing Follow-Up Checks Without Overreacting
The next step should match the pattern, and no single intervention is guaranteed to fix it. For suspected optimizer nonconvergence, inspect the termination details and determine whether estimates and objective values are stable in a careful rerun. For ill-conditioned curvature, examine parameter scaling, dependence among model terms, and the precision of affected estimates. For boundary or singular patterns, consider whether the data and design contain enough information to estimate the specified random-effects structure, and whether a scientifically defensible simpler structure is appropriate.
These checks are not substitutes for understanding the model. A reduced random-effects structure changes the model and should be justified by the design and scientific question, not chosen only to silence a warning. A Bayesian fit uses a different inferential framework and can yield a proper posterior under conditions where a likelihood estimate is unstable, but a prior does not create information in the data; conclusions may depend on that prior and should be assessed accordingly. Bayesian sampling diagnostics are not optimizer-convergence diagnostics.
| Output pattern | Primary classification | What it does not establish by itself |
|---|---|---|
| Iteration limit or explicit failure to meet stopping rule | Possible optimizer nonconvergence | That the model is unidentified or that another run will fail |
| Nearly singular curvature or unstable local standard errors | Ill-conditioned curvature | That the optimizer failed to stop |
| Variance at its lower bound | Boundary estimate | That the true variance is exactly zero |
| Rank-deficient estimated \(G\), or effectively zero eigenvalue | Singular random-effects covariance | That any single variance must be zero |
Common Classification Mistakes
- “It converged, so the estimates are reliable.” A convergence status only summarizes an algorithm's stopping test. Weak curvature, boundary estimates, or singular covariance can still be present.
- “A zero variance means the optimizer failed.” A constrained optimum can occur at a boundary. It may be a valid likelihood result, although it calls for careful interpretation.
- “Every variance is positive, so the covariance is nonsingular.” As the second matrix example shows, nonzero variances do not prevent a zero eigenvalue or perfect dependence.
- “Any warning means the model must be discarded.” Warnings differ in meaning and severity. Classify the pattern, check stability and design information, and then decide what can be supported.
Check Your Understanding
Classify each pattern and explain the key distinction. Brief solutions follow the questions.
- A fit reports that it reached its iteration limit. Is that evidence of a boundary estimate?
- A fit reports successful termination and an estimated random-intercept variance of exactly zero. Name the main pattern.
- A two-dimensional random-effects covariance has positive diagonal entries but determinant zero. What does this imply?
- In one sentence, distinguish weak curvature from optimizer nonconvergence.
Solutions. (1) No. An iteration-limit message indicates possible optimizer nonconvergence; it gives no variance estimate or other evidence of a boundary by itself. (2) It is a boundary estimate. The successful termination does not remove that classification. (3) The covariance is singular because determinant zero means it is rank-deficient; positive variances do not guarantee full rank. (4) Weak curvature means the objective changes little in some direction near the reported estimate, whereas nonconvergence means the algorithm did not establish that its stopping conditions were met.