Hessian Checks at the Fitted Solution

Why Check Curvature at a Fitted GLMM?

A convergence message reports that an optimizer stopped according to its criteria; it does not by itself establish that the reported parameter vector is a well-behaved local optimum. A Hessian check examines the curvature of the fitted objective nearby. This matters both for assessing the candidate solution and for the usual local calculation of standard errors. In a GLMM, the objective may be a marginal log-likelihood or an approximation to it, depending on the fitting method. The check must be applied to the same objective that was actually optimized.

The sign convention is important. A likelihood maximizer should have downward curvature, whereas a minimizer of negative log-likelihood should have upward curvature. The criterion is the same once the sign is made explicit. Here \(\ell(\theta)\) denotes the log-likelihood or the likelihood approximation being optimized, and \(\theta\) denotes the vector of free parameters in the coordinates used by the optimizer.

Positive definite means that \(v^\mathsf{T}Jv>0\) for every nonzero vector \(v\). Thus the local objective curves upward in every parameter direction when minimizing negative log-likelihood. This is a strict local-curvature condition. It is not a statement about the objective everywhere, nor does it certify that the optimizer found the best solution among all possible solutions.

Why the Check Connects to Local Uncertainty

Suppose the objective is twice continuously differentiable near an interior fitted solution and the gradient is zero there. A second-order Taylor expansion of the negative log-likelihood gives, for a small displacement \(d\),

The linear term is absent because the gradient vanishes at a stationary point. If \(J\) is positive definite, this quadratic approximation has a unique local minimum at \(d=0\). Under the usual large-sample likelihood approximation, the corresponding local likelihood for \(d\) is proportional to \(\exp\{-d^\mathsf{T}Jd/2\}\), whose covariance matrix is \(J^{-1}\). This is the basis for the familiar approximate covariance and standard errors:

These are local, model-based approximations, not universal uncertainty guarantees. They require a suitable likelihood approximation, adequate data for the approximation, and a fit for which the local quadratic description is meaningful. A Hessian can be positive definite while uncertainty estimates remain poor for other reasons, including weak information in the data or a misspecified model.

How to Carry Out and Interpret the Check

The Hessian should be evaluated at the final parameter values, using the exact objective and parameterization associated with the fit. Analytical derivatives are useful when available; numerical derivatives can be used, but depend on finite-difference steps and numerical precision. Since the exact Hessian is symmetric for a sufficiently smooth objective, noticeable asymmetry in a computed matrix is a warning to check derivative accuracy, parameter scaling, and the objective evaluation before interpreting its curvature.

For a symmetric matrix, positive definiteness can be checked by a Cholesky factorization: a successful factorization with positive diagonal entries establishes positive definiteness. For a small matrix, Sylvester’s criterion gives an equivalent check: all leading principal minors must be positive. Neither a small gradient nor an optimizer’s convergence flag replaces this check; the gradient concerns first derivatives, while the Hessian concerns local second-order behavior.

What a Failed Check Means

A failed positive-definiteness check means that the computed curvature does not meet the condition for an ordinary, nondegenerate interior maximum or minimum under the stated sign convention. The pattern helps guide investigation, but a failure does not identify its own cause. Possible explanations include an optimizer that has not reached a stationary point, inaccurate numerical derivatives, an objective that is poorly behaved near the reported fit, or a solution on or near a constraint boundary. Some causes concern the numerical calculation; others indicate a genuine feature of the likelihood or parameter space.

If the observed information is indefinite, its inverse may have negative diagonal entries. Those entries cannot be interpreted as variances, so ordinary standard errors derived from that inverse are not meaningful. If the matrix is singular, the inverse does not exist. If it is positive definite but close to singular, inversion can still produce highly unstable uncertainty estimates. The next step is to examine the fit and computation rather than to report the output mechanically.

A useful comparison is with a gradient check. A small gradient suggests that first-order changes in the objective are small at the fitted point; it does not show that the point is a minimum or maximum. Conversely, positive-definite curvature alone does not show that the gradient is sufficiently small. Both are needed to support the interpretation of a smooth interior optimum. Further comparisons, such as profile-based likelihood intervals or resampling methods, can assess uncertainty in ways that are less dependent on a local quadratic approximation, but they do not make a failed fit automatically trustworthy.

Boundaries, Diagnostics, and Limitations

The interior condition matters. At a constrained optimum, a minimum may occur at the boundary even though the gradient is not zero in the usual unconstrained sense. For example, a variance parameter \(\tau\) is restricted to \(\tau\geq0\). If a negative log-likelihood is \(q(\tau)=q(0)+3\tau+2\tau^2\), then \(q'(\tau)=3+4\tau>0\) for \(\tau\geq0\), so the constrained minimum is at \(\hat{\tau}=0\). Its second derivative is \(q''(0)=4>0\), but the standard interior stationary-point argument does not apply: \(q'(0)=3\), and a symmetric, unconstrained quadratic approximation does not represent the one-sided parameter space. A positive Hessian in this case does not, by itself, validate ordinary symmetric standard errors at the boundary.

When a check fails, first verify that the Hessian belongs to the objective and parameter values actually reported. Then check numerical symmetry and derivative stability, and relate the result to the optimizer’s gradient and any parameter constraints. Re-evaluating derivatives more accurately or restarting optimization can help distinguish a numerical issue from a persistent feature of the fitted objective. If the failure persists, report that the usual local uncertainty calculation is unsupported and investigate the fit before drawing inferential conclusions.

A common mistake is to reason, “The optimizer converged, so the Hessian must be positive definite and the standard errors are valid.” Convergence criteria can be satisfied without establishing positive curvature, and positive curvature itself does not establish global optimality or model adequacy. The correction is to treat the Hessian check as a specific local diagnostic, alongside—not instead of—gradient checks, boundary assessment, and substantive model evaluation.

The check also depends on parameter coordinates. A smooth, one-to-one change of coordinates preserves the local character of a true stationary point, but numerical Hessians and the appearance of curvature can be affected by scaling and finite precision. A Hessian computed for a conditional random-effects mode is not automatically the Hessian of the marginal likelihood for fixed and covariance parameters. Always match the matrix to the inferential target.