Sandwich Covariance Under Likelihood Misspecification

Why Likelihood Curvature May Not Measure Uncertainty

Maximum likelihood calculations often report uncertainty from the inverse curvature of the log-likelihood. That calculation has a special justification when the likelihood is correctly specified: the variability of the score and the curvature of the likelihood agree in expectation. If the likelihood is instead a working model, that agreement need not hold. The estimator can still target a meaningful parameter and be consistent for it, but its uncertainty depends on two distinct quantities: how variable the scores are across observations, and how sharply the expected log-likelihood curves around its optimum.

The sandwich covariance combines these quantities. It is not a different estimator of the parameter; it is a covariance estimator for the maximum-likelihood estimator under regular likelihood misspecification. Its central practical lesson is that inverse curvature alone can understate or overstate uncertainty when the working likelihood does not describe the full data distribution.

The Target and the Two Components

Suppose \(Y_1,\ldots,Y_n\) are independent and identically distributed observations from a true distribution \(P_0\). A working likelihood assigns observation \(i\) a log-likelihood contribution \(\ell_i(\theta)\), where \(\theta\) is a \(k\)-dimensional parameter. The contributions need not be the true log-density of the observations. Write the individual score as \(s_i(\theta)=\partial\ell_i(\theta)/\partial\theta\), a \(k\)-vector, and the individual Hessian as \(H_i(\theta)=\partial^2\ell_i(\theta)/\partial\theta\partial\theta'\), a \(k\) by \(k\) matrix.

The pseudo-true parameter is the best-fitting value according to the chosen expected log-likelihood criterion, not necessarily a parameter of the true distribution. The matrix \(A\) measures how strongly expected likelihood changes near that value. The matrix \(B\) measures how much individual scores fluctuate around zero under the true distribution. Under misspecification, there is generally no reason for \(A\) and \(B\) to be equal.

The result below assumes an interior, uniquely identified pseudo-true parameter; differentiability and suitable moment conditions; a nonsingular positive-definite \(A\); and an independent, identically distributed sample for which the score satisfies a central limit theorem and the sample Hessian converges to its expectation. These conditions support both a local score expansion and stable estimation of the two matrices. If the observations are dependent, the score covariance generally needs a dependence-appropriate form rather than the independent-observation formula given here.

Why the Sandwich Formula Appears

The maximum-likelihood score equation sets the sum of individual scores to zero. Expand that equation around \(\theta^*\). The average Hessian describes how much the average score changes when the parameter moves; its limit is \(-A\). The fluctuation of the average score at \(\theta^*\), by contrast, is governed by \(B\). Combining the two gives the sandwich.

This derivation separates the roles of the two matrices: curvature converts a score fluctuation into a parameter fluctuation, while score variability determines the size of the original fluctuation. It also explains why the matrix \(A^{-1}\) appears on both sides of \(B\). For a scalar parameter the two factors multiply to \(A^{-2}\), not \(A^{-1}\).

Estimating the Sandwich from a Fitted Model

Evaluate each contribution at the fitted estimate. A common empirical estimate of expected negative curvature is the observed negative Hessian averaged over observations. Estimate score variability with the average outer product of individual scores. The resulting covariance estimator includes a factor of \(1/n\), because \(A\) and \(B\) describe the covariance of the scaled estimator \(\sqrt n(\hat{\theta}-\theta^*)\), rather than the estimator itself.

At an unconstrained maximum, the sum of the fitted scores is zero, up to numerical tolerance. Thus their average is zero, and the displayed outer-product estimate is also their sample covariance when centered by their average. In other settings, explicitly centering the scores can be useful; the appropriate estimator depends on the estimating problem and its constraints. A second common presentation uses sums rather than averages: if \(\widehat{J}=-\sum_i H_i(\hat{\theta})\) and \(\widehat{S}=\sum_i s_i(\hat{\theta})s_i(\hat{\theta})'\), the same covariance is \(\widehat{J}^{-1}\widehat{S}\widehat{J}^{-1}\). The scaling must be consistent across the two presentations.

Correct Specification, Interpretation, and Pitfalls

When the likelihood is correctly specified and the usual regularity conditions hold, the information equality gives \(A=B\). The sandwich then simplifies to \(A^{-1}/n\), the familiar inverse-information covariance. Under misspecification, estimating only curvature still measures how quickly the working likelihood changes, but it does not capture the true variability of the scores. The sandwich retains both pieces.

A sandwich covariance supports standard errors, confidence intervals, and tests for the fitted parameters when the estimator is regular and targets the pseudo-true value. It does not make a misspecified likelihood correct, ensure that the pseudo-true parameter answers the scientific question of interest, remove bias from a poor target, or repair a failure of identification. Nor does the independent-observation formula automatically account for dependence or clustering. Those are separate issues requiring suitable assumptions and methods.

• Mistake: using curvature as the whole answer. The reasoning “the Hessian is steep, so the estimate must be precise” ignores the variability of the individual scores. Under misspecification, use both \(A\) and \(B\).
• Mistake: confusing the two matrix estimates. The negative Hessian estimates expected curvature; the outer products of individual scores estimate score variability. They are not interchangeable under misspecification.
• Mistake: omitting the sample-size scaling. The sandwich for \(\sqrt n(\hat{\theta}-\theta^*)\) is \(A^{-1}BA^{-1}\). The covariance for \(\hat{\theta}\) includes division by \(n\).
• Mistake: treating robust standard errors as a cure for model failure. They adjust uncertainty around the estimator's target; they do not turn the pseudo-true target into a true structural parameter or correct every form of misspecification.