Sandwich Covariance Under Criterion Misspecification

Why Misspecification Changes the Covariance

An estimator can converge to a well-defined population target even when the model or criterion used to construct it is not a correct description of the data. That target is often called a pseudo-true parameter: it is the value that optimizes the population criterion, or solves the population estimating equations. The sampling variation around this target depends on two different features of the problem: how sharply the criterion curves near the target, and how variable the individual observations’ scores are.

Under correct specification, familiar formulas can sometimes identify these two features with one another. Under misspecification, they generally need not agree. The resulting covariance has a “sandwich” form: a measure of score variability is surrounded by inverse curvature matrices. This matters because using only curvature can give incorrect standard errors, even when the estimator itself is consistent for its pseudo-true target.

Sandwich Covariance for a Smooth Criterion

Suppose \(Z_1,\ldots,Z_n\) are independent and identically distributed observations, and an estimator \(\hat\theta\) maximizes the sample average criterion \(Q_n(\theta)=n^{-1}\sum_{i=1}^n q(Z_i,\theta)\). Let \(\theta\) be a \(p\)-dimensional parameter, and write the individual score as \(\psi(Z,\theta)=\partial q(Z,\theta)/\partial\theta\). Assume \(\theta^*\) is an interior maximizer of \(Q\), so that \(E[\psi(Z,\theta^*)]=0\).

Define the curvature matrix \(A\) and score-variability matrix \(B\) by

For a regular strict maximum, \(A\) is positive definite. The matrix \(B\) is the variance of the score at \(\theta^*\), since its mean is zero. Suppose the estimator is consistent for \(\theta^*\), the score and its derivative satisfy suitable moment and smoothness conditions, \(A\) is nonsingular, and a law of large numbers applies locally to the derivatives. Also suppose the multivariate central limit theorem applies to the scores, with finite second moments. These conditions yield

Here \(N(0,V)\) denotes a multivariate normal distribution with mean vector zero and covariance matrix \(V\). The asymptotic covariance matrix of \(\hat\theta\) itself is therefore approximately \(A^{-1}BA^{-1}/n\). The first and last factors describe inverse curvature; the middle factor describes score variability. Because a criterion Hessian is symmetric, \(A^{-1}\) is symmetric here. In the more general estimating-equation formulation, the corresponding expression is \(G^{-1}\Omega G^{-T}\), where \(G=E[\partial\psi/\partial\theta']\) and \(\Omega=E[\psi\psi']\).

Why the Expansion Uses an Integral

The key step is to expand the sample first-order condition around \(\theta^*\). For an interior maximizer, assume \(\hat\theta\) satisfies \(n^{-1}\sum_i\psi(Z_i,\hat\theta)=0\). Put \(\Delta=\hat\theta-\theta^*\). Applying the fundamental theorem of calculus along the line segment from \(\theta^*\) to \(\hat\theta\) gives the exact identity

This is an integral expansion, not an assertion that there is one intermediate parameter value at which a matrix-valued derivative gives the exact remainder. Under the local convergence conditions above, the matrix in braces converges in probability to \(-A\). Rearranging the identity therefore gives the asymptotic linear representation

The multivariate central limit theorem gives \(n^{-1/2}\sum_i\psi(Z_i,\theta^*)\xrightarrow{d}N(0,B)\). Combining that limit with the representation gives the sandwich covariance. In particular, the covariance follows from the variability of the score after it is transformed by inverse curvature; it is not obtained by treating the estimated criterion’s curvature alone as the full answer.

Estimating-Equation Version

The same argument applies when an estimator solves sample estimating equations rather than maximizing a stated criterion. Suppose \(n^{-1}\sum_i\psi(Z_i,\hat\theta)=0\), and let \(\theta^*\) solve \(E[\psi(Z,\theta^*)]=0\). Define \(G=E[\partial\psi(Z,\theta^*)/\partial\theta']\) and \(\Omega=E[\psi(Z,\theta^*)\psi(Z,\theta^*)']\). If \(G\) is nonsingular and the same local expansion, moment, and central limit conditions hold, then

To see the sign and transpose convention, the first-order expansion gives \(0=n^{-1}\sum_i\psi(Z_i,\theta^*)+G(\hat\theta-\theta^*)+o_p(n^{-1/2})\). Hence the leading term in the estimator’s error is \(-G^{-1}n^{-1}\sum_i\psi(Z_i,\theta^*)\). The minus sign does not affect the covariance, while multiplication by \(G^{-1}\) on the left and its transpose on the right gives the stated formula.

A Matrix Example: Heteroskedastic Regression Scores

For a regression illustration, let \(X=(1,x)'\), with \(x=-1\) or \(x=1\), each with probability \(1/2\). Suppose \(Y=1+2x+u\). Conditional on \(x=-1\), let \(u\) equal \(-1\) or \(1\) with equal probability; conditional on \(x=1\), let it equal \(-3\) or \(3\) with equal probability. Thus \(E[u\mid x]=0\), but \(E[u^2\mid x=-1]=1\) and \(E[u^2\mid x=1]=9\). Consider the least-squares estimating function \(\psi(Z,\beta)=X(Y-X'\beta)\).

Estimating the Sandwich and Avoiding Common Errors

In applications, the population matrices are replaced by sample analogues evaluated at \(\hat\theta\). For a smooth criterion, a common estimator is

The estimated asymptotic covariance of \(\sqrt n(\hat\theta-\theta^*)\) is \(\hat A^{-1}\hat B\hat A^{-1}\); the estimated covariance of \(\hat\theta\) is this matrix divided by \(n\). For general estimating equations, use \(\hat G^{-1}\hat\Omega\hat G^{-T}/n\). These are large-sample approximations; their reliability depends on the sampling and regularity assumptions, not merely on calculating the matrix.

• Using curvature alone. The inverse Hessian describes the result only with additional restrictions linking score variability to curvature. Under misspecification, calculate both matrices rather than presuming that link.
• Confusing the pseudo-true target with a correctly specified parameter. Consistency here means convergence to the population criterion’s optimizer or equation’s solution. It does not establish that this value has the interpretation it would have under a correct model.
• Applying a single intermediate-point formula to a vector score. For a vector parameter, do not assume a single intermediate point gives an exact matrix Taylor expansion. Use the integrated Jacobian along the path, as in the derivation above.
• Forgetting the sample-size factor. The sandwich \(A^{-1}BA^{-1}\) is the limiting covariance for the root-\(n\) scaled estimator. The approximate covariance for the unscaled estimator includes division by \(n\).

The sandwich is useful because it separates the estimator’s local sensitivity from the actual variability of the data’s contributions. It is not a correction for every possible design or dependence structure: the iid assumptions used here matter. Within those assumptions, however, it provides a principled covariance calculation when a working criterion or estimating equation does not correctly specify the population distribution.