Sandwich Covariance for General Estimating Equations

Why a Sandwich Appears

Many estimators are defined not by minimizing a stated loss, but by solving equations built from the observations. The sample mean solves a sample average equation; regression coefficients solve equations involving regressors and residuals; and other estimators can be specified by their own moment conditions. In each case, sampling uncertainty depends on two distinct features: how much the sample equations fluctuate, and how sensitively those equations respond to a change in the parameter.

The sandwich covariance combines those features. Its middle component measures the covariance of observation-level moment contributions. The two outer components account for the way a change in the parameter changes the equations. This formulation is useful when the estimator is defined through general moments, including when the variance of the observations is not constant.

The prime denotes transpose. Both \(A\) and \(B\) are \(p\times p\) matrices. The derivative matrix \(A\) is sometimes called the sensitivity matrix: it describes how the mean estimating function changes locally with \(\theta\). The matrix \(B\), sometimes called the variability or score-covariance matrix, describes the variation in the individual contributions. Here “score” is a general term for an estimating-function contribution; it need not be a likelihood score.

Deriving the Asymptotic Covariance

The sandwich result requires regularity conditions. The observations are independent and identically distributed; the target is an interior, well-defined root of the population moment equation; \(\psi\) is differentiable in a neighborhood of \(\theta_0\); and \(A\) is nonsingular. Assume also that the moment contributions have finite second moments, that a central limit theorem applies to their sample average, and that \(\hat{\theta}\) is consistent and solves the sample equations. Appropriate smoothness and integrability conditions are needed for the Taylor remainder to be negligible. These conditions support a local linear approximation and ensure that the resulting covariance is finite.

Write \(\bar{\psi}_n(\theta)=n^{-1}\sum_{i=1}^n\psi(W_i,\theta)\). Since \(\bar{\psi}_n(\hat{\theta})=0\), a Taylor expansion around \(\theta_0\) gives

where \(\hat{A}_n\) is an average derivative evaluated between \(\theta_0\) and \(\hat{\theta}\). Under the stated conditions, consistency and the law of large numbers imply \(\hat{A}_n\) converges in probability to \(A\), while the Taylor remainder is negligible at the \(n^{-1/2}\) scale. Rearranging yields the first-order approximation

The central limit theorem gives \(n^{-1/2}\sum_i\psi(W_i,\theta_0)\) an asymptotic normal distribution with mean zero and covariance \(B\). Multiplication by \(-A^{-1}\) on the left and its transpose on the right therefore gives the covariance below. The minus sign has no effect on covariance.

Thus the asymptotic covariance matrix of \(\hat{\theta}\), rather than of \(\sqrt{n}(\hat{\theta}-\theta_0)\), is approximately \(A^{-1}BA^{-T}/n\). The two appearances of the inverse derivative matrix are essential: one transforms fluctuations into parameter changes, and the other is the corresponding transformation on the covariance’s transposed side. In a scalar problem this reduces to \(B/(nA^2)\).

Estimating the Two Components

In practice, replace the population quantities with sample analogues evaluated at \(\hat{\theta}\). Define the observation-level fitted contribution \(\hat{\psi}_i=\psi(W_i,\hat{\theta})\). The empirical derivative and empirical moment covariance are

When these quantities converge to \(A\) and \(B\), respectively, the plug-in sandwich covariance estimate is

The factors of \(1/n\) follow from this convention: both \(\hat{A}\) and \(\hat{B}\) are averages, and the covariance of the estimator is the covariance of the scaled estimator divided by \(n\). One can instead define sums for the matrix components, but then the corresponding scaling must change consistently.

This is a general estimating-equation formula, not a rule that every problem must use exactly the same observation-level function. The moment condition determines \(\psi\), and its derivative and outer product must both be calculated from that same function. In applications, alternative consistent ways of estimating \(B\) may be available, but the empirical outer product above is the basic observation-by-observation estimator.

Linear Regression as a Vector Example

For linear regression with an intercept and one regressor, let \(Z_i=(1,X_i)'\), let \(\theta=(\beta_0,\beta_1)'\), and use the moment contribution \(\psi_i(\theta)=Z_i(Y_i-Z_i'\theta)\). The two components of the sample equation set the sample residual sum and the regressor-weighted residual sum to zero. This example assumes independent observations and a nonsingular sample regressor cross-product matrix; the sandwich calculation does not require constant conditional variance.

What the Sandwich Does and Does Not Do

The matrix \(B\) captures the empirical or population variability of the moment contributions without requiring a particular constant-variance model. In regression, for example, the contribution \(Z_i e_i\) varies with both the regressor and the residual. Its outer product therefore allows the contribution’s variability to differ across observations. The derivative matrix \(A\) is a separate ingredient: it is determined by the estimating equation’s response to the parameter, not by the size of the residual variation alone.

The sandwich estimates sampling uncertainty around the parameter defined by the moment condition. It does not, by itself, guarantee that the moment equation defines the scientific quantity of interest, that the model is correctly specified for every purpose, or that the estimator is unbiased in a finite sample. Its large-sample justification also depends on the sampling assumptions: independence across observations is part of the setup here, and other dependence structures require appropriate covariance estimation.

A reliable calculation follows the same sequence each time: specify the observation-level moment, identify the population root and assumptions, differentiate that moment with respect to the parameter, calculate each fitted contribution, and combine the empirical components with the correct scaling. This makes clear which part of the calculation reflects moment variability and which part reflects sensitivity.