Sandwich Variance for Moment-Based Estimators

Why Moment Estimators Need a Sandwich

A method-of-moments estimator is determined by equations that match sample moments to their model values. Its uncertainty depends on two features: how sensitive those equations are to changes in the parameters, and how much the observed moments vary from sample to sample. When several moments are used, their variances may differ and they may be correlated. Treating them as independent, or treating their variances as equal, can therefore give an incorrect standard error.

The sandwich covariance combines these features in a single expression. It applies to moment-based estimators under regularity conditions, whether or not the moment equations arise from a likelihood. The result is an asymptotic covariance: it describes uncertainty for sufficiently large samples, not an exact finite-sample distribution.

Definition: Let $X_1,\ldots,X_n$ be independent, identically distributed observations, and let $\theta$ be a $p$-dimensional parameter. A $q$-dimensional vector $g(X,\theta)$ of moment functions defines population moment conditions $E[g(X,\theta_0)]=0$ at the true parameter $\theta_0$. Write $\bar g_n(\theta)=n^{-1}\sum_{i=1}^n g(X_i,\theta)$. An exactly identified estimator solves $\bar g_n(\hat\theta)=0$ with $q=p$. Define the sensitivity matrix $A=E[\partial g(X,\theta_0)/\partial\theta^T]$ and the moment covariance matrix $B=\operatorname{Cov}(g(X,\theta_0))$.

Here $A$ records how the expected moment vector changes with the parameters, while $B$ records the variability and dependence among the moment components. For an exactly identified estimator, assume $A$ is nonsingular, the moments have finite second moments, and standard differentiability and convergence conditions hold. Then the sandwich covariance is

\operatorname{Cov}(\hat\theta)\ \approx\ \frac{1}{n}A^{-1}BA^{-T}.

The approximation is asymptotic. The factor $1/n$ converts the covariance of a single observation's moment vector into the covariance scale for a sample average. The inverse sensitivity matrices on the outside translate moment variability into parameter variability; $B$ sits between them, which gives the expression its “sandwich” name.

Why the Formula Holds

The derivation is a first-order linearization of the estimating equations. Expand $\bar g_n(\hat\theta)$ around $\theta_0$. Since the estimator solves the equations, and the population moments vanish at the true value, the expansion gives

0=\bar g_n(\hat\theta) \ \approx\ \bar g_n(\theta_0)+A(\hat\theta-\theta_0).

Solving this approximate equation for the estimation error yields

\sqrt{n}(\hat\theta-\theta_0) \ \approx\ -A^{-1}\frac{1}{\sqrt{n}}\sum_{i=1}^n g(X_i,\theta_0).

The multivariate central limit theorem says that the sum on the right converges in distribution to a mean-zero normal vector with covariance $B$. Multiplying a random vector by $-A^{-1}$ transforms its covariance to $(-A^{-1})B(-A^{-1})^T=A^{-1}BA^{-T}$. Thus the asymptotic covariance of $\sqrt n(\hat\theta-\theta_0)$ is the sandwich expression without the $1/n$; dividing by $n$ gives the approximate covariance of $\hat\theta$.

In applications, the unknown matrices are estimated by substituting $\hat\theta$ and the observed data. A common estimate of $B$ is the sample covariance matrix of the fitted moment vectors. Retaining its off-diagonal entries is essential whenever the moments may be correlated.

Worked Example: A Moment Estimate of an Exponential Mean

A hypothetical portfolio has $n=100$ independent, uncensored claim amounts from an exponential distribution with unknown mean $\theta$. The observed sample mean is $\bar X=520$ dollars. Estimate $\theta$ by matching the sample mean to the model mean, and use the model-based sandwich formula for its standard error.

Method and assumptions. For an exponential distribution with mean $\theta>0$, $E[X]=\theta$ and $\operatorname{Var}(X)=\theta^2$. The observations are assumed identically distributed and independent, with no truncation or censoring. Use the scalar moment function $g(X,\theta)=X-\theta$.

Estimate. The sample equation is $\bar g_n(\theta)=\bar X-\theta=0$. Hence $\hat\theta=\bar X=520$ dollars.

Sensitivity and moment variance. Differentiating gives $A=E[\partial g/\partial\theta]=-1$. For an exponential observation, $B=\operatorname{Var}(X-\theta)=\theta^2$. Substituting the estimate gives $\hat B=520^2=270{,}400$ dollars squared.

Sandwich standard error. In one dimension, the sandwich variance estimate is $\hat B/(nA^2)$. Therefore,

\widehat{\operatorname{Var}}(\hat\theta) =\frac{270{,}400}{100(-1)^2} =2{,}704\ \text{dollars}^2, \qquad \widehat{\operatorname{SE}}(\hat\theta)=\sqrt{2{,}704}=52.00\ \text{dollars}.

The result is checked by the familiar exponential-mean variance estimate $\hat\theta^2/n=520^2/100=2{,}704$. The minus sign in $A$ does not make the variance negative: the sensitivity appears twice, as $A^{-1}$ and $A^{-T}$.

Unequal Variances and Correlated Moments

With a vector of moments, $B$ is a full covariance matrix, not merely a list of separate variances. The entries on its diagonal are the individual moment variances; the off-diagonal entries are their covariances. The matrix multiplication in the sandwich formula uses all of them.

Worked Example: Two Moments for Normal Log-Severities

Suppose the logged amounts $Z=\log X$ for a hypothetical severity model are independent and normally distributed with mean $\mu=1$ and variance $\tau=4$. Estimate $(\mu,\tau)$ using the first two raw-moment equations. Find their asymptotic covariance matrix and standard errors for a sample of $n=64$.

Method and moment equations. For $Z\sim N(\mu,\tau)$, $E[Z]=\mu$ and $E[Z^2]=\mu^2+\tau$. Use $g_1(Z,\mu,\tau)=Z-\mu$ and $g_2(Z,\mu,\tau)=Z^2-\mu^2-\tau$. The empirical equations give $\hat\mu=\bar Z$ and $\hat\tau=\overline{Z^2}-\bar Z^2$.

Sensitivity matrix. Differentiating each moment with respect to $(\mu,\tau)$ and evaluating at $(1,4)$ gives

A= \begin{pmatrix} -1&0\\ -2&-1 \end{pmatrix}, \qquad A^{-1}= \begin{pmatrix} -1&0\\ 2&-1 \end{pmatrix}.

Moment covariance matrix. At the stated parameter values, $E[Z^2]=1^2+4=5$. The first variance is $B_{11}=\operatorname{Var}(Z)=4$. The covariance is $B_{12}=E[(Z-1)(Z^2-5)]=8$. For a normal variable, $E[Z^4]=\mu^4+6\mu^2\tau+3\tau^2$, so $E[Z^4]=1+24+48=73$. Thus $B_{22}=\operatorname{Var}(Z^2)=73-5^2=48$, and

B= \begin{pmatrix} 4&8\\ 8&48 \end{pmatrix}.

Sandwich calculation. Multiplying the matrices gives

A^{-1}BA^{-T} = \begin{pmatrix} -1&0\\ 2&-1 \end{pmatrix} \begin{pmatrix} 4&8\\ 8&48 \end{pmatrix} \begin{pmatrix} -1&2\\ 0&-1 \end{pmatrix} = \begin{pmatrix} 4&0\\ 0&32 \end{pmatrix}.

Therefore, for $n=64$, the asymptotic covariance matrix of $(\hat\mu,\hat\tau)$ is $\operatorname{diag}(4/64,32/64)=\operatorname{diag}(0.0625,0.5)$. The corresponding standard errors are $0.25$ for $\hat\mu$ and $\sqrt{0.5}=0.7071$ for $\hat\tau$, rounded to four decimal places where needed. The zero off-diagonal result follows from the matrix calculation; it does not mean the original moments were uncorrelated, since $B_{12}=8$.

More Moment Conditions Than Parameters

If there are $q>p$ moment conditions, the sample equations will generally not all equal zero at the same parameter value. A common approach is to choose $\hat\theta$ to minimize $\bar g_n(\theta)^T W\bar g_n(\theta)$, where $W$ is a fixed, symmetric positive-definite weighting matrix. Define $A$ as before. If $A^TWA$ is nonsingular and the model's moment conditions hold, the asymptotic covariance is

\operatorname{Cov}(\hat\theta) \ \approx\ \frac{1}{n} (A^TWA)^{-1}A^TWBWA(A^TWA)^{-1}.

To see the structure, the first-order condition for minimizing the quadratic criterion is approximately $A^TW\bar g_n(\hat\theta)=0$. Linearizing the moments around $\theta_0$ gives an estimation error proportional to $-(A^TWA)^{-1}A^TW\bar g_n(\theta_0)$. Taking the covariance of this linear expression introduces $B$ between the two factors $A^TW$, giving the formula above. When $q=p$ and the equations are solved exactly, this reduces to $A^{-1}BA^{-T}/n$.

Worked Example: Combining Two Correlated Measurements

For each of $n=25$ hypothetical paired observations, measurements $Y$ and $Z$ satisfy $E[Y]=\theta$ and $E[Z]=2\theta$. Their covariance matrix is $B=\begin{pmatrix}4&3\\3&9\end{pmatrix}$. Use the unweighted criterion $W=I$. The observed means are $\bar Y=5.2$ and $\bar Z=2.9$. Estimate $\theta$, then calculate its asymptotic standard error.

Method and estimate. Set $g=(Y-\theta,\ Z-2\theta)^T$. The sensitivity is $A=(-1,-2)^T$. Minimizing the sum of squared sample moments gives $A^T\bar g_n=0$, or $-(\bar Y-\theta)-2(\bar Z-2\theta)=0$. Thus $5\hat\theta=\bar Y+2\bar Z$, and $\hat\theta=(5.2+2(2.9))/5=2.2$.

Sandwich calculation. Since $A^TA=1+4=5$, the scalar sandwich variance per observation is $A^TBA/(A^TA)^2$. The numerator is $(-1,-2)\begin{pmatrix}4&3\\3&9\end{pmatrix}(-1,-2)^T=4+12+36=52$. Therefore the asymptotic variance of $\hat\theta$ is $52/(25n)=52/625=0.0832$, and its standard error is $\sqrt{0.0832}=0.2884$, rounded to four decimal places.

Check and trap. If the covariance $3$ were wrongly set to zero, the numerator would be $4+36=40$, giving variance $40/625=0.0640$ and standard error $0.2530$. The correct covariance contribution is $2(-1)(-2)(3)=12$, so dropping it understates uncertainty. The correct calculation also retains the unequal moment variances, $4$ and $9$, rather than assigning both the same variance.

Practical Checks and Limits

Before reporting a sandwich standard error, check that the moment equations describe the assumed population, the observations meet the independence and sampling assumptions used in the calculation, and the sensitivity matrix has sufficient rank. If $A$ is singular, the moment equations do not locally identify all parameters through this formula. If observations are dependent or clustered, the independent-observation covariance $B$ is not appropriate without adjustment. Likewise, truncation, censoring, or other sampling mechanisms require moment conditions and covariance calculations appropriate to that mechanism.

Do not omit the $1/n$. The sandwich without $1/n$ is the asymptotic covariance of the scaled error $\sqrt n(\hat\theta-\theta_0)$, not of $\hat\theta$.
Do not discard covariance terms. Off-diagonal entries of $B$ can increase or decrease the final variance.
Do not confuse the moment covariance with parameter covariance. The matrix $B$ describes moment variation; the sensitivity matrices are needed to translate it into parameter uncertainty.
Check dimensions. For $q$ moments and $p$ parameters, $A$ is $q$ by $p$, and the final sandwich is $p$ by $p$.

Key takeaway: The sandwich variance is the variability of the moment equations, $B$, transformed by their sensitivity to the parameters, $A$. For multiple moments, use the complete covariance matrix, including unequal variances and all relevant correlations.

Check Your Understanding

For each question, use the stated model and assumptions. Give standard errors to four decimal places where needed.

A hypothetical sample of $36$ independent exponential claim amounts has sample mean $\$600$. Using the moment estimator of the exponential mean and a plug-in model-based sandwich standard error, find the estimate and its standard error. (Answer: $\hat\theta=\$600$; standard error $\$100.00$.)
For independent $Z_i\sim N(\mu,\tau)$, the moment estimator of $\tau$ uses the second raw moment and the estimated mean. At $\tau=1$ and $n=100$, what are the asymptotic variance and standard error of $\hat\tau$? (Answer: variance $0.0200$; standard error $0.1414$.)
An overidentified scalar estimator uses $A=(1,2)^T$, $W=I$, and $B=\begin{pmatrix}1&0.5\\0.5&4\end{pmatrix}$, based on $100$ independent observations. Find its sandwich variance and standard error. (Answer: variance $0.0076$; standard error $0.0872$.)
For normal log-severities with known true parameters $\mu=0$, $\tau=2$, the first two raw-moment equations estimate $(\mu,\tau)$ from $40$ independent observations. Give the sandwich asymptotic covariance matrix and the two standard errors. (Answer: covariance matrix $\operatorname{diag}(0.0500,0.2000)$; standard errors $0.2236$ and $0.4472$.)
An overidentified scalar estimator has $A=(1,1)^T$, $W=\operatorname{diag}(1,1/4)$, and $B=\begin{pmatrix}4&1\\1&9\end{pmatrix}$, from $100$ independent observations. Calculate the sandwich variance and standard error. (Answer: variance $0.0324$; standard error $0.1800$.)

What You'll Learn