Computing the Sandwich Bread and Meat

From Observation-Level Scores to Matrices

A sandwich covariance estimate is only as transparent as its two ingredients. The bread records how the estimating equation changes when the regression coefficients change; the meat records the variability of the observation-level estimating contributions. For independent regression observations, both can be assembled from one contribution per observation.

Consider independent responses \(Y_1,\ldots,Y_n\), with fixed covariate vectors \(x_i\in\mathbb{R}^p\). Write \(\eta_i=x_i^\mathsf{T}\beta\), \(\mu_i=E[Y_i]\), and \(\mu_i=m(\eta_i)\), where \(m\) is differentiable. Suppose the specified variance function is \(V(\mu_i)>0\), and the variance model is \(\operatorname{Var}(Y_i)=\phi V(\mu_i)\), with fixed positive dispersion \(\phi\). Define \(h_i=d\mu_i/d\eta_i\). The quasi-score contribution for observation \(i\) is

The quasi-estimator \(\hat\beta\) solves \(U(\hat\beta)=0\). Use \(\hat\mu_i=m(x_i^\mathsf{T}\hat\beta)\), \(\hat h_i=m'(x_i^\mathsf{T}\hat\beta)\), and residual \(\hat r_i=Y_i-\hat\mu_i\) to evaluate the contributions at the fitted coefficients. The empirical meat is the sum of their outer products:

Thus the meat depends on the squared residuals, as well as the fitted mean through the score weight. Independence is what permits a sum of individual score outer products: there are no cross-observation covariance terms in the variance of \(U\).

Two Bread Matrices to Distinguish

The sensitivity bread is the negative derivative of the total score. Its empirical version is \(\hat A_{\mathrm{obs}}=-\partial U(\beta)/\partial\beta^\mathsf{T}\) evaluated at \(\hat\beta\). Differentiating the observation contribution gives

The residual term appears because the score weight can change with \(\beta\). Another common bread is the expected sensitivity, with the residual term removed:

This is the negative expected score derivative evaluated at the fitted means: under the specified mean model, \(E[Y_i-\mu_i]=0\), so the derivative term multiplied by the residual has expectation zero. The two breads need not be identical in a particular data set. State which is being used; the observed bread is the empirical derivative, whereas the expected bread is its variance-model-based counterpart.

Nonsingularity matters: if the fitted design and weights do not give a full-rank bread, its inverse does not exist and this sandwich formula cannot be used as written. Dispersion in these expressions is treated as fixed or plugged in; differentiating with respect to an estimated dispersion parameter would require a larger estimating system.

Why the Model-Based Meat Can Match the Bread

Proof. Because \(x_i\), \(\mu_i\), and \(h_i\) are fixed at a fixed \(\beta\), the score contribution is a fixed vector multiplier of \(Y_i-\mu_i\). Its expectation is

Independence and zero expectations imply that the covariance of the sum is the sum of the individual covariance matrices. For each observation,

Summing these matrices proves the stated covariance identity. The empirical meat replaces the unknown score second moments with the observed score outer products; the identity does not say those observed products equal their expectations in a finite sample. \(\square\)

This calculation explains how a variance assumption enters each component. It determines the score weight, and hence the bread, through \(V(\mu_i)\) and \(\phi\). It also determines the model-based score covariance. In contrast, the empirical meat uses the observed squared residuals rather than replacing them by their variance-model expectations.

Worked Calculations

Practical Checks and Common Mistakes

A useful way to avoid arithmetic and conceptual errors is to construct the matrices observation by observation. For each row, compute the fitted mean, residual, score multiplier, and outer product \(x_ix_i^\mathsf{T}\); then sum the weighted matrices. The following checks are especially valuable:

• Check the fitted model. For a log link, fitted means must have log means linear in the covariates. Also verify that the fitted residuals satisfy the score equations to numerical tolerance.
• Keep the score multiplier squared in the meat. The meat weight is \(\{h_i/[\phi V(\hat\mu_i)]\}^2\hat r_i^2\), not simply \(\hat r_i^2\) for every link and variance function.
• Use an outer product. A contribution is \(\hat r_i^2x_ix_i^\mathsf{T}\) times its scalar weight. The off-diagonal entries are not generally zero, and symmetry is a useful arithmetic check.
• Do not confuse expected and empirical meat. Replacing squared residuals by \(\phi V(\hat\mu_i)\) gives a model-based expectation, not the observed meat.
• Identify the bread being used. The observed derivative can contain a residual-dependent term. Removing it is justified for expected sensitivity, not as an algebraic identity for every fitted data set.

The bread and meat are therefore not decorative labels for two interchangeable weighted cross-products. The bread describes score sensitivity, and the meat describes score variation. Their separate construction makes clear which parts depend on the fitted regression, which parts use residual evidence, and which parts follow from the assumed mean–variance relationship.