Deriving the OLS Sandwich Covariance

Why OLS Needs a Different Covariance Formula

OLS coefficients can remain useful when error variance changes across observations, but the familiar homoskedastic covariance formula no longer describes their sampling uncertainty. The key is to start with the exact finite-sample error in the OLS estimator. That decomposition reveals which part of the covariance depends on the design and which part depends on the errors’ conditional variances.

The resulting formula has a recognizable structure: a matrix determined by the design on each side, and a variance-weighted matrix in the middle. Since the conditional variances are unknown, the usual heteroskedasticity-robust OLS covariance estimator replaces that middle matrix with one built from squared OLS residuals. This tutorial derives that structure and its scaling; it does not require equal error variances.

Model and Finite-Sample Assumptions

Let \(y\) be an \(n\times 1\) outcome vector, \(X\) an \(n\times k\) design matrix, and \(\beta\) a \(k\times 1\) coefficient vector. The \(i\)th row of \(X\) is \(x_i'\), where \(x_i\) is a \(k\times 1\) vector of regressors. Write the linear model as \(y=X\beta+u\), where \(u\) is the \(n\times 1\) error vector.

These are assumptions about the data-generating process, not facts established by calculating a covariance estimator. In particular, a residual plot or a sample calculation cannot prove that the conditional mean is correctly specified or that observations are conditionally uncorrelated.

Under full column rank, the OLS estimator and its residual vector are

Conditional on \(X\), the design matrix is treated as fixed. Substituting \(y=X\beta+u\) into the estimator gives its finite-sample error exactly.

This is the starting point for the sandwich formula: OLS estimation error is the error vector transformed by \((X'X)^{-1}X'\).

Deriving the Finite-Sample Sandwich

Because \(E[u\mid X]=0\), the conditional mean of \(\hat{\beta}-\beta\) is zero. For any fixed matrix \(A\), the covariance of \(Au\), conditional on \(X\), is \(A\Omega A'\). Apply this rule with \(A=(X'X)^{-1}X'\). The matrix \(X'X\) is symmetric, so its inverse is symmetric as well.

The two copies of \((X'X)^{-1}\) are the bread. The middle matrix, \(X'\Omega X\), is the variance-weighted meat. These names describe the arrangement of the matrices; the derivation above, rather than the names, establishes the formula.

If errors are conditionally uncorrelated across observations, write \(\Omega=\operatorname{diag}(\sigma_1^2,\ldots,\sigma_n^2)\), where \(\sigma_i^2=\operatorname{Var}(u_i\mid X)\). Since \(X'\) has columns \(x_i\) in the corresponding rows of \(X\), the meat can then be written as a sum:

Thus heteroskedasticity changes the contribution each observation makes to the meat. If errors are conditionally correlated across observations, the full expression \(X'\Omega X\) still applies, but it contains cross-observation terms. The sum of individual \(x_i x_i'\) contributions by itself does not account for those terms.

From Unknown Variances to the HC0 Estimate

In practice, the individual conditional variances \(\sigma_i^2\) are unknown. A direct substitution of the OLS residual square \(\hat{u}_i^2\) for each \(\sigma_i^2\) gives the usual HC0 covariance estimator:

The name HC0 identifies the unadjusted residual-square version of the estimator. This derivation neither adds a small-sample correction nor assumes that one is needed. It also does not make the estimator robust to arbitrary dependence between observations: the displayed residual meat is designed for heteroskedasticity with conditional lack of cross-observation correlation.

Asymptotic Scaling and the Nonsingularity Condition

The finite-sample formula also reveals how uncertainty scales with \(n\). Define the normalized design matrix \(Q_n=n^{-1}X'X\) and the normalized variance-weighted design matrix \(B_n=n^{-1}X'\Omega X\). Whenever \(Q_n\) is invertible, substituting \(X'X=nQ_n\) and \(X'\Omega X=nB_n\) into the exact covariance formula gives

This identity explains the \(1/n\) covariance scale when the normalized matrices remain well behaved. To state a limiting covariance, however, convergence of the matrices alone is not enough: the limit of the normalized design must be invertible.

The displayed limit is the limit of the scaled conditional covariance. Interpreting it as the covariance in a limiting distribution for \(\sqrt{n}(\hat{\beta}-\beta)\) also requires an appropriate central limit theorem. The covariance algebra by itself does not establish that distributional result.

Common Mistakes and Why the Derivation Matters

• Using the homoskedastic formula by default. The expression proportional to \((X'X)^{-1}\) follows only under a common conditional error variance. With varying variances, the middle matrix must reflect observation-specific contributions.
• Calling the residual meat the true variance matrix. The true finite-sample meat is \(X'\Omega X\). HC0 substitutes residual squares because the conditional variances are unknown; the substitution makes it an estimator.
• Ignoring the required limiting rank condition. Finite-sample full column rank does not guarantee that \(Q_n\) converges to a nonsingular \(Q\). The inverse in the limiting covariance requires \(Q\) to be positive definite.
• Treating heteroskedasticity robustness as dependence robustness. The observation-level HC0 meat does not include cross-observation covariance terms. Conditional correlation requires a covariance formula that accounts for the relevant dependence structure.

The sandwich derivation matters because it separates two issues: how the regressors transmit error variation into coefficient variation, and how much error variation the data-generating process supplies. That separation makes the formula interpretable and clarifies which assumptions are needed before using it.