Why the Variance Must Follow the Clustering
In ordinary least squares, the coefficient estimate combines information from all observations. When observations in the same cluster may have correlated errors, however, those observations do not contribute independent pieces of information. The variance calculation must preserve how their contributions move together. The cluster sandwich variance does this by first adding each cluster’s regression score contributions, then taking an outer product of each cluster total.
For example, students in the same school may share conditions that affect their outcomes. Even if the fitted regression includes measured predictors, the remaining errors for students in one school may still be related. A calculation that treats every student’s score contribution as separate discards that possible relationship. The cluster sandwich keeps the contributions together. It changes the estimated uncertainty, not the ordinary least squares coefficient estimate.
Regression Scores and the Cluster Sandwich
Consider a linear regression with \(G\) clusters and \(n\) total observations. For observation \(i\) in cluster \(g\), let \(y_{ig}\) be the outcome, let \(\mathbf{x}_{ig}\) be a column vector of predictors (including an intercept if the model has one), and let \(\varepsilon_{ig}\) be the error. Write the model as \(y_{ig}=\mathbf{x}_{ig}'\boldsymbol{\beta}+\varepsilon_{ig}\). Let \(\mathbf{X}\) be the full design matrix, \(\mathbf{y}\) the outcome vector, and \(\hat{\boldsymbol{\beta}}\) the ordinary least squares estimate. For cluster \(g\), \(\mathbf{X}_g\) contains the rows of the design matrix for that cluster, and \(\mathbf{e}_g\) contains its ordinary least squares residuals.
The regression score contribution of observation \(i\) is \(\mathbf{x}_{ig}\varepsilon_{ig}\). The cluster’s score contribution is the sum of those observation-level contributions. In practice, the unobserved errors are replaced with fitted residuals.
The outer product \(\mathbf{s}_g\mathbf{s}_g'\) is a matrix, not a single squared score. Its diagonal entries contribute to estimated variances, and its off-diagonal entries contribute to estimated covariances between coefficient estimates. Summing scores within clusters before forming outer products is the key feature: the product includes cross-products between different observations in the same cluster.
Deriving the Formula
The derivation starts with the ordinary least squares identity. If the design matrix has full column rank, then \(\hat{\boldsymbol{\beta}}=(\mathbf{X}'\mathbf{X})^{-1}\mathbf{X}'\mathbf{y}\). Substitute \(\mathbf{y}=\mathbf{X}\boldsymbol{\beta}+\boldsymbol{\varepsilon}\), where \(\boldsymbol{\varepsilon}\) collects the errors:
Thus the coefficient error is the bread multiplied by the sum of the cluster error-score vectors. For the variance derivation, treat the design matrix as fixed and suppose errors have mean zero conditional on the design. Also suppose errors from different clusters have zero covariance with one another, while errors within a cluster may have any covariance structure. Let \(\boldsymbol{\Omega}_g=\mathrm{Var}(\boldsymbol{\varepsilon}_g\mid\mathbf{X})\) be the within-cluster error covariance matrix. Then the covariance of the sum has no cross-cluster terms:
This is the population variance expression behind the sandwich. The \(\boldsymbol{\Omega}_g\) matrices are unknown, so the formula cannot usually be evaluated directly. The cluster sandwich estimates each cluster’s contribution using its residual vector: it substitutes \(\mathbf{e}_g\mathbf{e}_g'\) for the unknown \(\boldsymbol{\Omega}_g\). The resulting meat is \(\sum_g \mathbf{X}_g'\mathbf{e}_g\mathbf{e}_g'\mathbf{X}_g\). Since \(\mathbf{X}_g'\mathbf{e}_g\) is \(\mathbf{s}_g\), this is exactly \(\sum_g\mathbf{s}_g\mathbf{s}_g'\), giving the stated estimator.
The substitution is an estimation step, not an assertion that one residual outer product equals the true covariance matrix for a cluster. Its justification is in the behavior of the aggregate estimator across clusters under suitable conditions. Those conditions and the limits of the approximation require separate attention; the formula alone does not guarantee reliable inference.
Worked Example: An Intercept-Only Regression
Suppose a hypothetical dataset has four observations in two clusters. The outcome values are 12 and 10 in cluster 1, and 9 and 9 in cluster 2. Fit an intercept-only regression and calculate the uncorrected cluster sandwich variance for the estimated mean.
The fitted intercept is the overall mean, \(\hat{\beta}=(12+10+9+9)/4=10\). The residuals are therefore \(2,0\) in cluster 1 and \(-1,-1\) in cluster 2. With an intercept-only model, each design row is 1, so each cluster score is the sum of its residuals: \(s_1=2+0=2\) and \(s_2=-1-1=-2\).
The bread is \((\mathbf{X}'\mathbf{X})^{-1}=1/4\), because \(\mathbf{X}'\mathbf{X}=4\). The meat is \(s_1^2+s_2^2=2^2+(-2)^2=8\). Thus
The variance is in squared outcome units, and the standard error is in outcome units. The residuals within cluster 2 point in the same direction, so their contributions reinforce each other in that cluster’s score sum. The fitted residuals also sum to zero across all observations, as required by the intercept-only least squares fit, but the separate cluster sums are not zero.
How the Outer Product Keeps Within-Cluster Covariance
To see what is retained, expand a cluster’s meat contribution:
Terms with \(i=j\) involve a single observation. Terms with \(i\ne j\) are cross-products between different observations in the same cluster. These are the terms that an observation-by-observation meat, \(\sum_{g}\sum_{i\in g}\mathbf{x}_{ig}e_{ig}^2\mathbf{x}_{ig}'\), leaves out. If errors within a cluster tend to move together, omitted cross-products can matter substantially. In terms of the population variance, the corresponding information is in the off-diagonal entries of \(\boldsymbol{\Omega}_g\).
Worked Example: Two Predictors and Two Cluster Scores
Consider a hypothetical regression with an intercept and a binary predictor \(x\). In each of two clusters, there is one observation with \(x=0\) and one with \(x=1\). Let the outcomes be 12 and 11 in cluster 1, and 8 and 13 in cluster 2, respectively. The fitted line is \(\hat{y}=10+2x\): the mean outcome at \(x=0\) is 10, and at \(x=1\) it is 12.
The residuals are \(2,-1\) in cluster 1 and \(-2,1\) in cluster 2. For a row \((1,x)\), the score contribution is \((e,xe)'\). Therefore the cluster scores are \(\mathbf{s}_1=(2-1,\;0(2)+1(-1))'=(1,-1)'\) and \(\mathbf{s}_2=(-2+1,\;0(-2)+1(1))'=(-1,1)'\). Their sum is zero, matching the least squares normal equations.
The design cross-product and its inverse are
The meat is \(\mathbf{s}_1\mathbf{s}_1'+\mathbf{s}_2\mathbf{s}_2' =2\begin{pmatrix}1&-1\\-1&1\end{pmatrix}\). Multiplying by the bread on both sides gives
For example, the estimated variance of the slope is the bottom-right entry, \(4.5\), so its cluster sandwich standard error is \(\sqrt{4.5}=2.121\) outcome units per one-unit increase in \(x\). This small example is for working through the matrix calculation, not for drawing a reliable inferential conclusion from two clusters.
Worked Example: Comparing Cluster and Observation Contributions
Suppose a hypothetical intercept-only regression has four outcomes, 12 and 13 in cluster 1 and 6 and 9 in cluster 2. The fitted mean is \((12+13+6+9)/4=10\), giving residuals \(2,3\) in cluster 1 and \(-4,-1\) in cluster 2. Each residual is also its observation-level score contribution.
The cluster scores are \(s_1=2+3=5\) and \(s_2=-4-1=-5\). The cluster meat is \(5^2+(-5)^2=50\), and the bread is \(1/4\). Hence the cluster sandwich variance is \((1/4)(50)(1/4)=3.125\) squared outcome units.
If the four score contributions were instead squared separately, their sum would be \(2^2+3^2+(-4)^2+(-1)^2=30\), giving \(30/16=1.875\). The difference in the meat is \(50-30=20\). Within cluster 1, the cross-product adds \(2(2)(3)=12\); within cluster 2, it adds \(2(-4)(-1)=8\). Together those within-cluster cross-products add 20. This shows directly how aggregating scores retains information that separate squares omit.
What the Formula Does—and Does Not—Do
The cluster sandwich is useful because it allows the errors within a cluster to be correlated and to have different variances, rather than requiring a single common error variance and zero correlation between all observations. Its meat is built from one score vector per cluster, so the cluster—not the individual observation—is the unit whose contribution is treated as independent in this calculation. The coefficient estimate itself remains the ordinary least squares estimate; only its estimated covariance matrix changes.
The formula is not a universal fix for dependence. In particular, it relies on a meaningful cluster definition and on independence across clusters for the variance decomposition used above. Nor does writing down CR0 guarantee that its approximation is accurate in a given dataset. The number and structure of clusters affect inference, and corrections or other methods may be needed in some settings.
A common mistake is to calculate \(\mathbf{x}_{ig}e_{ig}\) for every observation, square or take an outer product for each one, and then call the result cluster-robust. That calculation omits the \(i\ne j\) cross-products within clusters. The correction is to add the score vectors within each cluster first, then form the outer product of that cluster total. Reversing these steps changes the meat and generally changes the estimated variance.
Check Your Understanding
Use the definitions and derivation above to answer these questions.
- In words, what is the estimated score contribution \(\mathbf{s}_g\) for one cluster?
- Which part of the cluster sandwich is called the meat, and why is it a matrix?
- In the expansion of \(\mathbf{s}_g\mathbf{s}_g'\), what do the terms with different observation indices represent?
- Why does the cluster sandwich variance not change the ordinary least squares coefficient estimate?
- For an intercept-only model, residuals within a cluster are \(1\) and \(2\). What is that cluster’s score, and what is its contribution to the meat?