Why Conventional Standard Errors Fail Under Clustering

Why the Observation Count Can Mislead

A regression data set may contain thousands of rows and still provide far less independent information than that row count suggests. This happens when observations in the same school, clinic, firm, village, or other group share unobserved influences. If one group experiences a common shock, the errors for its members tend to move together. Treating every row as an independent source of information can then produce standard errors that are too small and tests that reject too often.

The issue is not simply that observations within a group resemble one another in their measured characteristics. It is dependence in the regression errors, conditional on the regressors. This tutorial identifies the assumption that conventional standard errors make, shows which terms they omit when that assumption fails, and illustrates why adding rows within the same groups does not necessarily solve the problem.

The Covariance Assumption Behind Conventional Standard Errors

Write the linear regression model for \(n\) observations as \(y=X\beta+u\), where \(y\) is the outcome vector, \(X\) is the regressor matrix, \(\beta\) is the coefficient vector, and \(u\) is the error vector. Assume \(X\) has full column rank. The ordinary least squares estimator is \(\hat{\beta}=(X'X)^{-1}X'y\). For inference conditional on \(X\), assume \(E(u\mid X)=0\). This condition gives the estimator its usual conditional unbiasedness; it does not, by itself, establish that a particular standard error is valid.

The condition \(\operatorname{Var}(u\mid X)=\sigma^2 I\) is stronger than merely requiring errors to have mean zero. It rules out both unequal error variances and any correlation across distinct observations. Earlier material on heteroskedasticity-robust inference relaxes the equal-variance part while retaining the requirement that distinct observations have zero conditional covariance. Thus an observation-level heteroskedasticity-robust calculation does not automatically handle clustering: it still does not include covariance between different rows.

For a general conditional covariance matrix \(\Omega=\operatorname{Var}(u\mid X)\), the actual conditional covariance of OLS is

The conventional formula follows when \(\Omega=\sigma^2 I\). If errors within groups are correlated, \(\Omega\) has off-diagonal entries for pairs of observations in the same group. Those entries contribute to the variance of \(\hat{\beta}\), but the conventional formula sets them to zero.

Shared Shocks and the Variance of a Mean

A simple grouped-outcome model makes the omitted terms visible. Suppose there are \(G\) independent clusters, with \(m\) observations in each cluster. For observation \(i\) in cluster \(g\), let

where \(a_g\) is a cluster-specific shock shared by all members of cluster \(g\), and \(e_{gi}\) is an observation-specific error. Assume \(E(a_g)=E(e_{gi})=0\); the cluster shocks are mutually independent with variance \(\tau^2\); the observation-specific errors are mutually independent with variance \(\sigma^2\); and the two collections are independent. Then each individual outcome has variance \(\tau^2+\sigma^2\), while two different outcomes in the same cluster have covariance \(\tau^2\).

The overall sample mean is \(\bar{Y}=(Gm)^{-1}\sum_g\sum_iY_{gi}\). Substituting the model and collecting terms gives

The two terms on the right are independent. The first averages \(G\) independent cluster shocks; the second averages \(Gm\) independent observation-specific errors. Therefore

A calculation that treats all \(Gm\) outcomes as independent, but uses their correct marginal variance, instead gives \((\tau^2+\sigma^2)/(Gm)\). The difference is \((m-1)\tau^2/(Gm)\), which is positive when \(m>1\) and \(\tau^2>0\). The problem grows more pronounced as more observations are added within each cluster while the number of independent cluster shocks remains fixed.

How Correlation Enters a Regression Coefficient

For regression, the same issue appears in the score contributions that determine the sampling variation of \(\hat{\beta}\). Let \(x_i\) denote the regressor vector for observation \(i\), and let \(u_i\) be its error. Before multiplying by \((X'X)^{-1}\), the relevant random sum is \(\sum_i x_i u_i\). Its conditional variance is

This expression contains both the \(i=j\) variance terms and the \(i\ne j\) covariance terms. The conventional formula keeps only the diagonal contribution implied by \(\sigma^2 I\). Observation-level heteroskedasticity-robust covariance calculations also build their variance estimate from individual-observation contributions; they do not include within-cluster cross-products. If two members of a cluster have positively correlated errors and their regressors point in similar directions, those cross-products can add substantially to the variance of a coefficient estimate. The size and direction of the effect depend on the regressor pattern and the covariance pattern; a smaller conventional standard error is common under positive shared shocks, but should not be asserted in every possible design.

A Direct Check on the Omitted Covariance

The omitted terms can also be checked for two observations in the same cluster. Suppose each has variance 1 and their covariance is 0.4. For their sum, the actual variance is

The diagonal-only calculation gives \(1+1=2\), omitting \(0.8\). This is the same algebraic omission that occurs in regression, where the errors are weighted by their regressor vectors before the covariance is assembled.

What Clustering Changes—and What It Does Not

Within-cluster dependence primarily changes the uncertainty calculation. If \(E(u\mid X)=0\) continues to hold and \(X\) has full column rank, OLS remains conditionally unbiased, even when errors are correlated. But a valid standard error is needed to quantify uncertainty, construct intervals, and conduct tests. If a reported standard error ignores relevant dependence, a confidence interval may be too narrow and a test may reject too frequently. A large observation count does not repair a covariance formula that systematically omits nonzero terms.

The effective amount of independent information depends on how the regressors vary relative to the clusters and how errors co-move within them. For a regressor that changes only between clusters, the number of clusters is especially important for comparison: many observations within each cluster can refine its average, but do not create new independent cluster-level regressor values. This does not mean that every clustered regression has information equal to the number of clusters. It means that the raw row count alone cannot establish that conventional inference is adequate.

The appropriate response is to use an inference method whose variance calculation reflects the dependence structure supported by the study design and assumptions. Heteroskedasticity-robust standard errors alone address unequal individual variances, not arbitrary within-cluster covariance. The construction and finite-sample behavior of cluster-aware covariance estimators require additional choices and are treated separately.

Common Mistake

Wrong reasoning: “There are 5,000 observations, so the standard error must be small and the test reliable. I already used heteroskedasticity-robust standard errors, so dependence is covered.”

Correction: A row count says how many observations were recorded, not how many independent error contributions the variance calculation contains. Observation-level heteroskedasticity-robust methods allow unequal variances but still omit covariance between distinct rows. If observations share a group shock, identify that dependence and use inference that accounts for it; do not infer validity from \(n\) alone.