Common Errors in Clustered Regression Inference

Why Clustered Inference Is Easy to Misread

A regression data set may contain hundreds or thousands of rows without containing that many independent pieces of information. When observations are grouped—for example, students within schools or repeated measurements within people—outcomes from the same group may share influences. Treating every row as independent can then understate uncertainty. But clustered standard errors are not a general repair kit: they cannot create independent clusters, remove bias in a coefficient, or make an unsuitable mean model appropriate.

The central discipline is to keep three questions separate: what is the observational unit, what units can reasonably be treated as independent, and what quantity does the regression coefficient describe? A correct answer to one does not automatically answer the others. For example, the observational unit may be a student, the independent sampling or assignment unit may be a school, and the fitted coefficient may describe a linear association rather than a causal effect.

What Cluster-Robust Inference Relies On

Write the observations in cluster \(g\) as a response vector \(\mathbf{y}_g\) and a design matrix \(\mathbf{X}_g\). In the linear regression model, \(\mathbf{y}_g=\mathbf{X}_g\boldsymbol{\beta}+\mathbf{u}_g\), where \(\boldsymbol{\beta}\) is the coefficient vector and \(\mathbf{u}_g\) is the vector of errors for that cluster. Ordinary least squares uses all rows to estimate \(\boldsymbol{\beta}\). Cluster-robust uncertainty allows the entries of \(\mathbf{u}_g\) to be related within a cluster rather than requiring them to be independent row by row.

The usual cluster-robust framework requires the clusters to be independent of one another, a suitable mean or estimating condition for the coefficient target, and enough independent clusters for the chosen approximation to work well. Cluster sizes and leverage also matter in practice. The exact assumptions depend on the sampling and regression setup; “robust” does not mean that inference is valid under arbitrary dependence, a poor mean model, or any number of clusters.

The cluster-robust sandwich estimator estimates uncertainty by combining the regression’s design information with cluster-level contributions to the score. In a standard OLS form, its uncorrected version is

Here \(\hat{\mathbf{u}}_g\) contains the fitted residuals for cluster \(g\), and \(G\) is the number of clusters. Small-sample adjustments may modify this estimator or the reference distribution. The formula’s middle term groups residual contributions by cluster; it does not count each row as an independent cluster.

A Short Justification—and an Important Qualification

The reason the cluster-level sum appears can be seen first using the true errors, not the fitted residuals. Under the stated linear model, the OLS estimation error can be written as

Define the true-error score contribution for cluster \(g\) as \(\mathbf{S}_g=\mathbf{X}_g'\mathbf{u}_g\). If clusters are independent and these contributions have mean zero, then the true-error score contributions are independent across clusters. Expanding the variance of their sum gives

because independence across clusters makes the cross-cluster covariances zero. Within each \(\operatorname{Var}(\mathbf{S}_g)\), however, dependence among observations in cluster \(g\) is permitted. This motivates estimating the variance with a sum of cluster-level contributions.

There is a crucial distinction: the displayed identity is about contributions formed using the true errors (or corresponding population influence contributions). Fitted residuals \(\hat{\mathbf{u}}_g\) depend on the coefficient estimate fitted using all clusters. Their resulting estimated scores are therefore not generally independent across clusters in a finite sample. The sandwich estimator plugs fitted residuals into an estimator of the variance implied by the true-error contributions; independence is not an exact claim about those fitted-residual scores.

Error 1: Clustering at the Wrong Unit

The relevant cluster definition should follow the dependence and design, not merely the row identifier or the most convenient grouping variable. If a policy is assigned to schools and student outcomes may share school-level influences, clustering by student does not account for dependence among students in the same school. Conversely, clustering at a very broad level when the relevant independent units are more numerous can discard useful independent variation and may answer a different dependence question. With nested or crossed sources of dependence, a single clustering dimension may not represent the structure adequately.

A useful check is to explain, in words, why two observations in the same proposed cluster might have related errors, and why observations in different clusters can reasonably be treated as independent for the intended inference. The choice should also respect how the data were sampled or assigned. Sensitivity to defensible alternative cluster definitions can reveal that conclusions depend strongly on this choice, but trying several definitions does not make an incorrect one valid.

Error 2: Treating Records as Independent Clusters

A large \(n\) can create a false sense of precision when most additional rows come from already-observed clusters. Repeated measurements from one person, patients from one clinic, or students from one school add within-cluster information, but they do not create new independent clusters. This distinction matters especially when a predictor varies only between clusters: observing many people in each school does not produce many independent school-level comparisons.

Cluster counts should therefore be reported alongside the number of observations, and the number of independent clusters should inform how cautiously results are interpreted. In particular, conventional large-cluster approximations can be unreliable when \(G\) is small, even if the row count is large. Small-sample corrections may help in some settings, but they do not turn a handful of clusters into a large independent sample.

A related mistake is to use the number of clusters alone as a complete quality check. Unequal cluster sizes, influential clusters, and variation in the predictor across clusters can also matter. For a cluster-level predictor, for instance, the information about its coefficient is tied to how that predictor varies across independent clusters, not simply to the total number of people with recorded outcomes.

Error 3: Expecting Robust Standard Errors to Fix the Mean Model

A cluster-robust standard error changes the estimated uncertainty for a fitted coefficient. It does not change the coefficient estimate, add omitted predictors, remove confounding, or correct a misspecified conditional mean. If the model’s fitted mean is unsuitable for the scientific question, a different standard error cannot make its coefficient answer that question.

It is important to be precise about the consequence. Under appropriate conditions, a cluster-robust variance estimator can support inference about the coefficient of a best linear approximation even when the conditional mean is not exactly linear. In that case, the coefficient describes the specified linear projection for the population and predictor distribution in question; it should not automatically be interpreted as a constant change in the conditional mean or as a causal effect. If the intended target is a structural or causal parameter and the mean model is misspecified or relevant variables are omitted, robust standard errors do not remove the resulting bias.

A Practical Error-Checking Sequence

Before interpreting a clustered regression result, work through the following checks. They make it harder to confuse an appropriate variance calculation with a suitable analysis overall.

Common Mistakes and Corrections

• Wrong: “There are 500 students, so there are 500 independent units.” Correction: Count the clusters that can reasonably be treated as independent; 500 rows may represent only a small number of schools.
• Wrong: “I clustered by person, so school-level dependence is handled.” Correction: A cluster definition must capture the relevant shared dependence. Student-level clustering does not absorb school-level dependence among students.
• Wrong: “Robust standard errors make the regression model robust to misspecification.” Correction: They address variance estimation under specified conditions. They do not change the fitted coefficient or make its interpretation appropriate.
• Wrong: “The cluster-robust formula sums independent fitted-residual scores exactly.” Correction: The exact independence variance identity applies to true-error contributions across independent clusters. Fitted-residual scores depend on the full-sample fit; they are plug-in estimates, not exactly independent quantities.
• Wrong: “Many records compensate for very few clusters.” Correction: More rows within existing clusters do not necessarily provide the independent-cluster information required by cluster-robust approximations.