Cluster-Robust Inference for DiD

Why Cluster-Robust Inference Matters in DiD

A DiD data set can contain many observations without containing many independent pieces of information. For example, thousands of patients may be observed in a small number of hospitals, while treatment is assigned at the hospital level. Outcomes for patients in the same hospital may share shocks, policies, or measurement practices. Treating all patients as independent can therefore make a DiD estimate appear more precise than the design supports.

Cluster-robust inference addresses this dependence by allowing observations within a cluster to be correlated. It does not change the DiD regression coefficient, determine which population is being averaged over, or establish a causal effect. Those are separate questions: the coefficient depends on the regression and its weighting, causal interpretation depends on identifying assumptions such as parallel trends, and the cluster-robust variance estimates uncertainty under a specified dependence structure.

In DiD, a useful default is to cluster at the level at which treatment is assigned, or at a broader level at which relevant shocks may be shared. If policy adoption is determined by state, for example, individuals or counties in the same state are not automatically independent sources of treatment variation. The exact choice still requires a design-based argument: clustering cannot protect against dependence that crosses the chosen cluster boundaries.

The DiD Regression and Its Clustered Variance

For a two-period DiD, a familiar regression is

Here \(Y_{it}\) is the outcome for unit \(i\) at time \(t\); \(Post_t\) indicates the post-intervention period; and \(Treat_i\) indicates membership in the treated group. In this specification, the coefficient \(\hat{\tau}\) on the interaction is the regression DiD estimate. In repeated-period or covariate-adjusted specifications, the design matrix \(X\) contains the corresponding terms, and the same cluster-robust construction applies to the fitted regression.

Let \(g=1,\ldots,G\) index clusters. Write \(X_g\) for the rows of the regression design matrix belonging to cluster \(g\), and \(\hat{u}_g\) for the vector of OLS residuals in that cluster. The unadjusted cluster-robust, or CR0, covariance estimator is

The middle term is often called the “meat” of the sandwich. Each cluster contributes its whole score product \(X_g'\hat{u}_g\hat{u}_g'X_g\), including cross-products between observations in that cluster. Those cross-products are precisely what would be lost by treating each row as independent. The estimated variance of \(\hat{\tau}\) is the diagonal entry of \(\widehat{V}_{CR0}\) corresponding to the interaction coefficient; its standard error is the square root of that entry.

A common finite-sample adjustment is CR1:

where \(N\) is the number of regression observations and \(K\) is the number of estimated regression coefficients, including the intercept. This is one common convention, not a universal definition: software may use different corrections or degrees-of-freedom rules. The adjustment scales the estimated covariance matrix; it does not alter the OLS coefficients or make a small number of clusters behave like a large one.

Why the Number of Clusters Matters

The sandwich is built from \(G\) cluster-level score contributions, not from \(N\) independent contributions. When there are many observations per cluster but few clusters, the apparent sample size \(N\) can be large while the information relevant to the cluster-robust approximation remains limited. A large number of rows does not, by itself, justify using a normal approximation for a cluster-robust test.

For large-cluster inference, a coefficient is commonly tested using a Wald statistic: subtract the null value from the estimate and divide by its cluster-robust standard error. Reference distributions and degrees-of-freedom conventions vary; a \(t\) reference with \(G-1\) degrees of freedom is one common choice. Whatever convention is used, the number of independent clusters should be reported and considered. A CR1 multiplier is a modest correction, not a remedy for a very small \(G\). In that setting, specialized small-cluster inference is needed.

Clustering Does Not Choose the Estimand

The clustering level and the weighting in the regression answer different questions. Clustering specifies which observations may have correlated errors for the variance calculation. Weighting determines how observations or groups contribute to the point estimate. Changing the cluster variable while holding the regression and sample fixed ordinarily leaves its OLS coefficient unchanged, although its standard error may change. Changing weights or the regression specification can change the coefficient and thus the estimand.

For example, an unweighted regression on individual records generally gives more total influence to larger clusters than to smaller ones. A regression using one equally weighted observation per cluster instead targets a different average when cluster sizes differ. Merely declaring clusters for the variance calculation does not convert the first regression into the second.

A Short Derivation and a Numerical Variance Calculation

The sandwich formula follows from the OLS score representation. Conditional on the design matrix, write \(A=X'X\). The OLS estimation error is

If cluster errors have mean zero conditional on \(X\), and distinct clusters are independent, the variance of the sum is the sum of its cluster variances. Thus

This step does not require independent errors within a cluster: \(\operatorname{Var}(\varepsilon_g\mid X)\) may have nonzero off-diagonal entries. Replacing the unknown errors by OLS residuals gives the CR0 estimator. The large-\(G\) justification is that the aggregate variation in the independent cluster score contributions can be estimated from their observed counterparts, subject to regularity conditions and no cluster dominating the total information.

Practical Choices and a Common Pitfall

Before reporting a clustered DiD result, make the dependence argument explicit. Ask what units receive treatment, which units share shocks that could induce residual dependence, and whether the proposed clusters can reasonably be treated as independent. Then report the number of clusters, the clustering level, the coefficient, and the standard error. If treatment is assigned at the state level, for instance, the number of individual records is not a substitute for the number of states when describing the basis of the cluster-robust approximation.

Common mistake: “There are 5,000 individual observations, so the cluster-robust standard error is based on a large sample.” This reasoning counts rows as if they supplied independent cluster information. If those records are nested in 12 states and the relevant dependence is within state, the sandwich aggregates scores into only 12 contributions. The correction is to report and evaluate the cluster count itself, and not to interpret the number of rows as the number of independent units for clustered inference.

A second mistake is to say that clustering at a different level “changes the DiD effect.” Holding the fitted OLS regression and observations fixed, changing the variance estimator changes uncertainty estimates, not the OLS interaction coefficient. If the point estimate changes, then something else has changed too, such as the weights, sample, or regression specification. Neither a small standard error nor a large cluster count repairs a failure of parallel trends; those are distinct inferential and causal questions.