Asymptotics with Many Independent Clusters

Why the Number of Clusters Matters

When observations within a cluster can share shocks, the independent pieces of information are not the individual observations but the clusters. A regression can contain many rows and still have limited independent information if those rows are concentrated in only a few clusters. The central asymptotic question is therefore what happens as the number of independent clusters grows.

The key result is that OLS can be asymptotically normal even when errors are dependent in arbitrary ways within each cluster. The dependence is accommodated by treating each cluster’s entire regression score as one contribution to the estimator. But independence across clusters alone is not enough: no small set of clusters should dominate the sum of scores. A condition on the scores themselves rules out that failure.

Model and Cluster Scores

Let $g=1,\ldots,G$ index independent clusters. Cluster $g$ contains $n_g$ observations, with response vector $y_g$, regressor matrix $X_g$ having $p$ columns, and error vector $u_g$. Consider the linear conditional-mean model

y_g=X_g\beta_0+u_g,\qquad g=1,\ldots,G.

The parameter $\beta_0$ is a fixed $p$-vector, with $p$ fixed as $G$ grows. Assume clusters are independent of one another and that $E[u_g\mid X_g]=0$. There is no requirement that errors within a cluster be independent, have equal variances, or share a particular covariance structure. The conditional-mean assumption rules out systematic error conditional on the cluster’s regressors; it does not rule out within-cluster dependence.

Definition: The cluster score at $\beta_0$ is $s_g=X_g'u_g$. It is the cluster’s contribution to the OLS estimating equation. The total score is $\sum_{g=1}^G s_g$; under the conditional-mean assumption, $E[s_g\mid X_g]=0$.

Write the OLS estimator as $\hat\beta=(\sum_g X_g'X_g)^{-1}\sum_g X_g'y_g$, when the inverse exists. Define the normalized design matrix and score covariance by

Q_G=\frac{1}{G}\sum_{g=1}^G X_g'X_g,\qquad \Omega_G=\frac{1}{G}\sum_{g=1}^G E[s_gs_g'].

A standard many-cluster setup assumes $Q_G$ converges to a finite positive-definite matrix $Q$, and $\Omega_G$ converges to a finite matrix $\Omega$. The first condition prevents the regressors from becoming unidentified or arbitrarily ill-conditioned in the limit. The second ensures that the accumulated score variance has a stable scale. For nondegenerate inference on every linear contrast, the corresponding limiting variance must be positive.

The Conditions Behind the Limit

The central limit theorem must apply to the cluster scores, not merely to individual errors. One useful condition is the vector Lindeberg condition: for every $\epsilon>0$,

\frac{1}{G}\sum_{g=1}^G E\!\left[\|s_g\|^2\, \mathbf{1}\{\|s_g\|>\epsilon\sqrt{G}\}\right]\longrightarrow 0.

Here $\|\cdot\|$ is the Euclidean norm and $\mathbf{1}\{\cdot\}$ is an indicator. The condition says that the portion of total score variance contributed by unusually large cluster scores vanishes. A convenient stronger condition is a uniform bound $E[\|s_g\|^{2+\delta}]\leq C$ for some $\delta>0$ and finite constant $C$. That bound directly controls the score tails and implies the Lindeberg condition when the normalized score variance is bounded.

Theorem: Suppose clusters are independent, $E[s_g]=0$, $Q_G\to Q$ with $Q$ positive definite, $\Omega_G\to\Omega$, and the cluster-score Lindeberg condition holds. Then $$ \sqrt{G}(\hat\beta-\beta_0)\ \xrightarrow{d}\ N\!\left(0,Q^{-1}\Omega Q^{-1}\right). $$ This result permits arbitrary within-cluster error dependence; it requires the stated conditions across the sequence of clusters.

The representation behind the result also shows why the cluster score is the relevant object. From the regression model,

\hat\beta-\beta_0 =\left(\sum_{g=1}^G X_g'X_g\right)^{-1}\sum_{g=1}^G s_g =Q_G^{-1}\left(\frac{1}{G}\sum_{g=1}^G s_g\right).

The independent-cluster central limit theorem gives $G^{-1/2}\sum_g s_g\xrightarrow{d}N(0,\Omega)$, under the covariance limit and Lindeberg condition. Since $Q_G^{-1}\to Q^{-1}$, multiplying the representation by $\sqrt G$ and applying the continuous mapping theorem yields the displayed limit. Also, the score sum divided by $G$ converges to zero in probability because its variance is of order $1/G$. Thus $\hat\beta\to\beta_0$: OLS is consistent under these conditions.

Why the Cluster-Robust Covariance Estimates This Limit

The cluster-robust covariance estimator, often called CR0, uses each cluster’s fitted residual vector $\hat u_g=y_g-X_g\hat\beta$:

\widehat V_{\mathrm{CR0}} = \left(\sum_{g=1}^G X_g'X_g\right)^{-1} \left(\sum_{g=1}^G X_g'\hat u_g\hat u_g'X_g\right) \left(\sum_{g=1}^G X_g'X_g\right)^{-1}.

The middle term is the sum of outer products of fitted cluster scores, $X_g'\hat u_g$. Unlike a formula that adds only observation-level squared residuals, this outer product includes cross-products between observations in the same cluster. Those terms allow the within-cluster covariance to be unrestricted.

For a simple sufficient setting for consistency of this covariance estimator, suppose cluster sizes and regressor magnitudes are uniformly bounded, the design limit above holds, and cluster scores have uniformly bounded moments of order $2+\delta$. These conditions give a law of large numbers for the score outer products and control the effect of replacing errors by fitted residuals. Indeed, with $A_g=X_g'X_g$, the fitted score is $X_g'\hat u_g=s_g-A_g(\hat\beta-\beta_0)$. Bounded $A_g$, consistency of $\hat\beta$, and bounded average score size make the average difference between fitted-score and true-score outer products vanish. Consequently,

G\widehat V_{\mathrm{CR0}} \ \xrightarrow{p}\ Q^{-1}\Omega Q^{-1}.

These are sufficient conditions, not a claim that bounded cluster sizes are necessary. In settings with growing or unbalanced clusters, the design, score-tail, and covariance conditions must be checked in a form appropriate to that sequence. The important point is that marginal moment bounds on regressors and errors, taken separately, do not automatically establish the needed control of their product $X_g'u_g$.

Worked Example: Computing a Cluster-Robust Slope Standard Error

Consider a hypothetical regression with an intercept and one regressor. There are three independent clusters, each containing two observations with the same regressor value. The values are $x=0,1,2$ across the clusters. Within each cluster the two responses are identical: $13/6$, $14/3$, and $49/6$, respectively. These data are an arithmetic illustration of the estimator, not a basis for reliable large-sample inference with only three clusters.

The fitted line is $\hat y=2+3x$. The residual in each of the two observations in the three clusters is $1/6,-1/3,1/6$, respectively. The residuals sum to zero, and their regressor-weighted sum is $2(0)(1/6)+2(1)(-1/3)+2(2)(1/6)=0$, so these coefficients satisfy the OLS normal equations.

For a cluster with repeated regressor value $x_g$ and common residual $r_g$, the score is $(2r_g,2x_gr_g)'$. The three cluster scores are therefore $(1/3,0)'$, $(-2/3,-2/3)'$, and $(1/3,2/3)'$. Their outer products sum to

\sum_g X_g'\hat u_g\hat u_g'X_g = \begin{pmatrix} 2/3&2/3\\ 2/3&8/9 \end{pmatrix}.

The design cross-product is $\sum_gX_g'X_g=\begin{pmatrix}6&6\\6&10\end{pmatrix}$, whose inverse is $\begin{pmatrix}5/12&-1/4\\-1/4&1/4\end{pmatrix}$. The slope variance estimate is the quadratic form of the second row of this inverse with the score outer-product sum:

\widehat{\operatorname{Var}}_{\mathrm{CR0}}(\hat\beta_1) =\frac{1}{16}\left(\frac{2}{3}-2\frac{2}{3}+\frac{8}{9}\right) =\frac{1}{16}\left(\frac{2}{9}\right) =\frac{1}{72}\approx0.01389.

Thus the CR0 slope standard error is $\sqrt{1/72}\approx0.1179$, in units of response per unit of $x$. The calculation shows how cluster scores enter the sandwich. With $G=3$, however, a many-cluster normal approximation should not be presumed accurate.

A Score-Tail Pitfall

Worked Example: Marginal Moments Do Not Guarantee a Cluster CLT

For each $G$, imagine $G$ independent clusters with one observation each. In a hypothetical triangular array, let $(X_g,u_g)=(1,\pm1)$ with total probability $1-1/G$, and $(X_g,u_g)=(G^{1/4},\pm G^{1/4})$ with total probability $1/G$; within either case, each sign has equal probability. The score $s_g=X_gu_g$ is $+1$ or $-1$ in the first case and $+\sqrt G$ or $-\sqrt G$ in the rare case. Its mean is zero.

The design and score variance limits appear well behaved:

E[X_g^2]=1-\frac{1}{G}+\frac{1}{\sqrt G}\longrightarrow1,\qquad E[s_g^2]=1-\frac{1}{G}+1\longrightarrow2.

Yet for any $0<\epsilon<1$, the rare score has magnitude $\sqrt G>\epsilon\sqrt G$. Thus the Lindeberg expression is

\frac{1}{G}\sum_{g=1}^G E[s_g^2\mathbf{1}\{|s_g|>\epsilon\sqrt G\}] = \frac{1}{G}\,G\left(\frac{1}{G}\cdot G\right)=1,

not zero. The marginal moments of $X_g$ and $u_g$ of order $3.5$ are uniformly bounded: each rare contribution is $G^{-1}G^{7/8}=G^{-1/8}$. But the score’s moment of order $2+\delta$ is not uniformly bounded for any $\delta>0$. This example shows why conditions must control the score (or sufficiently strong joint moments that imply score control), rather than relying on separate marginal moment bounds.

Large-Sample Inference in a Hypothetical Regression

Worked Example: A Cluster-Robust Confidence Interval

Suppose a hypothetical study has $G=400$ independent clusters and a regression with an intercept and a slope. Its normalized design matrix is $Q_G=\begin{pmatrix}1&0\\0&1\end{pmatrix}$, and the estimated normalized score outer-product matrix is $\widehat\Omega=\begin{pmatrix}2.5&1.5\\1.5&2.5\end{pmatrix}$. Suppose the estimated slope is $0.300$, measured in outcome units per unit of the regressor. Assume the conditional-mean, independence, design-limit, and score-tail conditions are credible, and that the sample score outer products estimate their limiting covariance adequately.

Since $Q_G^{-1}$ is the identity here, the estimated asymptotic slope variance is the slope diagonal entry divided by $G$:

\widehat{\operatorname{Var}}(\hat\beta_1)=\frac{2.5}{400}=0.00625,\qquad SE(\hat\beta_1)=\sqrt{0.00625}\approx0.07906.

For the two-sided test $H_0:\beta_1=0$ against $H_a:\beta_1\ne0$, the large-sample statistic is $z=0.300/0.07906\approx3.795$. Its two-sided standard-normal p-value is approximately $0.00015$. A 95% large-sample confidence interval is

0.300\pm1.96(0.07906) =[0.1450,\ 0.4550].

Under the stated asymptotic conditions, the data provide evidence against a zero slope. The interval describes plausible values for the slope in outcome units per regressor unit under the model and approximation; it is not a statement that the fixed parameter has a 95% probability of lying in this particular interval.

What the Result Does—and Does Not—Say

Many-cluster asymptotics justify normal approximations when the number of independent clusters grows and the stated design and score conditions hold. They do not say that a large observation count automatically guarantees accurate inference: the number of clusters, the spread of their scores, and the validity of the conditional-mean model all matter. Nor does the result impose a particular within-cluster correlation pattern; that flexibility is precisely why the cluster score aggregates all observations in a cluster before its outer product is formed.

Common mistake: “The clusters are independent, so the cluster-robust estimator is automatically asymptotically normal.” Independence is necessary for this argument, but it is not sufficient. A small fraction of clusters can carry a nonvanishing share of score variance, as the rare-event example demonstrates. The correction is to verify a score-level Lindeberg condition or a suitable stronger score-moment condition, alongside the design and covariance limits.

Key takeaway: With many independent clusters, OLS has a normal limit when its normalized design stabilizes and its cluster scores satisfy a central-limit condition. The cluster-robust sandwich estimates that limit by preserving each cluster’s complete score contribution.

Check Your Understanding

Use the model and conditions in this tutorial to answer the following.

Why can errors be arbitrarily dependent within a cluster while the many-cluster central limit argument still apply?
State in words what the cluster-score Lindeberg condition rules out.
In the consistency argument, what does convergence of $Q_G$ to a positive-definite matrix ensure?
Why do separate moment bounds on regressors and errors fail to guarantee a suitable score-tail condition?
If $G\widehat V_{\mathrm{CR0}}$ estimates the limiting covariance of $\sqrt G(\hat\beta-\beta_0)$, how does the estimated variance of a coefficient scale with $G$?