Influence Diagnostics for Clustered Regression

Why Cluster Influence Requires Its Own Diagnostics

In clustered regression, observations within the same group may share shocks, while different clusters are treated as independent. A school, clinic, firm, or geographic area can therefore contribute many observations but still represent one independent cluster. If one cluster has unusual covariate values, a large size, or outcomes that depart systematically from the fitted relationship, it may substantially affect the regression.

A useful diagnostic asks what the fitted model would look like if one entire cluster were omitted. This is not the same as deleting one observation: it tests sensitivity to removing a whole independent unit. The resulting coefficient changes and changes in cluster-robust standard errors describe influence, not whether a cluster is erroneous or whether a causal conclusion is justified.

Definition: Let \(G\) denote the number of clusters. For cluster \(g\), the leave-one-cluster-out fit is the regression estimated after removing all observations in that cluster. Write its coefficient vector as \(\hat{\beta}_{(-g)}\). The coefficient deletion change is \(\Delta_g=\hat{\beta}_{(-g)}-\hat{\beta}\), where \(\hat{\beta}\) is the full-sample OLS estimate. A large change in one or more components indicates that the full-sample estimate is sensitive to that cluster.

Model and Exact Deletion Formula

For \(g=1,\ldots,G\), let \(y_g\) be the outcome vector and \(X_g\) the design matrix for the observations in cluster \(g\). Stack these into \(y\) and \(X\), and consider the linear conditional-mean model \(y_g=X_g\beta+u_g\). The errors may be dependent within a cluster; the cluster-robust covariance estimator allows this dependence without specifying its exact form. The basic setup assumes the clusters are independent, the conditional mean is correctly specified for the target regression, and \(X\) has full column rank.

A deletion fit also requires that the retained design matrix \(X_{(-g)}\) has full column rank. If deleting a cluster removes the only variation that identifies a coefficient, the deletion estimate is not defined; report this rather than treating the coefficient as zero. With \(e_g=y_g-X_g\hat{\beta}\) denoting the full-fit residual vector for cluster \(g\), the exact OLS identity is:

\hat{\beta}_{(-g)}-\hat{\beta} =-(X_{(-g)}'X_{(-g)})^{-1}X_g'e_g.

This expression uses residuals from the full fit, but the matrix multiplying the cluster score reflects the design after deletion. The identity follows directly from the normal equations, rather than from an approximation.

Write the retained-sample normal equation.
Because \(\hat{\beta}_{(-g)}\) is the OLS estimate on the retained observations, \(X_{(-g)}'(y_{(-g)}-X_{(-g)}\hat{\beta}_{(-g)})=0\).

Evaluate the retained score at the full estimate.
The full-sample normal equation is \(X'e=0\), so \(X_{(-g)}'e_{(-g)}=-X_g'e_g\).

Subtract the two estimates.
Since \(y_{(-g)}-X_{(-g)}\hat{\beta}=e_{(-g)}\), the retained normal equation gives \(X_{(-g)}'X_{(-g)}(\hat{\beta}_{(-g)}-\hat{\beta})=X_{(-g)}'e_{(-g)}=-X_g'e_g\). Inverting the retained cross-product matrix yields the stated identity.

The vector \(X_g'e_g\) is the cluster’s contribution to the full-sample OLS score. A large score can matter, but the retained design matrix also matters: a deletion can have a large effect when the remaining data provide little information in a coefficient’s direction. Thus neither a large residual nor a large cluster size alone is a complete influence diagnostic.

Worked Example: A Large Cluster With a Modest Residual

Consider an intercept-only regression with four clusters. Their sizes are \(2,2,2,8\), and their cluster outcome means are \(8,11,9,10.5\), respectively. The numbers are hypothetical. Compare deletion of the first cluster with deletion of the largest cluster.

The full-sample mean is the size-weighted mean:

\hat{\beta} =\frac{2(8)+2(11)+2(9)+8(10.5)}{14} =\frac{140}{14}=10.

The first cluster has mean residual \(8-10=-2\), while the largest cluster has mean residual \(10.5-10=0.5\). Their cluster scores, the sums of their full-fit residuals, are \(2(-2)=-4\) and \(8(0.5)=4\). Despite the smaller residual magnitude, deleting the large cluster changes the estimate more:

\hat{\beta}_{(-4)} =\frac{2(8)+2(11)+2(9)}{6} =\frac{56}{6}=9.333, \qquad \Delta_4=9.333-10=-0.667.

By contrast, deleting the first cluster gives \(\hat{\beta}_{(-1)}=[2(11)+2(9)+8(10.5)]/12=124/12=10.333\), so \(\Delta_1=0.333\). The large cluster has greater influence here even though its mean residual is much smaller. This intercept-only example isolates the role of cluster size; with other regressors, covariate leverage also matters.

Measuring Coefficient and Standard-Error Changes

Inspect the components of \(\Delta_g\) in the units of their coefficients. For example, a deletion shift of 0.8 in a coefficient measured in outcome units per year has the same units as that coefficient. For comparisons across coefficients with different scales, a standardized shift can be useful. One descriptive option divides each component by its full-sample standard error; it reports the change in units of the corresponding estimated standard error. This standardization does not create a universal threshold for declaring a cluster influential.

For standard-error diagnostics, refit the regression without cluster \(g\), then recompute the same chosen cluster-robust covariance estimator on the retained sample. Compare the resulting standard error for a coefficient with its full-sample standard error. The standard error may increase or decrease: deletion changes both the residual scores and the design matrix. Comparisons should use the same covariance convention and finite-sample correction in both fits. A deletion-based standard error with very few retained clusters can itself be unstable and should not be treated as a reliable replacement for the full analysis.

Worked Example: Coefficient and Standard-Error Sensitivity

Suppose five hypothetical clusters each contain two observations. Within each cluster the regressor \(x\) is constant, taking values \(0,1,2,3,4\); the cluster outcome means are \(1,2,3,5,10\). To keep the cluster means explicit, the two outcomes in each cluster can be taken as its mean minus 0.5 and its mean plus 0.5. Fit an intercept and slope by OLS.

The sample has \(X'X=\begin{pmatrix}10&20\\20&60\end{pmatrix}\), with inverse \(\begin{pmatrix}0.3&-0.1\\-0.1&0.05\end{pmatrix}\). The weighted outcome sum is 42 and the weighted \(x\)-outcome sum is 126, so the fitted coefficients are \(\hat{\beta}_0=0\) and \(\hat{\beta}_1=2.1\). The cluster-mean residuals are \(1,-0.1,-1.2,-1.3,1.6\). The last cluster is at \(x=4\), where the residual is 1.6.

Deleting that cluster leaves means \(1,2,3,5\) at \(x=0,1,2,3\). Their mean \(x\) is 1.5 and mean outcome is 2.75. The centered cross-product is \((-1.5)(1)+(-0.5)(2)+(0.5)(3)+(1.5)(5)=6.5\); the centered sum of squares for \(x\) is 5. Thus the deleted slope is \(6.5/5=1.3\), and the deleted intercept is \(2.75-1.3(1.5)=0.8\). The coefficient changes are \(\Delta_0=0.8\) and \(\Delta_1=1.3-2.1=-0.8\).

For a standard-error comparison, use the uncorrected cluster sandwich estimator \(V=(X'X)^{-1}[\sum_g(X_g'e_g)(X_g'e_g)'](X'X)^{-1}\), recomputed after deletion. The full-fit slope variance is the sum of squared cluster contributions to the slope, \((-0.2)^2+(0.01)^2+0^2+(-0.13)^2+(0.32)^2=0.1594\), so its standard error is \(\sqrt{0.1594}=0.3992\). The two observations within a cluster have opposite deviations of 0.5 from their mean; because \(x\) is constant within that cluster, these deviations cancel in \(X_g'e_g\).

After deleting the last cluster, the retained cross-product inverse is \(\begin{pmatrix}0.35&-0.15\\-0.15&0.1\end{pmatrix}\), and the retained cluster-mean residuals are \(0.2,-0.1,-0.4,0.3\). The slope contributions are \(-0.06,0.01,-0.04,0.09\). Therefore the deleted-fit slope variance is \(0.0036+0.0001+0.0016+0.0081=0.0134\), and its standard error is \(\sqrt{0.0134}=0.1158\). Both the slope estimate and its estimated standard error change substantially in this illustration. The standard errors use no finite-sample scaling correction, and the four-cluster deleted fit is too small to support strong inferential claims.

Leverage, Unequal Sizes, and a Practical Diagnostic

For the full OLS fit, the cluster leverage block is \(H_{gg}=X_g(X'X)^{-1}X_g'\). Its trace summarizes the cluster’s share of the fitted-design leverage; high leverage means its covariate pattern is unusual or heavily represented. Leverage is a property of the design, not of the outcomes. It flags clusters whose deletion may matter, but it does not establish realized influence: the residual pattern must also be considered. There is no universal cluster-leverage cutoff that identifies influential clusters in every design.

Worked Example: Unequal Sizes and Covariate Leverage

Consider a hypothetical intercept-and-slope regression with clusters at \(x=-1,0,1,2\) and sizes \(2,2,2,8\). For a cluster whose \(x\) is constant, its leverage trace is \(n_g/N+n_g(x_g-\bar{x})^2/S_{xx}\), where \(N\) is the total number of observations and \(S_{xx}=\sum_i(x_i-\bar{x})^2\).

Here \(N=14\), \(\sum_i x_i=16\), and \(\bar{x}=16/14=8/7\). Also \(\sum_i x_i^2=36\), so \(S_{xx}=36-16^2/14=124/7\). For the size-eight cluster at \(x=2\), its leverage trace is \(8/14+8(2-8/7)^2/(124/7)=0.9032\). For the size-two cluster at \(x=-1\), it is \(2/14+2(-1-8/7)^2/(124/7)=0.6613\). The two other traces are \(0.2903\) at \(x=0\) and \(0.1452\) at \(x=1\); they sum with the first two to 2, the number of regression coefficients.

The largest cluster has the highest leverage, but the small cluster at \(x=-1\) also has substantial leverage because it lies far from the weighted covariate mean. These values identify clusters worth checking with deletion fits; without outcomes and residuals, they do not tell us which cluster actually changes the estimated coefficients most.

A practical assessment combines several pieces of evidence: the coefficient deletion vector, the standardized shifts when scales differ, changes in the chosen cluster-robust standard errors, cluster size, and leverage. A multivariate summary is possible—for example, the quadratic distance \(\Delta_g'V^{-1}\Delta_g\) when the full-sample covariance \(V\) is nonsingular—but its size depends on the setting and is not a universal decision rule. Report the component changes and their context rather than relying on one score.

Common Pitfalls and Interpretation

Deleting observations instead of clusters. This misses the sensitivity to removing a whole independent unit. The diagnostic should remove all rows belonging to the cluster.
Equating a large residual with influence. A residual does not account for cluster size, covariate leverage, or the information left after deletion. Examine the actual refit and coefficient changes.
Assuming standard errors must rise when a cluster is deleted. They can rise or fall because both residual contributions and the retained design change. Recompute the same covariance estimator on each sample.
Treating a deletion result as a test or correction. Influence diagnostics describe sensitivity. They do not show that the cluster is invalid, justify excluding it, or repair a misspecified model.
Using a cutoff without context. Unequal sizes, leverage, the number of clusters, and the coefficient of interest all affect interpretation. Compare diagnostics across clusters and explain the substantive size of any shift.

Key takeaway: Cluster influence is a leave-one-cluster-out sensitivity question. Use coefficient changes and consistently recomputed standard errors, while interpreting them alongside cluster size and leverage. A diagnostic flags dependence of the fitted result on a cluster; it does not by itself determine the correct model or the validity of an inferential claim.

Check Your Understanding

Use the definitions and calculations above to reason through these questions.

What condition on the retained design matrix is needed for a leave-one-cluster-out coefficient vector to exist?
In the intercept-only example, why did deleting the size-eight cluster shift the estimate more than deleting the size-two cluster, despite its smaller mean residual?
In the five-cluster regression, what are the slope deletion change and the full-fit slope standard error after deleting the cluster at \(x=4\)?
Why can a cluster have high leverage without being influential in the realized regression fit?
What must be held consistent when comparing cluster-robust standard errors before and after deletion, and why can neither standard error be assumed to be larger?