Why the Interval Must Reflect Clustering
A regression coefficient confidence interval describes uncertainty about a population coefficient. When observations are clustered—such as students within schools or patients within clinics—observations in the same cluster may share unmeasured influences. Treating every observation as independent can therefore understate uncertainty. A cluster-robust confidence interval uses a standard error designed to allow dependence within clusters and then uses a reference distribution that reflects the number of independent clusters.
The point estimate does not change merely because the standard error is cluster-robust. The interval changes because its estimated uncertainty, and often its critical value, differ from those in a conventional ordinary least-squares (OLS) interval. This tutorial focuses on intervals for individual regression coefficients; inference about a combination of coefficients is a separate task.
Model and Conditions for Cluster-Robust Inference
Consider a linear regression model \(Y_{ig}=X_{ig}^{\mathsf T}\beta+u_{ig}\), where \(Y_{ig}\) is an outcome, \(X_{ig}\) is a vector of predictors, \(\beta\) is the coefficient vector, and \(u_{ig}\) is the error. The coefficient \(\beta_j\) describes the model’s association between the outcome and predictor \(j\), holding the other included predictors fixed. Its substantive interpretation depends on the variables, their units, and the model specification.
Cluster-robust inference is appropriate when the observations can be grouped into \(G\) independent clusters, while errors within a cluster may be correlated and may have different variances. The regression model must also be suitably specified for the coefficient being studied, and the errors must have conditional mean zero given the included predictors: \(E(u_{ig}\mid X)=0\). The design matrix must contain enough independent information to estimate the coefficients. As with other regression intervals, the data and model should not be dominated by a small number of influential observations or clusters.
These conditions do not make the number of observations the relevant count for every part of inference. For cluster-robust uncertainty, the number of independent clusters matters because the estimator’s independent contributions come from clusters, not from treating every row as independent. Cluster-robust methods allow within-cluster dependence; they do not repair confounding, an inappropriate regression model, or a poorly chosen clustering level.
The cluster-robust standard error is obtained from a cluster-robust variance estimate, potentially with a small-sample correction. Use the corrected standard error and any degrees of freedom specified by the correction or inference method actually used. Do not substitute a conventional OLS standard error simply because the point estimate came from OLS.
Constructing and Interpreting the Interval
For a 95% interval, \(\alpha=0.05\), so the critical value is \(t_{0.975,\nu}\). With many independent clusters, a normal critical value may be a reasonable large-sample approximation. With a limited number of clusters, a \(t\) reference distribution with cluster-based degrees of freedom is generally more appropriate, and its critical value is larger than the standard normal critical value for the same two-sided confidence level.
A cluster-based \(t\) interval is typically an approximation, not an exact finite-sample guarantee. Cluster-robust variance estimators rely on having enough independent clusters for their sampling approximations to work well. Small-sample corrections and degrees-of-freedom choices address some finite-sample concerns, but cannot create information when there are very few independent clusters.
Why the Formula Has This Form
The interval follows by standardizing the coefficient estimate with its estimated cluster-robust standard error. Under the assumptions and approximation used for cluster-robust inference, the standardized statistic is approximately \(t\)-distributed with \(\nu\) degrees of freedom:
The central \(1-\alpha\) portion of this reference distribution lies between \(-t_{1-\alpha/2,\nu}\) and \(t_{1-\alpha/2,\nu}\). Therefore,
Multiplying through by the positive standard error and rearranging the inequalities gives
This argument explains both parts of the interval: the cluster-robust standard error sets the estimated scale of uncertainty, and the cluster-based reference distribution supplies the critical value. Repeated use of this procedure under the stated conditions would produce intervals covering the target coefficient at approximately the stated rate. It does not mean there is a 95% probability that a particular, already-calculated 95% interval contains a fixed coefficient.
Worked Example: Comparing Cluster-Robust and OLS Intervals
Suppose a hypothetical analysis relates students’ weekly study hours to their exam scores, measured in points, while adjusting for other specified predictors. The data include 120 students in 12 schools. Assume schools are the appropriate independent clusters and that the regression and cluster-robust inference conditions are reasonable. For study hours, the estimated coefficient is \(0.42\) exam points per additional weekly study hour, and the small-sample-adjusted cluster-robust standard error is \(0.16\) points per hour. Use \(\nu=G-1=11\), with \(t_{0.975,11}=2.201\).
The 95% cluster-robust interval is
The interval is approximately 0.068 to 0.772 exam points per additional weekly study hour, conditional on the other included predictors. It excludes zero, but it does not show that study hours cause higher scores; the fitted association may reflect other features of the data or study design.
For comparison, suppose a conventional OLS calculation reports a standard error of \(0.11\) and uses \(118\) residual degrees of freedom, giving \(t_{0.975,118}\approx1.980\). Its interval would be
That interval is narrower, but it relies on conventional OLS uncertainty calculations that do not account for the allowed within-school dependence. The comparison illustrates why the standard error and the critical value both matter. The cluster-robust interval’s exclusion of zero should still be interpreted cautiously with only 12 independent schools.
Worked Example: An Interval That Includes Zero
Consider a hypothetical regression relating a clinic’s average waiting time to a predictor, with waiting time measured in minutes. There are 8 independent clinics, with observations potentially dependent within clinic. Suppose the fitted coefficient is \(-3.1\) minutes per unit increase in the predictor and the corrected cluster-robust standard error is \(1.4\) minutes per unit. Use the conservative degrees of freedom \(\nu=G-1=7\), for which \(t_{0.975,7}=2.365\). Assume the model and cluster definitions are reasonable, while recognizing that eight clusters give limited information for assessing uncertainty.
The margin of error is
Thus the 95% interval is
The interval includes zero, so these data do not provide strong evidence at the two-sided 5% level that the coefficient differs from zero under this model and inference method. This is not evidence that the coefficient is exactly zero. The interval also indicates that the estimate is imprecise: effects in either direction remain compatible with the data and method.
Worked Example: Using a Method-Specific Degrees of Freedom
Suppose a hypothetical study estimates the association between a workplace training measure and an employee performance score, in score points per 10-unit increase in the measure. There are 24 independent workplaces. A specified small-sample procedure reports a corrected cluster-robust standard error of \(0.032\) points per 10-unit increase and a coefficient-specific approximate degrees of freedom of \(\nu=15\). Assume the procedure’s conditions are suitable for these data. The degrees of freedom are below \(G-1=23\), reflecting the method’s estimate of uncertainty for this coefficient.
With \(t_{0.975,15}=2.131\), the margin of error is
If the estimated coefficient is \(0.075\) points per 10-unit increase, its 95% interval is
The interval is positive but close to zero at its lower endpoint. The result suggests a positive adjusted association under this model and inference method; it does not establish a causal effect. In this example, using \(G-1\) automatically would ignore the degrees of freedom supplied by the specified procedure. The appropriate choice is the degrees of freedom that belong to the method used, not a universal value applied without checking.
What Changes Relative to an Ordinary OLS Interval?
| Feature | Cluster-robust interval | Conventional OLS interval |
|---|---|---|
| Point estimate | Usually the same fitted coefficient, \(\hat{\beta}_j\) | The same fitted coefficient when based on the same regression |
| Standard error | Allows dependence within clusters; may include a small-sample correction | Uses the conventional OLS variance assumptions, which generally do not allow within-cluster dependence |
| Reference distribution | Uses cluster-based degrees of freedom or a justified large-sample approximation | Typically uses residual degrees of freedom under conventional assumptions |
| Interpretation | Uncertainty for the coefficient, accounting for the chosen cluster structure | Uncertainty under the conventional OLS variance model |
A wider cluster-robust interval is common when within-cluster dependence is positive, but it is not guaranteed in every data set. Cluster-robust uncertainty can be smaller or larger than the conventional estimate depending on the data and model. The key distinction is not that one interval must always be wider; it is that the two intervals rely on different variance assumptions and often different reference distributions.
Practical Checks and a Common Mistake
Before reporting a coefficient interval, check that the cluster definition follows the study design and that clusters can reasonably be treated as independent. Count the independent clusters, not merely the observations. Confirm that the standard error is cluster-robust at the intended level and note whether a small-sample correction and method-specific degrees of freedom were used. Finally, report the coefficient, interval, units, and relevant model context together.
- Common mistake: Use the cluster-robust standard error but retain a normal critical value automatically, or use the ordinary residual degrees of freedom as though all observations were independent.
- Correction: Use the cluster-based reference distribution and degrees of freedom appropriate to the method. With few clusters, a cluster-based \(t\) critical value is often larger than the normal critical value, and the resulting interval reflects limited independent information.
- Another mistake: Treat a confidence interval that excludes zero as proof of a causal effect.
- Correction: An interval describes uncertainty about a model coefficient. Causal interpretation requires support from the study design and assumptions beyond cluster-robust variance estimation.
If the number of clusters is very small, do not present a corrected interval as if the correction solved the problem completely. The estimate may remain sensitive to individual clusters, and the reference-distribution approximation may be unreliable. Report the number of clusters and the inference method clearly, and interpret the result with appropriate restraint.
Check Your Understanding
Use the interval formula and the context of each problem to check your understanding.
- Why can a regression with many observations still have limited information for cluster-robust inference?
- A study has 15 independent clusters and uses the conservative choice \(\nu=G-1\). What degrees of freedom does that choice give?
- A cluster-robust 95% interval for a coefficient in minutes per treatment-unit includes zero. What can and cannot be concluded at the two-sided 5% level?
- Why is it not enough to calculate an interval with a cluster-robust standard error but an ordinary OLS critical value without justification?
- What additional interpretation is needed before describing a cluster-robust coefficient interval as evidence of a causal effect?