Intracluster Correlation and Design Effects

Why Clustering Changes Precision

In clustered data, observations from the same clinic, school, workplace, or community may share conditions that make their errors more alike than errors from different clusters. A sample can therefore contain many observations without containing as much independent information as its raw size suggests. The intracluster correlation describes this within-cluster similarity, while the design effect summarizes its consequence for variance in a specific setting.

The derivation in this tutorial concerns the variance of an equally weighted overall mean when clusters have the same size and errors have an exchangeable correlation structure. Under those assumptions, the design effect has a simple expression. It is a useful measure of how clustering changes precision, but it is not a universal multiplier for every regression standard error.

Definition: Suppose there are $G$ clusters, each with $m$ observations, so the total sample size is $N=Gm$. Let $u_{gi}$ be the error for observation $i$ in cluster $g$. Under an exchangeable error model, errors have mean zero, variance $\sigma^2$, correlation $\rho$ for two distinct observations in the same cluster, and zero covariance across different clusters. The intracluster correlation coefficient (ICC) is this within-cluster correlation, $\rho$. For the overall sample mean, the design effect is the ratio of its variance under this clustered model to its variance under independent errors with the same marginal variance. The corresponding effective sample size is the independent-observation sample size that would give the same variance.

Deriving the Design Effect

Consider the intercept-only model $Y_{gi}=\mu+u_{gi}$, where $\mu$ is the common mean of interest. Assume clusters are independent of one another, all observations have the same error variance $\sigma^2$, and every pair of distinct errors within a cluster has correlation $\rho$. The estimator is the equally weighted mean across all $N$ observations:

\bar{Y}=\frac{1}{Gm}\sum_{g=1}^{G}\sum_{i=1}^{m}Y_{gi}.

Since the errors have mean zero, $\bar{Y}$ is unbiased for $\mu$. To calculate its variance, sum the variances and covariances of all errors in the mean. There are $Gm$ individual variance terms, each equal to $\sigma^2$. Within each cluster there are $m(m-1)$ ordered pairs of distinct observations, each contributing covariance $\rho\sigma^2$. There are no cross-cluster covariance terms.

\begin{aligned} \operatorname{Var}(\bar{Y}) &=\frac{1}{(Gm)^2} \left[Gm\sigma^2+Gm(m-1)\rho\sigma^2\right]\\ &=\frac{\sigma^2}{Gm}\left[1+(m-1)\rho\right]\\ &=\frac{\sigma^2}{N}\left[1+(m-1)\rho\right]. \end{aligned}

Under independent errors with variance $\sigma^2$, the variance of the same mean would be $\sigma^2/N$. Dividing the clustered variance by that independent-error variance gives the result:

Theorem: For an equally weighted mean from $G$ independent, equally sized clusters with exchangeable within-cluster error correlation $\rho$, the design effect is $$ \operatorname{DEFF}=1+(m-1)\rho. $$ Thus $\operatorname{Var}(\bar{Y})=(\sigma^2/N)\operatorname{DEFF}$, and the variance-equivalent effective sample size is $$ N_{\mathrm{eff}}=\frac{N}{\operatorname{DEFF}}. $$ The standard error is multiplied by $\sqrt{\operatorname{DEFF}}$, not by $\operatorname{DEFF}$.

The distinction between variance and standard error matters. A design effect of 1.69 means the variance is 1.69 times the independent-error variance, while the standard error is $\sqrt{1.69}=1.30$ times as large. The effective sample size is a variance-equivalence calculation, not a claim that the data literally contain that many observations or independent clusters.

Worked Examples

Worked Example: Calculating the Effective Sample Size

Suppose a hypothetical survey samples 20 clinics, with 10 patients per clinic. For a patient-level outcome, assume the marginal error standard deviation is 12 outcome units and the within-clinic ICC is $\rho=0.08$. Under the exchangeable model, calculate the design effect, effective sample size, and standard error of the overall mean.

There are $G=20$, $m=10$, and $N=20(10)=200$ patients. The design effect is

\operatorname{DEFF}=1+(m-1)\rho=1+(10-1)(0.08)=1.72.

The effective sample size is

N_{\mathrm{eff}}=\frac{200}{1.72}=116.3.

For comparison, treating the 200 observations as independent would give variance $12^2/200=144/200=0.7200$ outcome-units squared and standard error $\sqrt{0.7200}=0.8485$ outcome units. Accounting for clustering gives

\operatorname{Var}(\bar{Y})=0.7200(1.72)=1.2384, \qquad SE(\bar{Y})=\sqrt{1.2384}=1.113.

Equivalently, the standard error multiplier is $\sqrt{1.72}=1.311$. Under the stated assumptions, the mean from these 200 patients has the same variance as a mean based on about 116 independent observations with the same marginal variance. This comparison describes precision; it does not alter the number of patients actually observed.

Worked Example: Same Total Sample, Different Cluster Sizes

Compare two hypothetical designs that each observe 240 people, with an assumed ICC of $\rho=0.10$. Design A has 60 clusters of 4 people; Design B has 20 clusters of 12 people. Find each design effect and effective sample size, and compare the standard errors for the overall mean.

For Design A, $m=4$, so

\operatorname{DEFF}_A=1+(4-1)(0.10)=1.30, \qquad N_{\mathrm{eff},A}=\frac{240}{1.30}=184.6.

For Design B, $m=12$, so

\operatorname{DEFF}_B=1+(12-1)(0.10)=2.10, \qquad N_{\mathrm{eff},B}=\frac{240}{2.10}=114.3.

The standard-error multipliers relative to independent observations are $\sqrt{1.30}=1.140$ for Design A and $\sqrt{2.10}=1.449$ for Design B. If the marginal standard deviation is $\sigma$, their mean standard errors are respectively $1.140\sigma/\sqrt{240}$ and $1.449\sigma/\sqrt{240}$. The ratio is $1.449/1.140=1.271$: under these assumptions, Design B has a standard error about 27.1% larger than Design A despite having the same total number of observations. Its observations are grouped into fewer, larger clusters, so more observations share correlated errors.

Worked Example: Planning for a Target Effective Sample Size

A hypothetical study plans equal-sized clusters of 8 observations and expects an ICC of $\rho=0.05$. How many observations and clusters are needed to obtain an effective sample size of at least 100, using the design-effect calculation?

First calculate the design effect:

\operatorname{DEFF}=1+(8-1)(0.05)=1.35.

Since $N_{\mathrm{eff}}=N/1.35$, reaching an effective sample size of 100 requires $N/1.35\geq100$, or $N\geq135$. With clusters of 8, the number of clusters must be an integer. Sixteen clusters provide $N=128$ and $N_{\mathrm{eff}}=128/1.35=94.81$, which is below the target. Seventeen clusters provide $N=136$ and

N_{\mathrm{eff}}=\frac{136}{1.35}=100.7.

Thus 17 clusters, with 8 observations each, meet the variance-equivalent target under the assumed ICC and model. This calculation is conditional on those assumptions; it does not account for other features of a regression analysis or uncertainty about the ICC.

How ICC and Cluster Size Affect Precision

For nonnegative $\rho$, the design effect is at least 1, so clustering does not improve the precision of the equally weighted mean relative to independent errors with the same marginal variance. If $\rho=0$, then $\operatorname{DEFF}=1$, and the formula reduces to the independent-observation variance. If $\rho>0$, each additional observation in an existing cluster adds less information than an independent observation would, because it shares some of its error with the other observations in that cluster.

Holding $\rho>0$ fixed, $\operatorname{DEFF}=1+(m-1)\rho$ increases with cluster size. Its effective sample size can also be written in terms of the number of clusters:

N_{\mathrm{eff}}=\frac{Gm}{1+(m-1)\rho}.

As $m$ grows, the effective sample size per cluster, $m/[1+(m-1)\rho]$, approaches $1/\rho$ when $\rho>0$. This illustrates why increasing the number of observations within a fixed set of clusters may yield diminishing precision gains. Increasing the number of independent clusters can add information in a different way, because it adds new independent groups.

An ICC is a correlation, so it is not mathematically required to be positive. For an exchangeable correlation matrix of size $m$, its allowable range is $-1/(m-1)\leq\rho\leq1$. Negative values can produce a design effect below 1 in this mean-variance calculation. In many applications of clustered errors, however, the substantive concern is positive within-cluster dependence, which inflates variance.

Scope and a Common Mistake

The formula was derived for an equally weighted mean with equal cluster sizes, a common marginal variance, and a single exchangeable within-cluster correlation. It is not automatically the variance multiplier for a regression slope or any other coefficient. In regression, the design matrix, variation in regressors within and between clusters, and the structure of the coefficient’s cluster-level contributions all affect the variance. A single ICC and this design effect can therefore be a useful illustration or approximation, but a coefficient-specific variance calculation requires the relevant regression model and covariance structure.

A related limitation is that the effective sample size here is specific to estimating this mean under the stated covariance assumptions. It should not be substituted for $N$ in every part of a regression analysis, nor treated as the number of independent clusters. Cluster-robust inference concerns dependence across observations within clusters; the design-effect calculation helps explain one way that dependence changes precision.

Mistake: “The design effect is 1.72, so the standard error is 1.72 times larger.”
Correction: The variance is multiplied by 1.72; the standard error is multiplied by $\sqrt{1.72}$.
Mistake: “There are 200 patients, so the effective sample size is also 200.”
Correction: With positive ICC and more than one observation per cluster, the mean’s variance-equivalent effective sample size is smaller than $N$.
Mistake: “The same design effect applies to every coefficient in a clustered regression.”
Correction: The derived result is for the equally weighted mean under specified assumptions. Regression coefficients can have different variance consequences from clustering.

Key takeaway: Under equal cluster sizes and exchangeable within-cluster correlation, the variance of the overall mean is multiplied by $1+(m-1)\rho$. The corresponding effective sample size is $N/[1+(m-1)\rho]$, and the standard-error multiplier is the square root of the design effect. Interpret these quantities for the mean and assumptions used in their derivation.

Check Your Understanding

Use the equal-cluster, exchangeable model unless a question says otherwise.

There are 15 clusters of 6 observations, and the ICC is 0.08. Calculate the design effect and variance-equivalent effective sample size.
If a design effect is 2.25, by what factor is the variance multiplied? By what factor is the standard error multiplied?
With the same total sample size and positive ICC, why does a design with larger clusters generally have a larger design effect?
State two assumptions used to derive the equal-cluster design effect for the overall mean.
Why is it generally incorrect to apply this mean design effect mechanically to every regression coefficient?