Intraclass Correlation and Effective Sample Size

Why Cell Counts Can Overstate Information

Single-cell experiments can measure thousands of cells from a donor, but cells from that donor share biological characteristics. If cells from the same donor tend to have similar expression values, treating every cell as an independent replicate makes the data appear more informative than they are. The intraclass correlation coefficient (ICC) describes this within-donor similarity; the design effect translates it into an approximate variance inflation and an effective number of cell measurements.

These quantities clarify the information contributed by clustered cell observations. They do not change the experimental unit: for a condition assigned to donors, the donors—not their cells—are the independent units for estimating a between-donor condition effect. The effective sample size is a variance-based summary for a specified cell-level mean, not a replacement count of biological replicates.

ICC Under a Random-Intercept Model

Definition: For a continuous cell-level outcome $Y_{ij}$, where $i$ indexes donors and $j$ indexes cells within donor $i$, a random-intercept model is $$ Y_{ij}=\mu+u_i+\varepsilon_{ij}. $$ Here $u_i$ is the donor-specific random effect, with mean 0 and variance $\sigma_u^2$, and $\varepsilon_{ij}$ is the cell-level residual, with mean 0 and variance $\sigma_\varepsilon^2$. Assume donor effects are independent across donors; residuals are independent across cells and donors and independent of the donor effects. The intraclass correlation is $$ \rho=\frac{\sigma_u^2}{\sigma_u^2+\sigma_\varepsilon^2}. $$ It is the correlation between two distinct cell outcomes from the same donor under this model.

The marginal variance of one cell outcome is $\sigma^2=\sigma_u^2+\sigma_\varepsilon^2$. Two cells from the same donor share $u_i$, so their covariance is $\sigma_u^2$; consequently their correlation is $\sigma_u^2/\sigma^2=\rho$. Under the stated variance-component model, $0\leq\rho\leq1$. The ICC is outcome- and model-dependent: it can change with the gene, cell type, normalization or transformation, and covariates included in the model.

In a condition comparison, the model may also include a condition effect and relevant covariates. The ICC used for planning should describe residual within-donor similarity for the outcome and analysis of interest. A single ICC should not automatically be applied to every gene or cell population.

Worked Example: Interpreting a Variance-Component ICC

Suppose a hypothetical normalized expression outcome has estimated donor variance $\sigma_u^2=0.27$ and residual variance $\sigma_\varepsilon^2=0.63$. Under the random-intercept assumptions, calculate the ICC and interpret it.

The total marginal variance is $0.27+0.63=0.90$. Therefore, $$ \rho=\frac{0.27}{0.27+0.63}=\frac{0.27}{0.90}=0.300. $$

The estimated ICC is 0.300, so the model attributes 30.0% of the marginal variance to differences among donors. The model-implied correlation between two distinct cells from the same donor is 0.300. This does not mean 30.0% of cells are alike, nor does it imply that all genes or cell types have this correlation.

Design Effect and Effective Sample Size

Suppose there are $D$ donors, each contributing $m$ cells, for $N=Dm$ cell measurements. The pooled cell mean gives every cell equal weight. Its variance depends on both the marginal cell variance and the within-donor covariance. If all cells were independent with the same marginal variance, the variance would be $\sigma^2/N$. Clustering increases this variance by the design effect.

Definition: With equal cluster size $m$ and common within-donor ICC $\rho$, the design effect for the pooled cell mean is $$ \operatorname{DE}=1+(m-1)\rho. $$ The corresponding effective sample size is $$ N_{\mathrm{eff}}=\frac{N}{\operatorname{DE}}=\frac{Dm}{1+(m-1)\rho}. $$ It is the number of independent observations, each with marginal variance $\sigma^2$, that would give the same variance for this pooled mean under the model.

A short derivation makes the dependence explicit. Write the pooled mean as $\bar{Y}=N^{-1}\sum_{i=1}^{D}\sum_{j=1}^{m}Y_{ij}$. There are $N$ variance terms, each $\sigma^2$, and $D m(m-1)$ ordered covariance terms, each $\rho\sigma^2$. Thus, $$ \operatorname{Var}(\bar{Y})=\frac{1}{N^2}\left[N\sigma^2+D m(m-1)\rho\sigma^2\right] =\frac{\sigma^2}{N}\left[1+(m-1)\rho\right]. $$

The multiplier is one when $\rho=0$, and it increases with either $m$ or $\rho$. If $\rho=1$, cells within a donor carry no independent cell-level variation under this model, and $N_{\mathrm{eff}}=D$. For intermediate ICC values, additional cells still help estimate a donor’s mean, but each additional cell adds less information than an independent observation.

Worked Example: Equal Cells per Donor

A hypothetical study measures 40 cells from each of 15 donors, giving $N=15(40)=600$ cells. Suppose the relevant outcome has ICC $\rho=0.12$. Calculate the design effect and effective sample size.

With $m=40$, $$ \operatorname{DE}=1+(40-1)(0.12)=1+4.68=5.68. $$ Then $$ N_{\mathrm{eff}}=\frac{600}{5.68}=105.6. $$

For the pooled cell mean, the 600 clustered measurements have the same model-based variance as approximately 105.6 independent measurements with the same marginal variance. This calculation does not mean the study has 105.6 independent donors: it has 15 donors. In particular, a between-condition effect with condition assigned to donors must be supported by independent donor replication.

Unequal Numbers of Cells per Donor

Cell yields are often unequal. With donor $i$ contributing $m_i$ cells and $N=\sum_i m_i$, the equal-cell-weighted pooled mean has variance $$ \operatorname{Var}(\bar{Y})=\frac{\sigma^2}{N^2}\left[(1-\rho)N+\rho\sum_{i=1}^{D}m_i^2\right]. $$ Relative to the independent-cell variance $\sigma^2/N$, this gives $$ \operatorname{DE}=1+\rho\left(\frac{\sum_{i=1}^{D}m_i^2}{N}-1\right). $$ The term $\sum_i m_i^2/N$ is a cell-weighted average cluster size: donors with many cells contribute disproportionately to it. If all $m_i=m$, then $\sum_i m_i^2/N=m$, recovering $1+(m-1)\rho$. These formulas assume equal marginal variances and the same within-donor ICC across clusters; more complex structures may require model-based calculations.

Worked Example: Unequal Donor Cell Counts

Four hypothetical donors contribute 10, 20, 30, and 40 cells, respectively. Let the ICC be 0.20. Calculate the unequal-size design effect and effective sample size for the equal-cell-weighted pooled mean.

The total is $N=10+20+30+40=100$, and the sum of squared cluster sizes is $$ \sum_i m_i^2=10^2+20^2+30^2+40^2=100+400+900+1600=3000. $$ Therefore, $$ \operatorname{DE}=1+0.20\left(\frac{3000}{100}-1\right) =1+0.20(29)=6.80, \qquad N_{\mathrm{eff}}=\frac{100}{6.80}=14.71. $$

For comparison, using the arithmetic mean cluster size $100/4=25$ as if all clusters had size 25 would give $1+24(0.20)=5.80$, which is too small for this unequal allocation. Squaring the cluster sizes captures the extra contribution of the largest clusters to the variance of an equal-cell-weighted mean.

Using the Calculation and Checking Its Limits

The design effect is useful for communicating why a large cell total does not necessarily provide proportionately large precision. It can also give a first-order adjustment to the variance of a pooled mean when the assumptions are reasonable. It is not a universal correction factor for every analysis. For example, a paired within-donor comparison, a donor-level pseudobulk analysis, or a model with multiple nested technical sources has a variance structure that is not summarized automatically by this one-ICC formula.

For a between-donor condition comparison, independent donor counts remain central. Cells improve the measurement of each donor’s expression profile, but they do not create additional independently randomized donors. A cell-level mixed model can represent donor clustering directly; donor-level pseudobulk analyses make the donor-level unit explicit. Cluster-robust standard errors are another approach when there are enough independent clusters and their assumptions are appropriate. An ICC-based design effect is a compact planning approximation, not a substitute for choosing an analysis aligned with the design.

Worked Example: Translating a Design Effect Into a Standard Error

Suppose a hypothetical pooled mean is based on 8 donors with 25 cells each, so $N=200$. Assume the marginal variance of the cell outcome is $\sigma^2=1.00$ expression units squared and the ICC is 0.12. Compare the independent-cell standard error with the clustered standard error under the equal-size model.

The design effect is $$ \operatorname{DE}=1+(25-1)(0.12)=1+2.88=3.88, \qquad N_{\mathrm{eff}}=\frac{200}{3.88}=51.55. $$ If cells were independent, the standard error would be $$ SE_{\mathrm{ind}}=\sqrt{\frac{1.00}{200}}=0.07071 $$ expression units. Accounting for clustering, $$ SE_{\mathrm{cluster}}=\sqrt{\operatorname{DE}}\;SE_{\mathrm{ind}} =\sqrt{3.88}(0.07071)=0.1393 $$ expression units. The clustered standard error is about 1.97 times the naive one. This illustrates uncertainty in the pooled mean; it does not turn the 8 donors into 51.55 independent biological replicates for a condition test.

Diagnostics should focus on whether the model describes the data and the intended estimand. Compare donor-level summaries, inspect cell yields and residual patterns, and assess whether covariates or cell subpopulations change the estimated variance components. ICC estimates based on few donors can be uncertain even when many cells are observed, because information about between-donor variation is limited by the number of donors. Report the scale, outcome, model, and uncertainty behind a planning ICC where possible.

Constrained maximum-likelihood or restricted maximum-likelihood estimates of the random-intercept variance components are nonnegative, so their ICC estimate is generally nonnegative and may lie at the zero boundary. A negative ICC can arise from sampling variation with an unconstrained estimator, such as a method-of-moments estimate; it is not a sensible default planning value under the stated random-intercept model. Investigate the fitted model and uncertainty rather than interpreting a boundary estimate as proof that donors contribute no clustering.

Assuming cell count equals replication. Cells from one donor share biological effects; use donor counts to describe independent donor replication.
Using one ICC indiscriminately. ICC depends on outcome, population, scale, and model. A value estimated for one gene or cell type may not apply to another.
Applying the equal-size formula to highly unequal clusters. Use the unequal-size expression for the specified equal-cell-weighted mean, or use a model that reflects the actual design.
Reading effective sample size as an exact test sample size. It summarizes variance for a particular mean under assumptions; it does not supply degrees of freedom or replace a valid donor-level analysis.

Key takeaway: For equal cluster sizes, within-donor similarity inflates the variance of a pooled cell mean by $1+(m-1)\rho$. Effective sample size describes that loss of cell-level information, while independent donor counts remain essential for biological condition comparisons.

Check Your Understanding

Answer the questions, then compare your reasoning with the solutions below.

If $\sigma_u^2=0.18$ and $\sigma_\varepsilon^2=0.42$, what is the ICC under the random-intercept model?
With 12 cells per donor and ICC 0.10, calculate the equal-size design effect.
Why is an effective sample size of 80 not evidence that a study has 80 independent donors?
For unequal clusters of sizes 5 and 15, with ICC 0.25, calculate the design effect for the equal-cell-weighted mean.
What does a zero-boundary ICC estimate from constrained variance-component fitting establish about donor similarity?

Solutions. (1) The total variance is $0.18+0.42=0.60$, so $\rho=0.18/0.60=0.300$. (2) $\operatorname{DE}=1+(12-1)(0.10)=2.10$. (3) Effective sample size is a variance-equivalent count for a specified cell-level mean; it does not alter the number of independent donor units, which is the relevant replication count for a donor-assigned condition. (4) Here $N=20$ and $\sum_i m_i^2=5^2+15^2=250$, so $\operatorname{DE}=1+0.25(250/20-1)=1+0.25(11.5)=3.875$. (5) It indicates that the fitted constrained estimate is at the zero boundary; it does not prove true independence, particularly when the number of donors is small or the estimate is uncertain.