Why Cell Counts Can Exaggerate Evidence
Single-cell experiments can measure thousands of cells, but cells from the same donor share biological and technical influences. If the scientific question concerns differences across donors or donor populations, the independent replication is primarily at the donor level. Treating every cell as an independent replicate can make a modest, noisy difference appear precisely estimated. This is pseudoreplication: the analysis counts subsamples as if they were independent experimental units.
The issue is not that cells contain no information. Additional cells can improve measurement of a donor’s cell population. The problem is using the total cell count as though it represented the same number of independent biological replicates. The derivation below isolates this issue under a simple model and shows how it distorts a standard error and a null test.
A Model for Correlated Cells
Suppose there are two groups, indexed by \(g=0,1\), with \(G\) independent donors in each group and \(m\) cells sampled per donor. Let \(Y_{gij}\) be the expression measurement for cell \(j\) from donor \(i\) in group \(g\), on a specified analysis scale. For this derivation, assume a balanced design and the random-intercept model
The donor effects \(b_{gi}\) are independent across donors, and the cell-specific errors \(\varepsilon_{gij}\) are independent across cells and donors and independent of the donor effects. The groups have independent donors. The model states that each cell measurement combines a group mean, a donor-specific deviation, and a cell-specific deviation. Normality is not needed for the variance calculation; it is used later only for a normal-reference test.
A cell’s marginal variance is \(\sigma^2=\tau^2+\sigma_e^2\). Two different cells from the same donor share \(b_{gi}\), so their covariance is \(\tau^2\), and their correlation is \(\rho=\tau^2/(\tau^2+\sigma_e^2)\). Thus \(0\leq\rho\leq1\) in this model. The group contrast of interest is \(\Delta=\mu_1-\mu_0\), the difference in mean expression on the chosen scale.
How the Standard Error Is Inflated
Let \(\bar{Y}_g\) be the average of all \(Gm\) cell measurements in group \(g\). The average includes \(Gm\) observations, but observations from a donor are correlated. To find its variance, add the variances of all cells and the covariances of all distinct within-donor cell pairs. Cells from different donors have zero covariance under the assumptions above.
The term \(m(m-1)\) counts ordered pairs of distinct cells within one donor: there are \(m\) choices for the first cell and \(m-1\) for the second. The factor \(G\) accounts for the donors. Since the two groups have independent donors, their group-mean variances add when forming the contrast.
An analysis that incorrectly assumes all cells are independent omits the covariance terms. Its model-based variance for the contrast is \(2\sigma^2/(Gm)\). The ratio of the correct variance to this naive variance is the design effect, \(D=1+(m-1)\rho\); the ratio of standard errors is \(\sqrt{D}\). The independent-cell calculation is therefore too small whenever \(\rho>0\) and \(m>1\).
This also motivates an effective sample size: \(Gm/D\) independent observations would have the same variance for a group mean as \(G\) clusters of \(m\) correlated observations, under this balanced model. It is a variance-equivalence calculation, not a literal count of independent cells and not a substitute for knowing how many donors were sampled.
Worked Example: Comparing Standard Errors for a Group Contrast
Suppose a hypothetical study has 8 donors in each group and 50 cells per donor. On the chosen expression scale, assume the marginal cell standard deviation is 1 unit and the within-donor correlation is \(\rho=0.10\). For illustration, suppose the observed group contrast is 0.30 units. The balanced-cluster model is being used as the calculation’s assumption; the values are hypothetical, not measurements from a real study.
First calculate the design effect: \(D=1+(50-1)(0.10)=1+4.9=5.9\). Thus the standard-error inflation factor is \(\sqrt{5.9}=2.429\).
The naive standard error, using 400 cells per group as if independent, is \[ SE_{\text{naive}}=\sqrt{\frac{2(1)^2}{8(50)}}=\sqrt{0.005}=0.07071 \] expression units. The variance calculation that includes clustering gives \[ SE_{\text{clustered}}=\sqrt{\frac{2(1)^2}{8(50)}(5.9)} =\sqrt{0.0295}=0.1718 \] expression units.
Using a normal reference for illustration, the naive statistic is \(0.30/0.07071=4.243\), whereas the statistic using the clustered standard error is \(0.30/0.1718=1.746\). The corresponding two-sided normal-reference p-values are approximately \(0.000022\) and \(0.0807\), respectively. Under these assumptions, the first calculation treats the contrast as far more precise than the donor-level covariance supports. In an actual analysis, variance-component estimation and the number of donors would also affect uncertainty and the reference distribution.
How Clustering Inflates False-Positive Rates
The standard-error distortion affects more than the appearance of a confidence interval. Under a true null hypothesis, a test that uses the naive standard error can reject too often. Consider a two-sided normal-reference test at nominal level \(\alpha=0.05\), with critical value approximately 1.96. Assume for this calculation that the marginal variance and \(\rho\) are known, the null contrast is zero, and the balanced-cluster model holds.
The naive statistic divides the estimated contrast by a standard error that is too small by a factor of \(\sqrt{D}\). Consequently, under the null, its distribution is approximately \(N(0,D)\), rather than \(N(0,1)\). Equivalently, dividing that statistic by \(\sqrt{D}\) gives a standard normal variable. The actual rejection probability is therefore
This calculation makes explicit why a correct null hypothesis does not protect against pseudoreplication: the problem is that the test’s reference distribution does not match the variability of its statistic. The formula is exact under the stated known-variance normal model. When variances are estimated, or when donor counts are small, this normal approximation need not give the exact rejection probability, but the source of the distortion remains.
Worked Example: A Nominal 5% Test With a 32.71% False-Positive Rate
Consider a hypothetical balanced experiment with 4 cells per donor and within-donor correlation \(\rho=1/3\). Then \(D=1+(4-1)(1/3)=2\), so the variance is twice the naive variance. For a transparent, more pronounced illustration, instead take 31 cells per donor and \(\rho=0.10\), giving \(D=1+(31-1)(0.10)=4\).
Under the null and the normal-reference assumptions above, the nominal 5% test rejects when the absolute naive statistic exceeds 1.96. Its actual null rejection probability is
\(2\Phi(-1.96/\sqrt{4})=2\Phi(-0.98)\approx0.3271\).
Thus the false-positive rate is approximately 32.71%, not 5%, under this idealized model. The calculation does not say that any particular significant result is false; it says that repeated use of this misspecified test under the null would reject about 32.71% of the time in these conditions.
More Cells, More Donors, and the Limits of the Calculation
The design effect explains why adding cells within existing donors may deliver diminishing gains for donor-population inference. With \(G=6\) donors per group, \(m=20\) cells per donor, and \(\rho=0.20\), the design effect is \(1+19(0.20)=4.8\). Each group has 120 cells, but the effective sample size under the variance-equivalence definition is \(120/4.8=25\).
Worked Example: Where Additional Replication Adds Information
Continue the hypothetical setting with 6 donors per group and \(\rho=0.20\). If cells per donor rise from 20 to 40, there are 240 cells per group, but \(D=1+39(0.20)=8.8\). The effective sample size becomes \(240/8.8=27.27\), an increase from 25 rather than a doubling.
If instead the number of donors rises from 6 to 12 while cells per donor remain 20, there are also 240 cells per group. The design effect remains 4.8, and the effective sample size is \(240/4.8=50\). Under the model, increasing donor replication reduces uncertainty more than adding the same number of cells within the existing donors. This comparison concerns the variance of a group mean; it does not imply that cell sampling is useless, since cells still help characterize each donor.
The equal-cluster calculation is useful for understanding the mechanism, not for mechanically correcting every analysis by a single factor. Unequal numbers of cells per donor, varying within-donor correlations, cell-type composition differences, batch effects, and non-normal or count-valued outcomes can all require more careful modeling. Applying \(D\) to a naive standard error does not by itself solve confounding, selection, or a mismatch between the target population and the sampled donors.
The main alternatives differ in how they represent dependence. A cluster-aware model specifies donor-level variation and cell-level variation; cluster-robust variance methods adjust uncertainty for dependence without requiring the same detailed covariance model, but generally need enough independent clusters for reliable approximation. Analyses based on donor-level summaries make the donor the observation, though the summary and its weighting must match the estimand. None of these approaches creates biological replication where there are few donors. The appropriate choice depends on the estimand, design, number of independent donors, and the plausibility of its assumptions.
Diagnostics and Common Pitfalls
Before interpreting a cell-level p-value, examine whether the observed data and design make independence plausible. Plot donor-level expression summaries or donor-specific group contrasts; look for donors that dominate the cell count or show systematically different expression; and estimate or otherwise assess within-donor similarity. A substantial positive within-donor correlation is a warning that the independent-cell standard error may be severely optimistic. Diagnostics cannot prove independence, especially with few donors, so study design and biological knowledge matter as well.
- Wrong: “There are 10,000 cells, so the analysis has 10,000 independent replicates.” Correction: Cells sharing donors contribute correlated information; the donor count and within-donor correlation determine how much independent evidence is available.
- Wrong: “The design effect is 5.9, so divide the p-value by 5.9.” Correction: The design effect multiplies the variance relative to the naive variance; the standard error is multiplied by \(\sqrt{5.9}\). A p-value must be recalculated using an appropriate statistic and reference distribution.
- Wrong: “A large cell count guarantees a reliable donor-population conclusion.” Correction: Many cells can estimate within-donor features precisely, but they do not replace independent donors for estimating between-donor variability.
The key limitation is that the design-effect formula depends on balanced clusters and a common pairwise correlation. It is a derivation of the direction and scale of the problem under explicit assumptions, not a universal correction formula. A careful analysis must align the unit of inference with the scientific question and account for dependence in its uncertainty calculation.
Check Your Understanding
Answer each question, then compare with the solutions below.
- With \(m=10\) cells per donor and \(\rho=0.05\), calculate the design effect and standard-error inflation factor.
- Under a balanced design, why do covariances affect the variance of a group mean even though every cell has the same marginal variance?
- For a nominal two-sided 5% normal test with \(D=9\), calculate the actual null rejection probability under the known-variance model.
- Explain why adding cells to existing donors and adding new donors are not equivalent for donor-population inference.
Solutions. (1) \(D=1+(10-1)(0.05)=1.45\), so the standard-error inflation factor is \(\sqrt{1.45}=1.204\). (2) The variance of a sum includes the covariance of every distinct pair; within-donor pairs have covariance \(\rho\sigma^2\), so omitting them understates the group-mean variance. (3) \(2\Phi(-1.96/\sqrt{9})=2\Phi(-0.6533)\approx0.5136\), or about 51.36%. (4) Cells added to an existing donor share that donor’s deviation and therefore add correlated observations, while new donors add independent biological clusters under the model. Hence equal totals of cells can yield different information about a donor-population mean.