Small-Sample Corrections With Few Donors

Why the Number of Donors Matters

Single-cell studies may measure thousands of cells, but cells from the same donor are not independent biological replicates. Cluster-robust standard errors account for within-donor dependence by treating each donor’s combined contribution to the estimating equation as a unit. Their usual justification, however, is an approximation that improves as the number of independent clusters grows. A large number of cells within each of a few donors does not supply the missing independent clusters.

The concern is not merely that a standard error might be a little too small. With few donors, the estimated variance can be highly sensitive to individual clusters, and the normal-reference test often used with a cluster-robust standard error can have a type I error rate above its nominal level. Finite-sample corrections address parts of this problem, but they do not create information absent from the study design.

Definition: Let \(G\) be the number of independent donors, and let \(N\) be the total number of observations, such as cells. A cluster-robust variance estimator allows observations within a donor to be dependent while relying on independence across donors. Its large-sample approximation is primarily in \(G\), not in \(N\). “Few donors” has no universal cutoff: risk depends on the number of donors, the balance of their contributions, the model design, and the inferential method.

The earlier material on cluster-robust standard errors gives the basic sandwich estimator. Here the focus is its finite-sample behavior and the practical alternatives. The working assumptions remain important: the mean model or estimating equation must be appropriate for the target coefficient, donors must be independent of one another for this variance calculation, and there must be enough independent information to estimate the coefficient. A correction cannot repair confounding, a misspecified mean model, or dependence between donors.

What the Ordinary Cluster-Robust Estimator Does

For a linear model with coefficient vector \(\hat{\beta}\), write \(X_g\) for the design matrix and \(\hat{e}_g\) for the residual vector of donor \(g\). The uncorrected cluster sandwich, often called CR0, is

\widehat{\mathrm{Var}}_{\mathrm{CR0}}(\hat{\beta}) = (X^\mathsf{T}X)^{-1} \left\{ \sum_{g=1}^{G} X_g^\mathsf{T}\hat{e}_g\hat{e}_g^\mathsf{T}X_g \right\} (X^\mathsf{T}X)^{-1}.

The middle term aggregates each donor’s residual-weighted design contribution before summing across donors. This permits arbitrary covariance among observations within a donor, but the empirical sum over donors must estimate the variability of the donor contributions. With small \(G\), that estimate is noisy. Fitted residuals also tend to be smaller for observations with high leverage, which can make the unadjusted variance estimate downward biased.

A common small-sample remedy is to multiply CR0 by a scalar. In a standard linear model, one CR1 convention uses the factor \(G/(G-1)\) times \((N-1)/(N-K)\), where \(K\) is the number of fitted coefficients. Software conventions vary, so an analysis should identify the correction actually used. Even after a variance correction, a test needs a reference distribution and degrees of freedom. Using a standard normal reference can remain too liberal when \(G\) is small.

Key distinction: A variance correction changes the estimated standard error; a degrees-of-freedom correction changes the reference distribution used for a test or interval. Both matter. Multiplying a variance by a finite-sample factor and then using a normal critical value does not generally reproduce a small-sample \(t\) procedure.

Worked Example: CR1 in a Six-Donor Intercept Model

Suppose a hypothetical analysis has one donor-level outcome per donor, with values \(2,0,1,3,2,4\) in arbitrary expression-score units. This is an intercept-only model, so \(G=N=6\) and \(K=1\). The six values are being treated as independent donor observations; for an exact small-sample \(t\) interpretation, additionally assume they are normally distributed with a common mean and variance.

The estimated mean is \(\bar{x}=2\). The residuals are \(0,-2,-1,1,0,2\), so their squared sum is \(10\). For this intercept model, the CR0 variance of the mean is

\widehat{\mathrm{Var}}_{\mathrm{CR0}}(\bar{x}) = \frac{10}{6^2} = 0.2778.

The CR1 factor is \((6/5)(5/5)=1.2\), giving

\widehat{\mathrm{Var}}_{\mathrm{CR1}}(\bar{x}) = 1.2(0.2778) = 0.3333, \qquad SE_{\mathrm{CR1}}=\sqrt{0.3333}=0.5774.

This equals \(s^2/G\), since the sample variance is \(s^2=10/5=2\). If testing a mean of zero, the statistic is \(2/0.5774=3.464\). Under the stated normal donor-level model, the reference distribution is \(t\) with \(5\) degrees of freedom, not a standard normal distribution. A two-sided 95% interval is \(2\pm 2.5706(0.5774)\), or approximately \(0.516\) to \(3.484\) score units. The example shows how the correction and the small-sample reference distribution work together; it does not establish that a scalar CR1 correction is sufficient for a more complex design.

CR2, CR3, and Degrees of Freedom

Scalar corrections treat all donor contributions alike. In unbalanced designs, or when a few donors have high leverage, the residual shrinkage can differ substantially from donor to donor. CR2 and CR3 address this by adjusting residual contributions for leverage. In an ordinary least-squares model under a homoskedastic working model, a common CR2 adjustment for donor \(g\) is based on \((I-H_{gg})^{-1/2}\), where \(H_{gg}\) is the donor’s block of the hat matrix. The adjusted residual is used to form the donor’s contribution to the sandwich.

CR2 is designed to make the variance estimator unbiased under its specified working model; it is not a guarantee of unbiasedness under every possible within-donor covariance pattern. It is often paired with a Satterthwaite approximation that estimates degrees of freedom for the particular coefficient or contrast. These degrees of freedom can be much smaller than \(G-1\), especially when the contrast depends heavily on only a few donors.

CR3 uses a stronger leverage adjustment, commonly related to deleting each cluster and refitting the model. This jackknife-like approach can be useful as a sensitivity analysis, but may yield larger standard errors and can be unstable when deleting a donor makes the design nearly rank-deficient. CR2 and CR3 therefore require attention to the design and implementation, not just selection from a menu.

Definition: A Satterthwaite degrees-of-freedom approximation matches the first two moments of an estimated variance to those of a scaled chi-squared random variable. For a representation \(\hat{V}=\sum_j \lambda_j Z_j^2\), where the \(Z_j\) are independent standard normal variables and \(\lambda_j\) are nonnegative, the moment-matched degrees of freedom are \((\sum_j\lambda_j)^2/\sum_j\lambda_j^2\). In applied CR2 analyses, this idea yields a contrast-specific approximation rather than one universal degrees-of-freedom value for the whole model.

Worked Example: Interpreting CR2 Degrees of Freedom

Consider a hypothetical CR2 analysis of a treatment coefficient, measured in log-expression units. Suppose its variance estimate has a moment representation with six contributions proportional to \(1,1,1,1,1,\) and \(4\). These are illustrative variance contributions, not six new donors or observed expression values. The Satterthwaite approximation is

df = \frac{(1+1+1+1+1+4)^2} {1^2+1^2+1^2+1^2+1^2+4^2} = \frac{81}{21} = 3.857.

Suppose the fitted treatment coefficient is \(0.40\) log-expression units and the CR2 standard error is \(0.18\) log-expression units. The test statistic for a zero coefficient is \(0.40/0.18=2.222\). With approximately \(3.857\) degrees of freedom, the two-sided 5% critical value is about \(2.817\), so an approximate 95% interval is \(0.40\pm 2.817(0.18)\), or about \([-0.107,0.907]\) log-expression units. It includes zero, so this procedure does not reject a zero coefficient at the 5% level. The estimated degrees of freedom express the limited independent information for this contrast; they are not a count of donors.

Donor-Level Inference as an Alternative

When the scientific design supports it, direct inference on donor summaries can be simpler to explain than a cell-level regression with a small-sample cluster correction. For example, a donor-level analysis may first form one summary per donor and compare those summaries across groups. The earlier material on one-sample and two-group inference across donors provides the standard calculations. Such methods make the independent biological unit explicit, but they answer the question defined by the summaries and may not represent every cell-level estimand.

For paired conditions measured on the same donors, form one within-donor contrast per donor and analyze those contrasts. For independent groups of donors, compare donor summaries using a procedure that allows unequal variances when appropriate. A donor-level \(t\) procedure is not automatically valid: its assumptions include independent donors, a meaningful summary, and a distributional approximation adequate for the number of donors. A very small sample leaves limited ability to check that approximation.

Worked Example: A Paired Donor-Level Contrast

Suppose six hypothetical donors have paired treatment-minus-control summaries on a log2 expression scale: \(1-3, 1+3, 1-4, 1+4, 1-\sqrt{20}, 1+\sqrt{20}\). The resulting contrasts are approximately \(-2,4,-3,5,-3.472,5.472\) log2 units. Assume the donors are independent and that the donor contrasts are plausibly sampled from a normal population. The mean contrast is \(1.000\) log2 units. The deviations from this mean are paired values with squared sum \(2(3^2+4^2+(\sqrt{20})^2)=90\), so the sample variance is \(90/5=18\) and the standard error is \(\sqrt{18/6}=\sqrt{3}=1.732\) log2 units.

The one-sample statistic for a mean contrast of zero is \(t=1/1.732=1/\sqrt{3}=0.5774\), with \(5\) degrees of freedom. Its two-sided \(p\)-value is approximately \(0.5887\). The 95% \(t\) interval is \(1.000\pm 2.5706(1.732)\), or approximately \([-3.452,5.452]\) log2 units. The interval includes zero, and the test does not reject the zero-mean-contrast null hypothesis at the 5% level. The \(p\)-value is the probability, assuming that null and the stated model, of a statistic at least as extreme as the observed one; it is not the probability that the null is true.

Choosing a Method and Checking Its Weak Points

Approach	What it changes or assumes	Useful caution
CR1 with a \(t\) reference	Scales the sandwich variance and uses a finite-cluster degrees-of-freedom convention.	The scaling is not leverage-specific; degrees of freedom may be crude.
CR2 with Satterthwaite degrees of freedom	Adjusts residual contributions for leverage and estimates degrees of freedom for a contrast.	Depends on a working covariance model and appropriate implementation.
CR3	Uses a stronger, jackknife-like cluster adjustment.	Can be sensitive to deleting high-leverage donors or losing model rank.
Donor-level summaries	Analyzes independent donor summaries or within-donor contrasts directly.	Targets the chosen summary; few donors still limit precision and distribution checks.

Diagnostics should start with the design, not just the reported standard error. Count donors in each condition, inspect how much each donor contributes to the coefficient, and verify that treatment is not perfectly confounded with donor or another design factor. Compare results under a defensible CR2 or donor-level analysis when feasible, and examine whether conclusions hinge on a particular correction. Large disagreement is a warning that the small-sample approximation is consequential, not a reason to select whichever result is preferred.

A common mistake is to argue that thousands of cells make a normal-reference cluster-robust test reliable even when only a handful of donors were sampled. That reasoning confuses the number of observations with the number of independent clusters. More cells can improve the precision of each donor’s summary, but they do not provide more independent information about between-donor variation.

No finite-sample correction removes the fundamental limitation of few donors. CR1 may not address leverage, CR2 and CR3 may depend on modeling and design choices, and donor-level tests may have weak distributional support with very small samples. Report the number of donors, the correction and reference distribution, the degrees of freedom, and the unit of the effect estimate. Interpret uncertainty in the biological units of the analysis, and avoid describing non-significance as proof of no effect.

Establish the independent unit.
Identify donors as the clusters and report their number by condition; do not substitute the number of cells for the number of replicates.

State the estimator and correction.
Specify whether the variance is CR0, CR1, CR2, or CR3, and state how the reference degrees of freedom were obtained.

Check design dependence.
Review balance, leverage, and whether the treatment contrast is supported across donors rather than dominated by one donor.

Compare defensible analyses.
Where their assumptions fit the design, compare corrected cluster-robust inference with donor-level inference and explain differences in estimand or assumptions.

Key takeaway: Cluster-robust inference is asymptotic in the number of independent donors. Small-sample corrections can improve variance estimation and reference distributions, but do not replace biological replication or excuse a design with insufficient independent information.

Check Your Understanding

Try the questions first, then check the solutions below.

A study has 12,000 cells from four donors. Which count is central to the large-cluster justification for cluster-robust inference, and why?
In the six-donor intercept example, why did CR1 change the CR0 variance from \(0.2778\) to \(0.3333\)?
What is one distinction between CR2 and CR1?
A CR2 analysis reports a contrast-specific \(df=3.2\) with 10 donors. Does this imply that the analysis used only 3.2 donors?
Why can a donor-level analysis be a useful alternative, and what limitation remains if there are very few donors?

Solutions. (1) The relevant count is four independent donors. Cells within a donor can be dependent, so 12,000 cells do not provide 12,000 independent cluster contributions. (2) In this intercept-only setup, the CR1 factor is \((G/(G-1))((N-1)/(N-K))=(6/5)(5/5)=1.2\), and \(1.2(0.2778)=0.3333\). (3) CR1 applies a scalar factor, while CR2 adjusts residual contributions for leverage under a working model and can use contrast-specific Satterthwaite degrees of freedom. (4) No. The \(3.2\) is an approximate degrees-of-freedom value for the contrast’s reference distribution, not a count of donors. (5) Donor-level analysis makes the biological replicate explicit and may provide a transparent summary or paired contrast; very few donors still mean limited precision and limited ability to assess distributional assumptions.