Few Clusters And GEE Inference

Why the Number of Clusters Matters

GEE estimates a population-averaged mean relationship while allowing outcomes within a cluster to be dependent. For robust inference, the key replication is the number of independent clusters, not simply the total number of observations. A study with hundreds of repeated measurements from a handful of clinics still has only a handful of independent units contributing to the large-sample justification for its sandwich variance.

This distinction matters because the robust variance estimates variability across clusters. With many independent clusters, the cluster contributions can average out and the sandwich estimator can approximate the sampling variance. With few clusters, that approximation may be inaccurate: the estimated variance can be too small, confidence intervals can be too narrow, and tests can reject too often. These are risks, not guaranteed outcomes; small cluster counts do not imply that every sandwich standard error is biased downward.

For cluster \(g\), let \(U_g(\hat{\beta})\) denote its contribution to the GEE score at the fitted coefficient vector \(\hat{\beta}\). In standard notation, the robust sandwich variance has the form

where \(G\) is the number of independent clusters and \(\hat{A}\) is the estimated sensitivity, or derivative, of the estimating equation. The middle “meat” estimates variation using the observed cluster contributions. Thus, adding many observations within the same few clusters does not provide the same information about between-cluster variation as adding independent clusters.

Why Few Clusters Can Make the Sandwich Unreliable

A useful justification begins with the large-sample argument. If clusters are independent and no one cluster dominates, the sum of their score contributions is approximately normal as \(G\) grows. The sandwich estimates the variance of that sum after accounting for the sensitivity of the estimating equation. This is a cluster-level central limit argument: the approximation improves mainly as the number of independent clusters increases.

In a finite sample, the fitted coefficients have already been chosen to make the total estimating equation close to zero. The same fitted coefficients are then used to calculate each cluster’s residual-based score contribution. This fitting constrains and can shrink the observed cluster contributions, particularly when the number of clusters is small relative to the number of fitted coefficients or when a few clusters have high leverage. The empirical meat may consequently understate the variation that would be seen across repeated studies.

A simplified scalar illustration makes the issue visible. Suppose the target is the mean of independent cluster-level values \(\theta_g\). Its estimate is \(\bar{\theta}\), and the sandwich analogue is \(s_\theta/\sqrt{G}\), where \(s_\theta\) is their sample standard deviation. This is a reasonable estimate of uncertainty when \(G\) is sufficiently large. When \(G\) is small, however, \(s_\theta\) itself is uncertain, and a normal reference distribution does not account for that uncertainty. In a general GEE, the coefficient estimator and its sandwich variance are more complicated; there is no universal exact finite-sample t distribution.

What Changes for Confidence Intervals and Tests

A standard large-sample Wald interval for coefficient \(\beta_j\) is \(\hat{\beta}_j\pm z_{1-\alpha/2}SE(\hat{\beta}_j)\), and the corresponding two-sided test uses a standard normal reference distribution. Those procedures are justified asymptotically, not by the total observation count alone. When clusters are few, the normal approximation and the estimated standard error may both be poor. The resulting p-value can be too small and the nominal coverage of a confidence interval can be too low.

Some small-sample procedures use a t reference distribution with degrees of freedom related to the number of clusters and fitted parameters. Others adjust the sandwich covariance to reduce finite-sample bias, for example by modifying cluster contributions to account for leverage. Such adjustments include methods commonly known as Mancl–DeRouen, Kauermann–Carroll, and Fay–Graubard corrections. Their details and suitable degrees of freedom vary; no single correction is automatically best for every design or software implementation. A correction cannot create independent information that the study did not collect.

Diagnostics and Practical Cautions

No diagnostic can turn a handful of clusters into a large-sample design, but several checks can reveal especially fragile inference. Begin by reporting \(G\), not just the total number of observations. For cluster-randomized studies, also report the number of clusters in each treatment group: a large total can conceal very few independent units in one group. Then inspect cluster score contributions, leverage, and the sensitivity of the estimate and standard error to removing one cluster at a time. A large change after omitting one cluster signals that the result depends heavily on that cluster.

These checks are warnings, not mechanical pass/fail tests. A cluster that contributes strongly may be influential because it contains important information, not because it is erroneous. Investigate data quality and design context before deciding what to do. Do not remove a valid cluster merely to obtain a smaller p-value or a more convenient result.

The main alternatives have different assumptions and targets. A model-based GEE standard error can be more efficient if the working covariance is correctly specified, but it does not provide the same protection against covariance misspecification as the robust estimator. A small-sample corrected sandwich or a t-based procedure aims to improve finite-sample inference while retaining a marginal GEE analysis, but its performance depends on the design and method. A GLMM models cluster dependence through random effects and can support inference under its own distributional and modeling assumptions; it is not simply a small-cluster correction to GEE, and its conditional effects need not equal population-averaged GEE effects.

The limitations are fundamental. When clusters are very few, there may be too little information for reliable estimation of cluster-level variability, and different small-sample methods may produce meaningfully different intervals or tests. Report the method and degrees of freedom or correction used, describe the number of independent clusters, and interpret marginal results with appropriate caution. Do not present a nominal 95% interval as having guaranteed 95% coverage in a design where its calibration is uncertain.

A Short Justification for the Small-Sample Warning

In the simplified mean example, let the independent cluster values have variance \(\sigma^2\). Then \(\operatorname{Var}(\bar{\theta})=\sigma^2/G\), and the usual estimate replaces the unknown \(\sigma^2\) by \(s_\theta^2\). With few clusters, that replacement is noisy. Even if the cluster values are normally distributed, estimating the variance costs degrees of freedom: the standardized statistic \((\bar{\theta}-\mu)/(s_\theta/\sqrt{G})\) follows a t distribution with \(G-1\) degrees of freedom, not a standard normal distribution. The t critical value is therefore larger than the normal critical value for a two-sided 95% interval when \(G\) is small.

That argument justifies the caution in the simple mean problem, but it does not prove an exact t result for general GEE coefficients. A GEE involves estimated nuisance structure, a vector estimating equation, and potentially unequal leverage across clusters. Small-sample GEE methods adapt the variance estimate and/or reference distribution to those features; their validity depends on the method’s assumptions.