Unequal Cluster Sizes and Fit Stability

Why cluster-size imbalance matters

A GLMM may contain many observations but comparatively few independent clusters. When cluster sizes vary greatly, the largest clusters can contribute strongly to estimation of some parameters, while small clusters contribute much less. This uneven contribution can make a random-effect estimate sensitive to a few clusters, and can make numerical fitting more difficult. Imbalance alone does not imply that a model is wrong or that an optimizer has failed; it is a reason to inspect what is informing the fit.

The key distinction is between the number of observations and the number of independent realizations of the random effects. For a random-intercept model, each cluster supplies one realization of its latent intercept. Additional observations within that cluster can estimate its response pattern more precisely, but they do not create additional independent random intercepts.

Definition: Let \(J\) denote the number of clusters and \(n_i\) the number of observations in cluster \(i\), for \(i=1,\ldots,J\). Cluster sizes are balanced when the \(n_i\) are equal or similar, and unbalanced when they differ substantially. In a GLMM, observations are conditionally independent given the random effects and covariates, and the random effects follow the specified distribution, commonly a mean-zero normal distribution with covariance matrix \(G\). These assumptions, along with a correctly specified conditional response distribution and link, are needed for model-based estimation and its usual information calculations.

The consequences depend on the parameter. A large cluster may contribute substantial information about a fixed effect that varies within that cluster. It may also estimate that cluster's conditional response level precisely. But information about between-cluster variation depends importantly on having independent clusters with different latent effects. In binary or count GLMMs, the response variance and link make the precise contribution more complicated than a simple function of \(n_i\). A Gaussian random-intercept model provides a useful exact illustration of the underlying imbalance mechanism.

An information calculation for a random-intercept model

Suppose the response is continuous and follows the Gaussian random-intercept model

Y_{ij}=\mu+b_i+\epsilon_{ij},\qquad b_i\sim N(0,\tau^2),\qquad \epsilon_{ij}\sim N(0,\sigma^2),

where \(b_i\) and \(\epsilon_{ij}\) are mutually independent across clusters and observations, and \(\mu,\tau^2,\sigma^2\) are common to all clusters. For this calculation, treat \(\sigma^2\) as known. The cluster mean \(\bar{Y}_i\) has variance \(\tau^2+\sigma^2/n_i\). Thus clusters of different sizes have different precision for their cluster means.

The expected Fisher information in one normal cluster mean for \(\tau^2\) is one half the reciprocal square of its variance. Consequently, the total information for \(\tau^2\) is the sum of the cluster contributions. This is a local measure of curvature in the likelihood, not a guarantee that the finite-sample estimate will be stable or normally distributed.

Justification: If \(Z_i=\bar{Y}_i\sim N(\mu,V_i)\), where \(V_i=\tau^2+\sigma^2/n_i\), its log likelihood contribution, apart from a constant, is \(-\tfrac12\log V_i-\tfrac12(Z_i-\mu)^2/V_i\). Differentiating with respect to \(\tau^2\) gives \(-1/(2V_i)+(Z_i-\mu)^2/(2V_i^2)\). Since \(E[(Z_i-\mu)^2]=V_i\), the expected squared score is \(1/(2V_i^2)\). Independent cluster contributions add, giving \(I(\tau^2)=\tfrac12\sum_{i=1}^J(\tau^2+\sigma^2/n_i)^{-2}\). In this normal model the mean and variance scores are orthogonal, so estimating the common mean does not change this expected information expression.

Worked Example: Equal totals, different information

Consider 100 observations in 10 clusters, with \(\tau^2=0.25\) and \(\sigma^2=1\). Compare a balanced allocation of 10 observations per cluster with an unbalanced allocation of 9 clusters of size 1 and one cluster of size 91. The variances are assumed known for this information comparison; the calculation is not a fit to observed data.

Balanced design. Each cluster mean has variance \(0.25+1/10=0.35\). Therefore,

I_{\text{balanced}}(\tau^2) =\frac12\frac{10}{(0.35)^2} =40.8163.

Unbalanced design. The nine singleton cluster means each have variance \(0.25+1=1.25\); the size-91 cluster mean has variance \(0.25+1/91\). Thus,

I_{\text{unbalanced}}(\tau^2) =\frac12\left\{\frac{9}{(1.25)^2} +\frac{1}{(0.25+1/91)^2}\right\} \approx 10.2204.

The information ratio is \(40.8163/10.2204\approx 3.99\). In this model, with the stated variances and equal total sample size, the unbalanced allocation supplies about one quarter as much expected information about \(\tau^2\) as the balanced allocation. Under a regular large-sample approximation, the corresponding standard-error ratio would be about \(\sqrt{3.99}\approx 2.00\). With only 10 clusters, that approximation is not a promise of actual standard errors; the calculation isolates how allocation can affect information.

What the calculation does and does not say about a GLMM

For a non-Gaussian GLMM, one cannot generally substitute a cluster size into the Gaussian formula and obtain the exact information for a variance component. For instance, in a logistic random-intercept model, observations with probabilities near zero or one contribute little information about a conditional log-odds, while observations near one half contribute more. Covariates, outcome patterns, and the random-effect distribution also affect the likelihood. Nevertheless, the basic issue remains: highly unequal sizes can make contributions to the fitted likelihood uneven, and the largest cluster can dominate particular parts of the estimation.

A large cluster may identify its own conditional response level well, but it remains one cluster-level realization. If the random-effect variance is supported mainly by a few large clusters, the fitted variance can be sensitive to their outcomes or covariate patterns. Conversely, many small clusters can provide numerous independent realizations but weak within-cluster information. Neither pattern is automatically preferable in every model; the relevant question is which parameter is being estimated and how the design informs it.

Worked Example: Many binary observations do not mean many clusters

Suppose the response is binary and there are 10 clinics: 9 clinics enroll 1 patient each and one clinic enrolls 91 patients. For illustration, suppose the marginal response probability is near \(0.20\). The expected numbers of responses are \(1(0.20)=0.20\) in each singleton clinic and \(91(0.20)=18.2\) in the large clinic. These expectations need not be integers.

The 91 observations can give a comparatively precise estimate of the large clinic's response pattern, subject to the GLMM and its covariates. But the design still has only 10 independent clinic intercepts, and 9 of those intercepts are informed by one binary observation each. A singleton clinic's observed outcome is either 0 or 1; it cannot by itself reveal whether that clinic's underlying propensity is modestly above or below the population level.

Therefore, reporting “100 observations” alone would obscure the limited and uneven information about between-clinic variation. A useful description gives both the number of clinics and the distribution of clinic sizes, such as the minimum, median, maximum, and total. The conclusion is not that the model cannot be fitted, but that variance estimation and its numerical stability deserve particular scrutiny.

Assessing dependence on the largest clusters

One practical diagnostic is to compare how much a few clusters contribute to information or to the fitted objective. The exact contribution is model-dependent, so this is not always a simple additive decomposition for a GLMM. Still, examining cluster-level scores, likelihood contributions, or refits that omit influential clusters can expose dependence that a total sample size conceals. Such refits are sensitivity checks: they do not establish that the omitted cluster is erroneous, nor do they replace the primary analysis.

Worked Example: How concentrated is the variance information?

Use the unbalanced Gaussian design from the earlier calculation, with known \(\tau^2=0.25\) and \(\sigma^2=1\). The nine singleton clusters together contribute

\frac12\frac{9}{(1.25)^2}=2.88

units of expected information. The size-91 cluster contributes the remainder, approximately \(10.2204-2.88=7.3404\). Its share is \(7.3404/10.2204\approx 0.718\), or about 71.8%. In contrast, each size-10 cluster in the balanced design contributes \(\tfrac12/(0.35)^2\approx 4.0816\), exactly 10% of the total information for that design.

This calculation shows concentration, not an error. If the large cluster were removed, the unbalanced design would retain only 2.88 units of information under these assumptions. Removing one balanced cluster would leave \(9(4.0816)\approx36.7347\), or 90% of its original information. For an actual GLMM, analogous diagnostics should use the fitted model and be interpreted in context; a single cluster's likelihood contribution is not necessarily equal to its influence on a variance estimate.

A useful stability review examines whether reasonable changes in starting values or optimizer produce comparable parameter estimates and objective values, whether the fitted solution passes the scaled-gradient and Hessian checks, and whether estimates change markedly when the largest clusters are inspected or omitted in sensitivity analyses. The first checks address numerical behavior; the latter address design dependence. A fit can converge numerically and still be highly dependent on one cluster, or show a warning without imbalance being the cause.

Also inspect the outcome and covariate distributions by cluster size. If the largest cluster is the only one covering a predictor range, its contribution may be important for fixed effects as well as random effects. Near-constant outcomes in small clusters, sparse outcomes in the largest clusters, or covariate patterns found only in one size category can further reduce effective information. These are reasons to interpret estimates cautiously and report the size distribution, not reasons to assume a particular failure has occurred.

Alternatives, limits, and common pitfalls

The main alternative to an observation-weighted GLMM is not simply “correcting” unequal sizes. The ordinary likelihood gives observations their contribution under the specified conditional model. Reweighting clusters equally changes the objective and may change the target: it gives a small cluster the same total influence as a large one, rather than using the observation-level likelihood. Such weighting may be justified by a distinct sampling or estimand argument, but it is not a generic cure for unstable variance estimation.

Cluster-level summaries can clarify which groups have substantial data, but analyzing one summary per cluster discards within-cluster response information and may not reproduce the GLMM likelihood. A robust or sandwich variance estimator can alter uncertainty estimates under some forms of misspecification; it does not create additional independent clusters or repair a weakly supported random-effect distribution. If the scientific target differs, an alternative model or estimand may be appropriate, but it should be selected for that target rather than merely to make cluster sizes look balanced.

Common pitfall: “There are 100 observations, so the random-effect variance is well supported.” This confuses total sample size with the number and distribution of independent clusters. In the worked comparison, both designs have 100 observations and 10 clusters, yet the expected information about \(\tau^2\) differs by about a factor of four. Report cluster counts and size distribution, then examine parameter-specific information and sensitivity.

Unequal sizes have limits as a diagnostic: they do not by themselves identify bias, nonconvergence, singularity, or an incorrect random-effects structure. The Gaussian information calculation depends on known variance components and a normal response; real GLMM information depends on the full model and observed design. With few clusters, standard large-sample uncertainty approximations can also be unreliable. Interpret imbalance alongside the model assumptions, optimization diagnostics, and substantive design.

Check Your Understanding

Answer each question, then compare with the solutions.

Two studies each have 120 observations and 12 clusters. One has 10 observations per cluster; the other has 1 observation in 11 clusters and 109 in the last. Why should their random-intercept variance estimates not be assumed equally stable?
In the Gaussian illustration, what is the variance of a cluster mean for a cluster of size 4 when \(\tau^2=0.25\) and \(\sigma^2=1\)?
Why does a cluster with 91 observations not count as 91 independent realizations of a clinic random intercept?
Does convergence of an optimizer prove that the variance estimate is insensitive to unequal cluster sizes? Explain.

Solutions. (1) The total sample size and number of clusters match, but the second allocation concentrates within-cluster information in one group. Contributions to variance estimation can therefore be much more uneven, so sensitivity and information should be checked. (2) The variance is \(\tau^2+\sigma^2/4=0.25+1/4=0.50\). (3) The 91 outcomes share one latent clinic intercept; conditional observations can improve estimation of that clinic's response pattern, but they do not supply 91 independent draws from the distribution of clinic intercepts. (4) No. Convergence indicates that the numerical procedure met its stopping criteria for the fitted objective. Sensitivity to influential clusters is a separate design and model-dependence question.