Heterogeneous Within-Cluster Variances

Why Within-Cluster Variance Matters

In clustered data, observations in the same clinic, school, or other group may have different expected values, but their variability around those expected values can differ too. A common residual variance assumes that, after accounting for the modeled mean and cluster structure, the remaining spread is the same throughout the data. That assumption can be a poor fit: measurements may be much more variable in some clusters, or at some values of a covariate, than in others.

For imputation, the consequence is direct. The mean model determines where missing values are centered; the residual variance determines how far the imputed values can vary around that center. A model with an adequate mean but an inadequate variance can therefore produce imputations that look plausible on average while being too tightly concentrated in high-variability groups and too dispersed in low-variability groups.

Definition: Within-cluster residual variance is the variance of level-1 observations around their modeled conditional means, within a cluster. It is distinct from between-cluster variance, which describes variation in cluster-level means or random effects. Heterogeneous within-cluster variances occur when the level-1 residual variance differs across clusters or as a function of observed covariates.

A Multilevel Model with Unequal Residual Variances

Let \(Y_{ij}\) be a continuous variable for individual \(i\) in cluster \(j\), with \(X_{ij}\) denoting predictors in the conditional mean. A useful starting model is

Y_{ij}=X_{ij}^{\mathsf T}\beta+u_j+e_{ij},\qquad u_j\sim N(0,\tau^2),\qquad e_{ij}\mid j,Z_{ij}\sim N(0,\sigma_{ij}^2).

Here \(u_j\) represents a cluster-level deviation in the mean, while \(e_{ij}\) is the level-1 residual. The residuals are assumed conditionally independent across individuals given the modeled cluster effect and predictors. To allow their variances to differ, one option is a variance regression:

\log(\sigma_{ij}^2)=Z_{ij}^{\mathsf T}\gamma.

The logarithm ensures that the modeled variance is positive. If variability differs between a small number of known groups, a simpler specification uses a separate variance for each group, such as \(\sigma_A^2\) and \(\sigma_B^2\). A cluster-level group indicator can define those groups; a continuous predictor may instead enter \(Z_{ij}\). The mean model and variance model have different jobs: predictors of the mean belong in \(X_{ij}\), and predictors of the residual spread belong in \(Z_{ij}\). A variable can appear in both.

These models rely on substantive and data-support assumptions. The chosen mean and variance predictors must describe relevant differences; the conditional residual distribution should be reasonably represented by the model; and each estimated variance pattern must be informed by enough observed values. For an imputation model, the assumed missingness mechanism must also be defensible conditional on the variables included. In particular, a cluster-specific variance cannot be learned reliably from a cluster with very few observed outcomes. An indicator for every cluster is not automatically a sound variance model.

Worked Example: Estimating Two Group-Specific Variances

Suppose a hypothetical study has two groups, with four observed residuals in each. Assume the conditional means have already been specified correctly and are known for this calculation. The residuals are \(-1,1,-1,1\) in group A and \(-3,3,-3,3\) in group B. We compare a separate-variance model with a common-variance model.

For normally distributed residuals with known conditional means, the maximum-likelihood variance estimate in a group is its sum of squared residuals divided by its number of observations. In group A, the sum is \(1+1+1+1=4\), so \(\hat{\sigma}_A^2=4/4=1\). In group B, the sum is \(9+9+9+9=36\), so \(\hat{\sigma}_B^2=36/4=9\).

Under a common-variance model, the two groups' squared residuals are pooled. The estimate is \(\hat{\sigma}^2=(4+36)/(4+4)=40/8=5\). Thus the fitted residual standard deviations are 1 in group A and 3 in group B under the separate-variance model, compared with \(\sqrt{5}\approx2.236\) in both groups under the common-variance model. The common model has not changed the specified means; it has compromised between the two very different spreads.

Why the Common Estimate Pools the Spreads

The pooling in the example follows from the normal likelihood. Let \(S_g\) be the sum of squared residuals in group \(g\), and let \(n_g\) be the number of observed residuals. With group-specific variances and known means, the log likelihood, apart from no omitted data-dependent terms, is

\ell=-\frac{1}{2}\sum_g\left[n_g\log(2\pi\sigma_g^2)+\frac{S_g}{\sigma_g^2}\right].

Differentiating with respect to \(\sigma_g^2\) and setting the result equal to zero gives

-\frac{n_g}{2\sigma_g^2}+\frac{S_g}{2(\sigma_g^2)^2}=0 \quad\Longrightarrow\quad \hat{\sigma}_g^2=\frac{S_g}{n_g}.

If the model instead imposes one common variance, the same calculation uses the sum of all groups' squared residuals and the total number of observations:

\hat{\sigma}^2=\frac{\sum_g S_g}{\sum_g n_g}.

So the common estimate is a sample-size-weighted average of the group-specific maximum-likelihood estimates. It is not generally the right variance for any particular group. This derivation uses known conditional means to keep the calculation focused; in practical fitting, means are estimated too, and residual variance estimates must reflect that estimation.

Worked Example: Predictive Spread for a Missing Value

Continue the hypothetical two-group example. Suppose a missing outcome in each group has conditional mean 10. For a simple comparison, treat the fitted mean and variance as known and use the normal model's central 95% predictive interval, mean plus or minus \(1.96\) residual standard deviations.

Under the heterogeneous model, group A has standard deviation 1, so its interval is \(10\pm1.96(1)=[8.04,11.96]\). Group B has standard deviation 3, giving \(10\pm1.96(3)=[4.12,15.88]\). These intervals describe residual predictive spread conditional on the assumed parameters, not uncertainty about a population parameter.

Under the common model, the standard deviation is \(\sqrt{5}\approx2.236\), so both groups receive the interval \(10\pm1.96(2.236)\), or approximately \([5.62,14.38]\). In group A this is much wider than the model's group-specific interval; in group B it is too narrow. If missing outcomes are imputed using the common variance, the model therefore overstates residual variation for A and understates it for B.

Variance Differences Can Change Tail Probabilities

The impact is not limited to interval widths. When an analysis depends on thresholds, an incorrect spread can also distort the proportion of imputations above or below a clinically or substantively meaningful value. The following calculation keeps the mean model fixed to isolate this effect.

Worked Example: A Threshold Probability in Two Groups

Suppose a hypothetical outcome has conditional mean 12 for an individual with specified covariates. The residual standard deviation is 1 in group A and 3 in group B. A missing outcome belongs to group B, and interest concerns the probability that its value exceeds 15.

Under the group-specific model, \(Y\mid B\sim N(12,9)\), where the second parameter is the variance. Standardizing gives \(z=(15-12)/3=1\), so \(P(Y>15\mid B)=1-\Phi(1)\approx0.1587\).

The common-variance estimate from the equal-sized example is 5, giving a standard deviation of \(\sqrt{5}\). Under that model, \(z=3/\sqrt{5}\approx1.342\), and \(P(Y>15)\approx1-\Phi(1.342)\approx0.0898\). The common-variance model assigns substantially less probability to a value above 15 in this high-variance group. These are model-based probabilities under the stated normal distributions, not claims about an actual study.

Diagnostics and Practical Choices

Variance differences should be investigated after considering the conditional mean. If the mean model omits a predictor or nonlinear pattern, residuals can appear more dispersed in one group simply because their centers were modeled poorly. Useful checks include residual plots against cluster or variance predictors, group-specific residual standard deviations, and comparisons of observed and imputed distributions within those groups. For a variance regression, check whether the fitted spread changes in the directions supported by the residuals and whether predictions are driven by a few observations.

In multiple imputation, the variance model must describe the distribution used to generate missing values, not merely improve a residual diagnostic. If the model fits a common variance where spread truly differs, it can make imputations systematically too concentrated for some groups and too diffuse for others. That can distort group-specific summaries, associations, and uncertainty in later analyses.

The main alternatives have different trade-offs:

Approach	When it may be useful	Main caution
Common residual variance	When diagnostics and subject knowledge support a shared spread	Can misstate imputation spread when groups differ materially
Separate variance by a small number of groups	When group membership is known and each group has enough observed outcomes	Estimates can be unstable with small or sparse groups
Variance regression using covariates	When spread changes systematically with measured predictors	A misspecified variance relationship may extrapolate poorly
Outcome transformation	When a transformation gives a more nearly constant spread and suits the analysis	Changes the modeled scale; back-transformation and interpretation require care

A practical MICE model should use the simplest variance structure that captures defensible, material differences. Group-specific variances require enough observed values within each group; a smooth variance regression can share information across covariate values but imposes a functional form. With few observations or clusters, apparent differences may be sampling noise. Treating estimated variances as exact also ignores their uncertainty; this is especially consequential when the variance pattern is weakly informed. The model should not claim precision that the available observed outcomes cannot support.

Common Pitfall

A frequent mistake is to add cluster indicators to the mean model and assume that this automatically handles different within-cluster variances. Cluster indicators can allow cluster means to differ, but they do not by themselves permit residual spreads to differ. The correction is to ask two separate questions: does the expected outcome change across clusters, and does the residual variance change across clusters or covariate groups? A good mean fit does not establish a good variance fit.

Another warning sign is trying to estimate an unrestricted variance for every cluster even when many clusters have only a few observed values. Such estimates can be extremely noisy, and clusters with no observed outcomes provide no direct residual information. In that setting, use a variance structure supported by available predictors and observed data, and report the limitations of that structure rather than treating unstable estimates as reliable.

Key takeaway: Heterogeneous within-cluster variance concerns the spread of level-1 residuals, not just differences in cluster means. If the imputation model assumes a common spread when residual variability differs materially, its imputations can be too narrow in high-variance groups and too wide in low-variance groups.

Check Your Understanding

Answer each question, then compare your reasoning with the brief solutions below.

In a multilevel model, what is the difference between a random intercept variance and a level-1 residual variance?
Two equally sized groups have residual sums of squares 18 and 50, with 5 observations in each. What are the separate maximum-likelihood variance estimates and the common-variance estimate, assuming the conditional means are known?
A group has residual standard deviation 2, but an imputation model uses a common standard deviation of 1. What happens to the imputed residual spread for that group?
Why is a separate residual variance for every cluster often a poor choice when some clusters have very few observed outcomes?
What should be checked before attributing larger residual spread in one group to heteroscedasticity?

Solutions. 1. A random intercept variance describes variability in cluster-level means; a level-1 residual variance describes variability around the modeled mean within a cluster. 2. The estimates are \(18/5=3.6\) and \(50/5=10\); the common estimate is \((18+50)/(5+5)=68/10=6.8\). 3. The model makes imputations too concentrated: its residual standard deviation is half the group-specific value. 4. Few observations provide little information about a cluster's residual spread, so separate estimates can be unstable; clusters with no observed outcomes provide no direct residual information. 5. Check whether the conditional mean is adequately modeled, since mean misspecification can create an apparent difference in residual spread.