Why Sparse Observed Clusters Require Special Care
A level-2 covariate has one value per cluster: for example, a hospital-level staffing measure or a school-level policy indicator. If that covariate is missing for a cluster, it is missing for every individual in that cluster. A data set might contain hundreds of individual records while the covariate is observed in only a handful of clusters. Those records do not turn a handful of cluster-level measurements into hundreds of independent observations.
The practical challenge is not simply choosing a conditional distribution. It is determining what the observed clusters can support. A model with many predictors may look rich because it uses individual-level data, but information about the covariate's between-cluster variation still comes mainly from the clusters where it is observed. When those clusters are few, elaborate models can be unstable, and predictions for clusters unlike those observed may be mostly extrapolation.
A defensible imputation still needs the usual conditions for multiple imputation: the conditional imputation model must be adequate for the intended analysis, relevant observed information should be included, and the missingness assumptions must be plausible. Under a missing-at-random assumption conditional on variables in the imputation model, valid inference also depends on proper imputation: uncertainty in model parameters and in the missing values must be carried into the imputations. If missingness depends on the unobserved value even after conditioning on observed information, a standard MAR-based model alone does not resolve that concern.
Count Independent Cluster Information
The first diagnostic is a cluster-level missingness table: how many clusters have observed \(W_j\), how many do not, and how those groups differ in cluster size and in available cluster-level predictors? Also inspect whether observed and missing clusters overlap in the predictors that will be used to impute \(W_j\). If all observed clusters have low values of a predictor and all missing clusters have high values, the imputation model must extrapolate beyond its observed support.
Individual-level outcomes and covariates can be informative about a missing cluster's \(W_j\), but their use must respect clustering. One route is to include suitable cluster summaries or model-based information from the individual records. A second is to fit a multilevel conditional model that represents the individual records' dependence. Simply copying \(W_j\) across a cluster's rows and fitting an ordinary row-level regression as though those copies were independent is not a valid substitute: it artificially increases the apparent sample size for the level-2 relationship.
Worked Example: Counting the Actual Observations
Suppose four clusters have observed values of a level-2 covariate, \(2,4,6,\) and \(8\), and each cluster contains three individuals. The table therefore has 12 rows with values of \(W\), but only four independently observed cluster-level values.
The observed cluster mean is \(\bar{W}= (2+4+6+8)/4=5\). The sample variance across clusters is \(s^2=[(2-5)^2+(4-5)^2+(6-5)^2+(8-5)^2]/(4-1)=20/3=6.667\), so \(s=2.582\). The standard error of the mean across these four clusters is \(2.582/\sqrt{4}=1.291\).
A tempting but incorrect calculation uses that cluster-level standard deviation and divides by \(\sqrt{12}\), giving \(2.582/\sqrt{12}=0.745\). That calculation treats the three copies within each cluster as additional independent observations. They are repetitions of the same cluster value, so they add no independent observations of between-cluster variation. The relevant cluster-level count remains four.
This illustration assumes that the four observed clusters are independent draws from a common distribution and is about the uncertainty in their mean. In a real imputation, the model may condition on predictors and need not estimate a single common mean. The lesson remains: row count and cluster-level information are not interchangeable.
Build a Cautious Conditional Model
A level-2 imputation model should be formulated for the cluster-level variable, even when information from individual-level variables is used. Start with a small set of substantively necessary predictors: variables in the substantive analysis, strong available predictors of \(W_j\), and variables related to its missingness when appropriate. With sparse observed clusters, adding predictors simply because they are available can consume scarce information, create unstable coefficients, and yield extreme predictions.
For a continuous \(W_j\), a simple starting point might be a cluster-level regression \(W_j=\beta_0+\beta_1 Z_j+\epsilon_j\), where \(Z_j\) is an observed cluster-level predictor and the residuals are independent, normally distributed with mean zero and variance \(\sigma^2\). This model assumes a linear conditional mean, constant residual variance, and adequate predictor support for the clusters being imputed. With only a few observed clusters, these assumptions are difficult to assess; a simple model is not automatically a correct one.
When individual-level information matters, the model can incorporate carefully chosen cluster summaries or appropriately modeled individual-level information. A raw cluster mean may be noisier for a small cluster than for a large one, so treating all such summaries as equally precise can also mislead. The model and its uncertainty need to reflect how those summaries were generated. The earlier material on joint modeling and multilevel chained equations, and on building conditional models for multilevel FCS, provides the broader modeling framework; here the emphasis is on how little between-cluster information may be available.
Use proper draws rather than filling missing values with fitted conditional means. In each imputation, uncertainty in estimated model parameters and the residual or predictive variation should contribute to the missing \(W_j\). Shrinkage or regularization may stabilize a model with few observed clusters, but it does not create information: results can depend materially on the regularization assumptions. For categorical level-2 variables, the corresponding conditional model must respect the categories and their observed cluster counts; a category seen in very few clusters can lead to separation or highly uncertain predictions.
Why Prediction Uncertainty Can Be Large
A short calculation shows why predictions from sparse cluster data should not be reported as if they were known. Under the simple normal linear model above, let \(G\) be the number of observed clusters, let \(S_{ZZ}=\sum_{j:R_j=1}(Z_j-\bar{Z})^2\), and let \(s\) estimate the residual standard deviation. For a new cluster with predictor \(Z_0\), the prediction-error variance includes both uncertainty in the fitted mean and residual variation for the new cluster:
The \(1\) is the new cluster's residual variation. The other terms account for uncertainty in the estimated intercept and slope at \(Z_0\). Under independent normal residuals and the stated linear model, replacing \(\sigma\) with \(s\) gives the conventional prediction standard error. The corresponding prediction interval uses a \(t\) critical value with \(G-2\) degrees of freedom, provided \(G>2\). This result is a diagnostic illustration, not a claim that this particular model is appropriate for every imputation.
Worked Example: A Wide Prediction Interval From Four Clusters
Suppose four clusters have observed pairs \((Z_j,W_j)=(0,10),(1,13),(2,13),(3,16)\). Consider a straight-line normal model with independent residuals of constant variance. These assumptions are being used to illustrate the calculation; with only four clusters, they cannot be checked reliably.
The means are \(\bar{Z}=1.5\) and \(\bar{W}=13\). The slope is \(b_1=\sum(Z_j-\bar{Z})(W_j-\bar{W})/\sum(Z_j-\bar{Z})^2=9/5=1.8\), and the intercept is \(b_0=13-1.8(1.5)=10.3\). Thus, the fitted value at \(Z_0=2.5\) is \(\widehat{W}_0=10.3+1.8(2.5)=14.8\).
The fitted values for \(Z=0,1,2,3\) are \(10.3,12.1,13.9,15.7\). The residuals are \(-0.3,0.9,-0.9,0.3\), so the residual sum of squares is \(1.8\). There are \(4-2=2\) residual degrees of freedom, giving \(s=\sqrt{1.8/2}=\sqrt{0.9}=0.9487\). Also, \(S_{ZZ}=5\), and \((Z_0-\bar{Z})^2=1\). Therefore,
Using \(t_{0.975,2}=4.303\), the 95% prediction interval is \(14.8\pm4.303(1.142)\), or approximately \(9.89\) to \(19.71\) units of \(W\). Under the model assumptions, this interval reflects uncertainty about a new cluster's value, not just uncertainty about the fitted mean. Its width is a warning: four observed clusters provide weak information, even though a line can be calculated.
In MICE, a proper imputation would draw model parameters and a predictive value rather than assign every missing cluster the fitted value \(14.8\). The interval does not itself specify a complete MICE algorithm, but it makes clear why omitting parameter and predictive uncertainty would understate uncertainty.
Choosing Among Approaches and Checking Performance
The main alternatives differ in how they use the data and what assumptions they require. A parsimonious cluster-level regression is transparent when there are enough observed clusters to estimate its terms and the predictors cover the missing clusters. A simple distributional model can be a useful baseline when there are too few clusters to support several predictors, but it assumes little or no relevant predictor structure. A multilevel conditional model can use individual-level information while accounting for dependence, but it can still be weakly identified when only a few clusters inform the level-2 relationship. No choice removes the fundamental scarcity.
A row-level regression with repeated copies of \(W_j\) may appear to use all individuals, but it risks pseudo-replication unless its likelihood correctly accounts for cluster dependence and the level-2 sampling structure. Likewise, deterministic mean prediction is not a multiple-imputation substitute: it suppresses variation among plausible missing values. If predictor support for missing clusters lies outside the observed range, any regression-based estimate may be driven by its functional form rather than by direct evidence.
After fitting the imputation model, inspect chain traces and distributions of imputed \(W_j\) across chains and imputations. Check for implausible values, extreme coefficients, unstable category probabilities, and strong sensitivity to small changes in model specification. Compare observed and imputed cluster-level distributions, but do not expect them to match exactly when missingness depends on observed predictors. Also compare relevant relationships with the individual-level variables used in the substantive analysis; a plausible marginal distribution alone does not establish that the conditional model preserves those relationships.
- Too few observed clusters for the fitted model. Large coefficient changes, wide uncertainty, or unstable imputations suggest that the model is trying to estimate more than the observed cluster count supports.
- Poor overlap. Predictions for missing clusters far outside observed predictor ranges are extrapolations. Report that limitation and examine simpler or alternative models.
- Incorrect level of analysis. If repeated rows are counted as independent evidence about \(W_j\), the apparent precision can be misleading.
- Convergence mistaken for adequacy. Stable chains do not prove that a model is well specified or that missing-cluster predictions are supported by data.
The number of clusters is not a universal cutoff for whether imputation is possible. The necessary amount of information depends on the number of model parameters, predictor variation, missingness pattern, and how the imputed variable relates to other observed data. When the available clusters cannot support a credible model, that is a limitation to disclose, not a problem that can be fixed by counting more individual rows.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A covariate is observed in 7 of 30 clusters, each containing 20 individuals. How many observed cluster-level values inform its between-cluster distribution?
- Why is a row-level regression using 140 repeated observed values not automatically equivalent to a model with 140 independent observations?
- In the prediction-interval formula, what does the leading \(1\) represent, and what happens to prediction uncertainty as \(Z_0\) moves far from \(\bar{Z}\)?
- What is one reason a model with many predictors may be a poor choice when only a few clusters have observed covariate values?
- Why should missing cluster values not simply be replaced by fitted conditional means in multiple imputation?
Solutions. (1) There are 7 observed cluster-level values; the 140 individual records do not change that count. (2) The repeated values share the same cluster-level value and are dependent by construction, so treating them as independent can create pseudo-replication. (3) The leading \(1\) is residual variation for the new cluster. As \(Z_0\) moves away from the observed predictor mean, the leverage term \((Z_0-\bar{Z})^2/S_{ZZ}\) increases, widening the prediction uncertainty. (4) Sparse clusters may not support stable estimation of many coefficients, especially if predictors are correlated or have little variation among observed clusters. (5) Fitted means omit variation in both model parameters and the unobserved cluster's predictive value, making imputations too certain and understating uncertainty.