Imputing Missing Level-1 Values

Why Level-1 Imputation Must Respect Clusters

When a value is missing for one person in a clinic, school, or other cluster, information from that person's cluster may be relevant to the missing value. At the same time, people in the same cluster are not generally independent: they may share unmeasured features that make their outcomes more alike. An imputation model that treats every row as unrelated can therefore produce values with an unrealistic pattern of within-cluster variation.

The task here is to impute a missing level-1 value: a measurement recorded for individuals nested within clusters. The cluster identifier is retained as part of the data structure, and relevant level-2 variables—variables that are constant for all individuals in a cluster—can help predict the missing level-1 value. This is distinct from imputing a missing cluster-level value.

Definition: A level-1 imputation model is a conditional model for an individual-level variable given relevant observed data. In nested data, a cluster-aware model represents dependence among individuals in the same cluster, commonly through a cluster-specific random intercept. Each imputed value remains attached to the original individual's cluster.

A Cluster-Aware Conditional Model

For a continuous level-1 target \(Y_{ij}\), a useful starting point is a two-level normal regression model:

Y_{ij} = \beta_0 + \beta_W(X_{ij}-\bar X_j) + \beta_B\bar X_j + \gamma Z_j + u_j + e_{ij}.

Here \(i\) indexes individuals and \(j\) indexes clusters; \(X_{ij}\) is an individual-level predictor; \(\bar X_j\) is its cluster mean; and \(Z_j\) is a level-2 predictor. The term \(u_j\) is a random intercept for cluster \(j\), and \(e_{ij}\) is an individual residual. The within-cluster term \(X_{ij}-\bar X_j\) distinguishes an individual's deviation from their cluster's mean from the between-cluster information in \(\bar X_j\).

A standard version assumes \(u_j\sim N(0,\tau^2)\), \(e_{ij}\sim N(0,\sigma^2)\), and that the random intercepts and individual residuals are mutually independent, conditional on the model and predictors. It also assumes the conditional mean and variance structure are adequate for the data. For a missing \(Y_{ij}\), the imputation model should include predictors needed to preserve its relationships with the variables in the substantive analysis, including relevant level-2 predictors. Missingness must be ignorable for the imputation model under the usual missing-at-random assumption, conditional on included observed data; that assumption cannot be verified from observed data alone.

In chained equations, the model is a conditional imputation step within an iterative sequence. Parameters and cluster effects should be drawn to represent uncertainty, followed by a predictive draw for the missing value. An imputed value is therefore not just a fitted mean. Its draw must also reflect appropriate residual variation. For a missing predictor rather than an outcome, its conditional model should include the outcome and other variables needed to preserve associations in the analysis.

Worked Example: Imputing a Missing Clinic Outcome

Suppose a hypothetical clinic has four patients. Their observed baseline severity values \(X_{ij}\) are 4, 6, 8, and 8; the fourth patient's outcome \(Y_{ij}\) is missing. The clinic's staffing score is \(Z_j=4\). Assume the current model draw has \(\beta_0=50\), \(\beta_W=2\), \(\beta_B=1\), \(\gamma=-0.5\), a clinic random-intercept draw \(u_j=0.7\), and an individual residual draw \(e_{ij}=1.4\).

The cluster mean uses every patient's observed \(X\), including the patient whose \(Y\) is missing: \( \bar X_j=(4+6+8+8)/4=6.5 \). The missing outcome's within-cluster deviation is \(8-6.5=1.5\).

The fixed part of the predictor is \( \eta_{ij}=50+2(8-6.5)+1(6.5)-0.5(4)=50+3+6.5-2=57.5 \). Adding the stated random-intercept and residual draws gives \( Y_{ij}^{\mathrm{imp}}=57.5+0.7+1.4=59.6 \). Thus this illustrative imputed outcome is 59.6 score units. In an actual imputation run, the model parameters and cluster effect would be drawn to reflect their uncertainty; the numbers here represent one hypothetical draw, not a fixed prediction rule.

Dependence and Predictive Variation

The role of the random intercept can be seen directly from the model. For two different individuals \(i\) and \(k\) in the same cluster, the shared \(u_j\) contributes to both outcomes. Under the assumptions above,

\operatorname{Var}(Y_{ij}\mid X,Z)=\tau^2+\sigma^2, \qquad \operatorname{Cov}(Y_{ij},Y_{kj}\mid X,Z)=\tau^2, \qquad \operatorname{Corr}(Y_{ij},Y_{kj}\mid X,Z) =\frac{\tau^2}{\tau^2+\sigma^2}.

The last ratio is the intraclass correlation under this random-intercept model. It expresses the proportion of conditional outcome variance attributable to differences between clusters. The result follows because the two individuals share the same random intercept, while their individual residuals are independent: the covariance is \(\operatorname{Var}(u_j)=\tau^2\). Each individual's variance is the sum of the independent random-intercept and residual variances, \(\tau^2+\sigma^2\).

Worked Example: Quantifying the Effect of Clustering

Suppose a hypothetical fitted model has between-cluster variance \(\tau^2=0.64\) and within-cluster residual variance \(\sigma^2=1.44\), both in squared outcome units. Its total conditional variance is \(0.64+1.44=2.08\) squared units, and its conditional standard deviation is \(\sqrt{2.08}=1.442\) outcome units.

The intraclass correlation is \( \rho=0.64/(0.64+1.44)=0.64/2.08=0.3077 \). So, under this model, about 30.8% of the conditional variance is between clusters. If the random intercept were ignored and only the residual standard deviation \(\sqrt{1.44}=1.200\) were used to describe predictive variation, the shared cluster component would be omitted. The difference matters for imputed values as well as for uncertainty summaries.

Choosing Predictors and Updating the Chain

The imputation model should preserve the structure relevant to the analysis, rather than merely predict the target as accurately as possible. In practice, consider the following:

Retain the cluster identifier. It determines which individuals share a cluster effect. Do not shuffle, relabel inconsistently, or impute it as if it were an ordinary numeric variable.
Include relevant level-2 predictors. If a cluster characteristic predicts the level-1 target or belongs in the substantive analysis, include it in the level-1 conditional model.
Represent important within- and between-cluster information. When a level-1 predictor has distinct within- and between-cluster associations, a model may need its individual deviation and cluster mean, rather than a single undifferentiated coefficient.
Respect the target's scale. A normal model is not automatically suitable for a binary, ordinal, nominal, or count target. The conditional distribution should match the variable type and its plausible values.
Update derived quantities consistently. If a model uses a cluster mean involving a variable that is itself imputed, calculate that mean from the current completed data at the appropriate iteration. Do not freeze a mean that omits imputed values.

For a continuous level-1 predictor \(X_{ij}\) with a missing value, the conditional model may include the outcome \(Y_{ij}\), observed individual-level covariates, relevant level-2 predictors, and a cluster effect. Including the outcome can be important for preserving the association between that predictor and the outcome. The model must still be plausible for \(X\) conditional on those variables; simply copying the outcome model with the variable names exchanged is not generally justified.

Worked Example: A Missing Level-1 Predictor

In a hypothetical study, an individual's continuous exposure \(X_{ij}\) is missing. The outcome \(Y_{ij}=10\) and cluster staffing score \(Z_j=3\) are observed. Suppose the conditional imputation model for \(X\) has the form \(X_{ij}=\alpha_0+\alpha_1Y_{ij}+\alpha_2Z_j+v_j+r_{ij}\). For one hypothetical draw, let \(\alpha_0=2\), \(\alpha_1=0.3\), \(\alpha_2=-0.2\), \(v_j=-0.1\), and \(r_{ij}=0.5\).

The conditional fixed part is \(2+0.3(10)-0.2(3)=2+3-0.6=4.4\). Adding the cluster-effect and individual-residual draws gives \(X_{ij}^{\mathrm{imp}}=4.4-0.1+0.5=4.8\) exposure units. This calculation illustrates why the outcome, level-2 information, and shared cluster structure can all matter. The chosen conditional model and its distributional assumptions must be checked for the actual target variable.

Diagnostics, Alternatives, and Limits

A cluster-aware model can still be misspecified. Inspect chain traces and summaries of imputed values, and compare observed and imputed distributions overall and across clusters. Check whether the imputation reproduces plausible within-cluster spread and whether level-2 predictors have sensible relationships with the imputed values. Look for impossible values, strong residual patterns, and clusters for which the model systematically predicts poorly. If an imputed cluster mean is used, verify that it is computed from the current values for all relevant individuals.

The main alternatives have different trade-offs. A single-level imputation model ignores the shared cluster component and can misrepresent within-cluster dependence. Cluster indicators can allow a separate intercept for each observed cluster, but can be unwieldy and do not provide the same partial pooling as a random-intercept model. A joint multilevel model provides another way to represent the nested variables together; chained equations instead specify conditional models variable by variable. Neither approach removes the need to choose suitable variables, distributions, and dependence structures.

A random-intercept model is only a starting point. It can be inadequate if within-cluster associations vary in ways the model does not capture, if the conditional distribution is substantially non-normal, or if relevant predictors are omitted. It also relies on enough information to estimate its cluster and residual variation. These are reasons to diagnose and revise the model, not reasons to assume that adding a cluster identifier alone has solved the problem.

Common Pitfall

Common mistake: “The outcome is missing for one person, so that person's predictor values should be left out when calculating the cluster mean.”

Correction: Missingness in \(Y_{ij}\) does not make an observed \(X_{ij}\) missing. If \(\bar X_j\) is defined over the cluster's individuals, include every observed \(X_{ij}\), including the person with missing \(Y_{ij}\). If values of \(X\) are themselves imputed, compute any derived mean from the current completed values as specified by the model.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A cluster has observed predictor values 2, 4, and 6. The third individual's outcome is missing. What is the predictor mean if all three predictor values are observed?
In a random-intercept model, \(\tau^2=0.25\) and \(\sigma^2=0.75\). Calculate the intraclass correlation and interpret it.
A missing level-1 exposure is associated with the outcome and with a cluster-level predictor. Name two features the conditional imputation model should consider.
Why can a single-level regression imputation fail to reproduce the dependence among individuals in the same cluster?

Solutions. (1) The outcome's missingness does not remove the observed predictor: \(\bar X_j=(2+4+6)/3=4\). (2) \(\rho=0.25/(0.25+0.75)=0.25\). Under the stated random-intercept model, 25% of the conditional variance is between clusters. (3) Include the outcome if needed to preserve its association with the exposure, and include the relevant level-2 predictor; also represent shared cluster dependence when appropriate. (4) A single-level model omits the shared cluster component \(u_j\), so its draws do not reproduce the covariance \(\tau^2\) between distinct individuals in the same cluster.