Why the School a Student Attends Can Matter
Suppose a data set records students’ test scores and study hours, but some study-hour values are missing. Students from the same school may share teachers, schedules, resources, or local conditions. As a result, knowing a student’s school can help predict that student’s study hours. If an imputation procedure treats all students as interchangeable, it can replace a missing value with one that is plausible for the overall student population but not for that student’s school.
The same issue affects more than the individual replacement. Imputed values can change apparent differences between schools, and ignoring dependence can make estimates look more precise than the data support. These problems matter whenever the analysis or imputation concerns measurements from students nested in schools.
For a simple illustration, write a measurement for student \(i\) in school \(j\) as \(X_{ij}=\mu+u_j+e_{ij}\). Here \(\mu\) is an overall mean, \(u_j\) represents a school-specific departure from that mean, and \(e_{ij}\) represents a student-specific departure within the school. This is a conceptual decomposition, not a prescription for how to fit an imputation model. If schools differ, a student’s expected value given the school is generally not the same as the overall expected value.
How Ignoring Schools Can Distort Replacements
A pooled imputation model that omits school information effectively uses a distribution averaged over schools. When school distributions differ, that pooled distribution can be too broad for any one school and can give the wrong conditional mean for a student in a particular school. In MICE, an incomplete variable is imputed conditionally on other variables in the chained system; if school structure predicts that variable but is omitted, the conditional distribution used for replacement can be misspecified.
This is a modeling issue, not a claim that every student in a school has the same value. A school-specific prediction should still allow student-to-student variation. The goal is to preserve the distinction between variation among students in one school and variation across schools, rather than treating the two sources as interchangeable.
Worked Example: A Missing Study-Hours Value in One School
Hypothetical data. Suppose study hours per week have a within-school standard deviation of 5 hours. Two equally sized schools have mean study hours of 40 and 60 hours per week. One student in the first school has a missing value. Assume the school label is known, the school means are appropriate conditional expectations for this illustration, and the observed students represent those distributions.
A pooled procedure that ignores school uses the equally weighted overall mean:
Its expected replacement for the missing student is therefore 50 hours per week. A procedure that conditions on the student’s school has expected replacement 40 hours per week. The difference in expected replacements is \(50-40=10\) hours per week. The pooled value is 10 hours above the school-specific expectation, even though no individual student is known to study exactly 40 hours.
The pooled distribution also includes differences between schools. The variance of the two school means around 50 is \((40-50)^2/2+(60-50)^2/2=100\) square hours. Adding the within-school variance, \(5^2=25\), gives a pooled variance of \(125\) square hours in this simplified equal-size mixture. Its standard deviation is \(\sqrt{125}=11.18\) hours. That is much larger than the within-school standard deviation of 5 hours. A pooled draw can thus be both centered incorrectly for the first school and much more variable than values within that school.
The example shows why cluster membership can be predictive even when the incomplete variable is measured at the student level. It also shows why “more variable” does not automatically mean “more appropriate”: some of the pooled spread reflects differences between schools, which may not describe variation among students within a given school.
Why Between-School Variation Can Be Attenuated
Suppose each school has a different mean for an incomplete variable. If missing values in every school are replaced from a common pooled distribution, the replacements tend to pull each school’s completed mean toward the overall mean. This can make schools appear more similar than they are. The derivation below isolates this shrinkage in expectation; it does not claim that every realized imputed data set will have exactly these means.
Worked Example: Pooled Replacements Pull School Means Together
Hypothetical data. Consider four schools with study-hours means of 40, 50, 60, and 70 hours per week. Suppose each school has the same number of students and 50% of values are missing in each school. Assume, for this illustration, that observed values represent their school means in expectation and that pooled replacements have expected value equal to the overall mean. These assumptions let us isolate the effect of using a common replacement expectation.
The overall mean is \((40+50+60+70)/4=55\) hours per week. With \(q=0.5\), the expected completed mean for the first school is \(0.5(40)+0.5(55)=47.5\) hours per week. The corresponding expected means for the other schools are \(52.5\), \(57.5\), and \(62.5\) hours per week.
Before imputation, the variance across the four school means, treating the four schools as the group of interest, is \([(-15)^2+(-5)^2+5^2+15^2]/4=500/4=125\) square hours. The expected completed school means have deviations from 55 of \(-7.5,-2.5,2.5,7.5\). Their variance is \([(-7.5)^2+(-2.5)^2+2.5^2+7.5^2]/4=125/4=31.25\) square hours. This equals \((1-0.5)^2(125)=31.25\), as the derivation predicts.
This calculation compares the variation across the expected school means. Individual imputed draws also vary, so a realized set of completed means will not necessarily have variance 31.25. The result nevertheless shows the direction of the structural distortion: a shared pooled expectation draws each school toward 55 and reduces the between-school differences in expectation.
Why Standard Errors Can Be Too Small
Students in one school do not necessarily contribute as much independent information as the same number of students sampled from many unrelated schools. If outcomes are positively correlated within schools, treating every student as independent generally understates the sampling variability of an overall mean. An imputation procedure that ignores this dependence can carry the same false-independence assumption into the completed data and make subsequent standard errors too small.
For a transparent calculation, assume \(J\) independent schools, each with \(m\) students, and equal marginal variance \(\sigma^2\). Assume any two students in the same school have correlation \(\rho\), while students in different schools are independent. For the overall mean of all \(n=Jm\) students, the variance is
The factor \(1+(m-1)\rho\) is the design effect under these assumptions. It equals 1 when \(\rho=0\), and exceeds 1 when within-school correlation is positive. This formula concerns the sampling variance of the mean under the stated cluster structure. It is not a universal multiplier for every coefficient or every multiple-imputation variance.
Worked Example: Comparing Independent and Clustered Standard Errors
Hypothetical data-generating assumptions. Suppose 100 students are sampled from 10 independent schools, with 10 students per school. A continuous outcome has marginal standard deviation \(\sigma=10\) points and within-school correlation \(\rho=0.20\). Assume equal school sizes, the same variance in each school, exchangeable within-school correlation, and a target that is the overall student mean.
If the 100 observations were independent, the standard error would be \(10/\sqrt{100}=1.000\) point. Under the clustered assumptions, the design effect is \(1+(10-1)(0.20)=1+1.8=2.8\). Thus the variance of the overall mean is \((10^2/100)(2.8)=2.8\) square points, and the standard error is \(\sqrt{2.8}=1.673\) points.
The independent-observation calculation is smaller by a factor of \(\sqrt{2.8}=1.673\). In this illustration, ignoring clustering would report a standard error of 1.000 rather than 1.673 points. This is a sampling-variance comparison, not a complete multiple-imputation analysis. In an actual analysis, uncertainty from missing values and uncertainty from the clustered sampling structure both need appropriate treatment.
Assumptions, Diagnostics, and Alternatives
Accounting for clusters is useful only when the imputation assumptions are also considered. The imputation variables and their relationships should be suitable for the target analysis; school structure should be included when it helps predict missing values or captures dependence. As in other MICE applications, an assumption such as missing at random is conditional on the information used: a missingness mechanism may be plausibly missing at random given school and observed student information, but not given student information alone. Including school does not establish that assumption or solve missing-not-at-random problems.
The numerical examples used deliberately simple conditions: equal school sizes, clear school differences, and straightforward correlation structures. Real data may have unequal numbers of students per school, very few schools, school-level predictors, or different relationships between predictors and the incomplete variable across schools. The simple design-effect formula and shrinkage calculation should not be applied mechanically in those settings.
Useful checks include comparing observed distributions across schools, examining whether school membership predicts the incomplete variable or its missingness, and comparing imputed with observed values within schools as well as overall. Inspect the school means and spreads across completed data sets: values that collapse toward the overall mean, or school-specific distributions that are implausibly broad, may indicate that the imputation has pooled distinct sources of variation. Convergence checks for the chained equations remain important, but convergence alone cannot show that the dependence structure is represented correctly.
| Approach | Potential benefit | Main caution |
|---|---|---|
| Pooled imputation that omits schools | Simple and can be adequate when cluster dependence is negligible for the imputation and analysis. | Can use the wrong conditional mean and mix within-school with between-school variation. |
| Imputation that includes school information | Can preserve school-related prediction and dependence more effectively. | Requires enough information to estimate school-related patterns; a simplistic treatment may still miss important structure. |
| Cluster-aware or multilevel imputation | Can represent dependence and variation at more than one level. | Requires appropriate assumptions and careful matching to the estimand and analysis. |
These are broad alternatives, not interchangeable recipes. A school indicator or a multilevel approach can each be more appropriate than pooling, depending on the number of schools, the data structure, and the scientific target. The key point here is that an imputation method must not discard dependence that matters for predicting missing values or for estimating uncertainty.
Common Mistake and Key Takeaway
Common mistake: “There are 100 students, so the standard error is the standard deviation divided by \(\sqrt{100}\), and a pooled imputation is adequate because it uses all students.” The correction is that the number of rows is not always the number of independent pieces of information. Shared school conditions can make students’ values dependent, and a pooled conditional distribution can replace a student’s value using variation that belongs to differences between schools. The imputation and the analysis should reflect clustering when it is relevant.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- Two equally sized schools have means of 30 and 50 units. Under a pooled mean replacement, what is the expected replacement for a missing value in the first school, and how far is it from that school’s mean?
- In the four-school example, what fraction of the original between-school variance remains in the variance of expected completed school means when \(q=0.25\)?
- With 8 students per school and within-school correlation \(\rho=0.10\), calculate the design effect under equal cluster sizes.
- Why can a pooled imputation distribution be too variable for a student whose school is known?
- Does including school in an imputation model prove that missingness is at random? Explain.
Solutions. (1) The pooled mean is \((30+50)/2=40\) units. The expected replacement is 40 units, which is \(40-30=10\) units above the first school’s mean. (2) The variance factor is \((1-q)^2=(1-0.25)^2=0.75^2=0.5625\), so 56.25% remains in the variance of expected completed school means under the stated illustration. (3) The design effect is \(1+(8-1)(0.10)=1+0.70=1.70\). (4) The pooled distribution can include between-school differences in addition to within-school spread. Those between-school differences need not describe variation among students within the known school. (5) No. Including school can make missing at random more plausible conditional on the included information, but it neither proves the assumption nor rules out dependence of missingness on unobserved values.