From Missing Values to a Trial Estimate
In a cluster-randomized trial, treatment is assigned to groups such as clinics or schools, while measurements may be collected from individuals within each group. Missingness at either level can affect the treatment estimate, and an imputation that ignores clustering can misrepresent how much information the trial contains. A complete MICE analysis therefore starts by mapping variables to their levels and ends by checking that the completed datasets still support the intended analysis.
The hypothetical trial used here randomizes 12 clinics, six to each treatment arm, and enrolls 10 patients per clinic. The outcome is a symptom score in points at follow-up; lower scores indicate fewer symptoms. Baseline symptom severity is measured for each patient, and a deprivation score is recorded once per clinic. The target is the adjusted mean treatment difference at follow-up.
This analysis assumes that the cluster identifiers and treatment assignments are correctly recorded, treatment is constant within each clinic, and clusters are independent under the trial design. For the missing-data analysis, the key identifying assumption is that missing values are missing at random conditional on variables included in the imputation models. That assumption cannot be established from observed data alone. The conditional models must also be sufficiently well specified, and the imputation and analysis models must preserve the associations relevant to the treatment estimate.
Map the Trial Before Choosing Models
Worked Example: Classifying Variables and Missingness
Suppose baseline severity is missing for 22 of the 120 patients, and follow-up outcome is missing for 10 patients. Four patients have both values missing. Deprivation and treatment assignment are fully observed. Identify the levels of the variables and summarize the missingness.
There are \(12\times10=120\) patient records. Baseline severity is individual-level, and its missing proportion is \(22/120=0.1833\), or 18.33%. Follow-up outcome is also individual-level, with missing proportion \(10/120=0.08333\), or 8.333%. Since four patients are missing both values, the number of distinct patients missing at least one of these variables is \(22+10-4=28\), or \(28/120=0.2333\), 23.33%.
Treatment assignment and deprivation are cluster-level: each has 12 values, not 120 independent values. The clinic identifier is a design variable, not an unknown quantity to impute. Treatment should likewise not be imputed; its known randomized allocation is part of the design. Repeating a clinic’s deprivation value on each patient row does not create 10 separate deprivation observations.
The reported missingness counts do not establish the missing-at-random assumption. They are useful for describing the pattern, but the analysis must consider whether observed predictors of missingness and of the incomplete measurements are represented in the imputation models.
A useful working table lists each variable’s level, missingness, role in the substantive analysis, and imputation method. It prevents a common data-structure error: treating a cluster-level value repeated down the patient file as if it varied independently by patient. The model must preserve that each clinic contributes one treatment assignment and one deprivation score.
Specify the Chained Conditional Models
For this example, use multilevel normal conditional models for the two incomplete continuous variables. Each model includes treatment, deprivation, relevant observed predictors, and a clinic-level random intercept. The baseline-severity model also includes follow-up outcome; the outcome model includes baseline severity. This allows the observed association between baseline and follow-up measurements to contribute to imputations. If the planned analysis includes an interaction or nonlinear term, the imputation models need terms that preserve it as well.
Here \(i\) indexes patients, \(j\) indexes clinics, \(X_{ij}\) is baseline severity, \(Y_{ij}\) is follow-up severity, \(A_j\) is treatment assignment, and \(Z_j\) is clinic deprivation. The clinic effects \(u_j^{(X)}\) and \(u_j^{(Y)}\) represent residual within-clinic dependence in the respective conditional models. In an actual implementation, the model and parameter draws must be suitable for proper multiple imputation; simply substituting conditional means would fail to represent uncertainty about missing values.
These equations describe conditional models, not a guarantee that any chosen set of conditionals defines a coherent joint distribution. Their adequacy depends on the variable types, the pattern of missingness, and the target analysis. For example, a continuous normal model is inappropriate for a binary variable, and a model that omits a treatment-by-baseline interaction may fail to preserve that interaction. The trial’s cluster structure must be represented in the imputation and in the eventual analysis.
Worked Example: One Conditional Imputation Draw
Consider a patient with missing baseline severity in a clinic assigned to treatment. For illustration, suppose a fitted conditional model, given this patient’s observed follow-up score, treatment, deprivation, and clinic information, gives a predictive distribution with mean 12.4 symptom points and standard deviation 3.0 points. A simulated standardized residual for this draw is \(-0.5\). Find the resulting imputed value.
The draw is computed as the conditional mean plus the predictive standard deviation times the simulated standardized residual:
Thus this draw imputes 10.9 points. It is not a claim that the patient’s unobserved value is known to be 10.9. A different random draw, and parameter draws that represent model uncertainty, would generally produce a different completed dataset. In practice the model assumptions should be checked against observed values and the trial’s data structure before relying on such draws.
The chain cycles through incomplete variables, drawing values conditional on the current completed values of the other variables. After initial iterations, retain completed datasets from later iterations, with adequate separation or other safeguards against excessive dependence. The number of retained datasets should reflect the amount of missing information and the required precision of the pooled estimates.
Fit the Planned Analysis and Pool Its Results
For each completed dataset, fit the same prespecified trial analysis. In this example, the analysis estimates the adjusted treatment difference in follow-up symptom score, accounting for baseline severity, clinic deprivation, and the clustering by clinic. The treatment coefficient is expressed in symptom-score points. Use a small-sample complete-data degrees-of-freedom method appropriate to the cluster-randomized design; the 120 patient rows do not create 120 independent treatment assignments.
The within-imputation component represents uncertainty in each completed-data analysis; the between-imputation component represents variation in the estimates across completed datasets. The factor \(1+1/m\) accounts for the finite number of imputations. When the complete-data degrees of freedom are limited, a small-sample adjustment to the degrees of freedom can materially affect the confidence interval.
Worked Example: Pooling a Clustered-Trial Treatment Estimate
Suppose five completed datasets produce a mean adjusted treatment estimate \(\bar{Q}=4.20\) points, average within-imputation variance \(\bar{U}=3.388\) points squared, and between-imputation variance \(B=0.560\) points squared. Positive values indicate higher, and therefore worse, follow-up symptom scores in the treatment arm. Use a complete-data degrees of freedom of 10, as specified by the small-sample method for this hypothetical cluster-randomized analysis.
The total variance and pooled standard error are
For the old Rubin degrees of freedom, first calculate the relative increase in variance:
Because the complete-data degrees of freedom are only 10, use the Barnard–Rubin small-sample adjustment. The fraction of total variance attributable to missingness is
The observed-data degrees of freedom are \(df_{\mathrm{obs}}=[(df_{\mathrm{com}}+1)/(df_{\mathrm{com}}+3)]df_{\mathrm{com}}(1-\lambda)\). Thus
The two-sided 95% \(t\) critical value with approximately 6.74 degrees of freedom is 2.383. The confidence interval is therefore
Under the stated imputation and analysis assumptions, the pooled estimate is 4.20 points higher in the treatment arm, but the 95% confidence interval ranges from 0.602 points lower to 9.002 points higher. Since the interval includes zero, these results do not establish a difference at the two-sided 5% level. They also do not show that there is no treatment effect: the interval includes both modest benefit and potentially important harm.
The degrees of freedom are not a cosmetic detail. Using the old Rubin value of about 146 would give a much smaller critical value than the adjusted value based on the limited number of clusters. The trial’s design and complete-data inference method determine the appropriate degrees of freedom; one should not use the patient count as a substitute for the number of randomized clusters.
Check the Completed Datasets
Diagnostics should examine both the imputations and the pooled analysis. Convergence plots or summaries for imputed means and variances across iterations can reveal chains that drift or fail to mix. Compare imputed and observed distributions within relevant treatment arms and clinics, while remembering that observed cases need not represent the missing cases. Check ranges and scientific constraints, and inspect whether imputations reproduce plausible within-clinic and between-clinic variation.
Other warning signs include a few clinics dominating the imputation, implausible differences in imputed values by treatment, or completed datasets that erase associations present in the observed data. The imputed values should vary across datasets; identical fills suggest that uncertainty has been suppressed. A convergence trace that appears stable is not proof of the missing-at-random assumption or of correct conditional models.
Confirm the number of clinics, patients per clinic, treatment allocations, and the level and missingness of each variable.
Inspect chain summaries, imputed-value distributions, ranges, and cluster-specific patterns; investigate drift, implausible values, or collapsed cluster variation.
Fit the same planned model in every completed dataset, then confirm that pooled uncertainty includes both within- and between-imputation variation and respects the cluster design.
Alternatives and Common Pitfalls
| Approach | Potential advantage | Main concern for this trial |
|---|---|---|
| Complete-case analysis | Simple to describe and fit | May lose precision and can be biased when complete cases do not support the target analysis under its assumptions. |
| Single-level MICE | Can be straightforward to implement | Treating patients as independent can understate dependence and distort within- and between-clinic relationships. |
| Multilevel chained equations | Can represent cluster dependence while imputing incomplete variables | Requires appropriate conditional models, adequate cluster information, and careful diagnostics; the assumptions are not guaranteed by the procedure. |
A common mistake is to repeat each clinic’s treatment assignment on all patient rows and then fit an ordinary patient-level imputation model, reasoning that the larger row count supplies more information about treatment. It does not: there are still only 12 independently randomized clinics. The correction is to model the cluster structure and use an analysis and degrees-of-freedom method appropriate to the cluster-randomized design.
Another limitation is that multilevel MICE cannot recover information absent from the observed data. With few clusters, cluster-level model parameters and small-sample inference can be unstable. Results can also be sensitive to the conditional model specification and the plausibility of missing at random. The introductory material on Missing Data (C38) gives the broader missing-data framework; here, the essential point is to state assumptions, check model behavior, and interpret the pooled result in the context of the trial design.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- In the hypothetical trial, why is treatment assignment cluster-level even though it appears on every patient row?
- If six of 120 patients have both baseline severity and outcome missing, and 22 have baseline severity missing, how many distinct patients have at least one of these variables missing?
- What is the pooled total variance when \(m=5\), \(\bar{U}=2.0\), and \(B=0.5\)?
- Why is a stable trace of imputed means not evidence that missingness is missing at random?
Solutions: (1) Assignment was made once for each clinic, so the clinic—not each patient row—is the randomized unit. Repetition in the file records the same assignment. (2) If the six patients with both values missing are included among the 22 with baseline severity missing, then \(22+10-6=26\) distinct patients have at least one variable missing, where 10 is the hypothetical number missing outcome in the trial description. (3) \(T=2.0+(1+1/5)(0.5)=2.0+0.6=2.6\) squared units. (4) The trace assesses behavior of the imputation algorithm; missing at random concerns whether missingness depends on unobserved values after conditioning on observed variables, which cannot be verified from the trace.