Why Examine the Interaction Pattern?
An imputation model can reproduce the marginal distribution of each variable and still fail to preserve a relationship that matters to the analysis. An interaction is especially vulnerable: the association between an exposure and an outcome may change across levels of a modifier. If that pattern is weakened or altered when missing values are filled in, the analysis of the completed data may tell a different story from the analysis of complete cases.
The purpose of this diagnostic is not to decide that the complete-case result is correct. Complete cases may be a selected subset, and their coefficient can be biased for the target population. Instead, compare the patterns and ask whether the differences are plausible, whether they are supported by the data, and whether the imputation model preserves the structure required by the analysis.
For a binary exposure coded 0 or 1 and a continuous modifier, consider the linear mean model \(E(Y\mid A,B)=\beta_0+\beta_A A+\beta_B B+\beta_{AB}AB\). Here \(B\) must have the same units and centering in every comparison. At modifier value \(b\), the modeled difference in mean outcome between \(A=1\) and \(A=0\) is \(\beta_A+\beta_{AB}b\). Thus, \(\beta_{AB}\) is the change in the exposure-associated mean difference for a one-unit increase in \(B\). It is not a universal measure of interaction: the scale and model matter.
A useful diagnostic combines grouped plots with fitted coefficients. Plots reveal whether the observed pattern looks different in parts of the data; coefficients summarize the interaction under a particular model. Agreement between them is reassuring but does not prove compatibility. Disagreement is a reason to investigate, not an automatic verdict against either result.
Assumptions and a Useful Comparison
A comparison is interpretable only when the complete-case and imputed-data analyses use the same outcome definition, coding, model terms, reference categories, and units. For example, changing the origin of \(B\) changes the main-effect coefficient \(\beta_A\), even though it does not change the interaction coefficient in this linear model. Changing the unit of \(B\) changes the numerical value of \(\beta_{AB}\). Keep the parameterization fixed before interpreting a difference.
The model must also be a reasonable description of the conditional mean for the question at hand. A linear interaction is a straight-line change in the exposure effect as \(B\) changes; a curved effect pattern may require a more flexible model. In addition, imputation-based inference relies on an adequately specified imputation procedure, proper representation of imputation uncertainty, and a missingness assumption appropriate to the analysis. In particular, an analysis that assumes missing at random relies on conditioning on information sufficient to make that assumption plausible. No plot of completed data can verify that assumption.
For grouped displays, choose clinically or scientifically meaningful modifier groups, and use the same cut points for each comparison. Show the number of observed and imputed outcomes contributing to each group, where relevant. Do not treat repeated completed versions of a person as independent observations. A plot of one arbitrarily selected completed dataset can hide variation between imputations; summaries across imputations should show that variation or clearly identify it.
Worked Example: A Difference-in-Differences Across Two Modifier Groups
Suppose a hypothetical study compares a binary treatment \(A\) with a control and groups a binary modifier \(B\) into low and high levels. The table gives group mean outcomes in points. The complete-case analysis includes only records complete on the variables required for this analysis; the imputed-data values summarize the corresponding groups across completed datasets.
| Modifier group | Control mean, complete cases | Treatment mean, complete cases | Control mean, imputed data | Treatment mean, imputed data |
|---|---|---|---|---|
| Low | 10 | 12 | 10 | 13 |
| High | 14 | 17 | 14 | 20 |
With binary \(A\) and \(B\), the saturated mean model is \(E(Y\mid A,B)=\beta_0+\beta_A A+\beta_B B+\beta_{AB}AB\), where \(B=0\) denotes low and \(B=1\) denotes high. The exposure difference in the low group is the treatment mean minus the control mean. It is \(12-10=2\) points among complete cases and \(13-10=3\) points in the imputed-data summary.
In the high group, the corresponding differences are \(17-14=3\) points and \(20-14=6\) points. The interaction coefficient is the difference between these exposure differences:
The grouped pattern therefore suggests a stronger increase in the treatment-associated mean difference across modifier groups in the imputed-data summary: 3 points rather than 1 point. This is a diagnostic discrepancy, not proof that the imputation model created distortion. The groups may differ in composition, the complete cases may be selected, and uncertainty in both summaries matters. Group sizes and the spread of estimates across imputations would be needed before judging how stable this contrast is.
Why the Product Coefficient Matches the Group Contrast
The binary-modifier calculation above follows directly from the model. When \(B=0\), the model reduces to \(E(Y\mid A,B=0)=\beta_0+\beta_A A\), so the exposure difference is \(\beta_A\). When \(B=1\), it becomes \(E(Y\mid A,B=1)=(\beta_0+\beta_B)+(\beta_A+\beta_{AB})A\), so the exposure difference is \(\beta_A+\beta_{AB}\). Subtracting the low-group difference from the high-group difference gives \(\beta_{AB}\). This identity explains why a grouped plot and the fitted product coefficient should tell a consistent story when the model is saturated for these binary variables and the same observations and weighting are used.
With a continuous modifier, group plots are only a coarse view of the fitted relationship: grouping discards within-group variation and can conceal curvature. The fitted coefficient describes the specified model, while the grouped display checks whether the data show a broadly compatible pattern without relying only on that model.
Worked Example: Comparing Interaction Slopes for a Continuous Modifier
In a hypothetical analysis, \(A=1\) denotes treatment and \(A=0\) denotes control. Modifier \(B\) is measured in decades and centered at its reference value. The same linear interaction model is fitted to complete cases and to each imputed dataset. Suppose the complete-case estimate is \(\hat{\beta}_{AB}=0.12\) points per decade, with \(SE=0.08\), while the pooled imputed-data estimate is \(\bar{Q}=0.38\) points per decade, with \(SE=0.15\).
The complete-case model implies that the treatment-associated mean difference changes by 0.12 points for each one-decade increase in \(B\). The pooled imputed-data model implies a change of 0.38 points per decade. If the fitted treatment difference at the centered reference value is 0.40 points in the complete-case model and 0.35 points in the imputed model, then at one decade above that reference the respective fitted differences are \(0.40+0.12(1)=0.52\) points and \(0.35+0.38(1)=0.73\) points. At one decade below it, they are \(0.40+0.12(-1)=0.28\) points and \(0.35+0.38(-1)=-0.03\) points.
A useful uncertainty check is to form approximate 95% intervals using estimate \(\pm 1.96SE\). For complete cases, \(0.12\pm1.96(0.08)\) gives \((-0.037,\,0.277)\) points per decade. For the pooled imputed estimate, \(0.38\pm1.96(0.15)\) gives \((0.086,\,0.674)\) points per decade. These intervals describe uncertainty under their respective analysis procedures and assumptions. Their difference does not by itself establish that the two estimates differ statistically: the analyses are not independent, and a formal comparison would need to account for their dependence. The intervals also do not establish that either imputation or complete-case analysis is correct.
Reading Pooled Estimates and Grouped Displays
Multiple imputation produces an estimate and its variance in each completed dataset. For the interaction coefficient, let \(Q_j\) be its estimate and \(U_j\) its estimated variance in imputation \(j\), for \(j=1,\ldots,M\). The pooled estimate is \(\bar{Q}=M^{-1}\sum_{j=1}^M Q_j\). The average within-imputation variance is \(\bar{U}=M^{-1}\sum_{j=1}^M U_j\), and the between-imputation variance is \(B=(M-1)^{-1}\sum_{j=1}^M(Q_j-\bar{Q})^2\). Rubin's total variance is \(T=\bar{U}+(1+1/M)B\). The standard error is \(\sqrt{T}\).
The between-imputation component matters to this diagnostic. If the interaction estimate varies substantially from one imputed dataset to another, a plot from a single completed dataset is particularly uninformative. A pooled point estimate alone also hides this instability; examine the imputation-specific estimates or their spread alongside the pooled estimate and standard error.
Worked Example: Distinguishing a Shift From Imputation Variability
Suppose, hypothetically, that five completed datasets give interaction estimates of 0.8, 0.9, 1.0, 0.8, and 1.0 outcome points per decade. Each has within-imputation variance 0.04 points squared. The pooled estimate is
The mean is 0.90 points per decade. Deviations from the mean are \(-0.1, 0, 0.1, -0.1, 0.1\), so
Here \(\bar{U}=0.04\), and \(1+1/M=1.2\). Therefore,
The imputation-specific estimates are fairly close to their pooled mean, and their between-imputation variance adds 0.012 to the total variance. If a complete-case estimate were 0.20 points per decade, the point estimates would differ by 0.70 points per decade. That difference warrants checking the grouped pattern, the distribution of missingness across exposure and modifier levels, and the imputation model's representation of the interaction. It does not show that the pooled estimate is biased: the complete-case estimate may reflect selection, and both estimates have sampling uncertainty.
Diagnostics, Alternatives, and Limits
Use grouped plots to locate where a discrepancy occurs, then use the fitted interaction coefficient to summarize it on the analysis scale. When feasible, examine observed outcomes and completed-data summaries separately. Label imputed values as imputed, report group counts, and avoid presenting an average across completed datasets as if it were a set of directly observed outcomes. Sparse cells, extreme imputed values, or a pattern driven by only one modifier range are important warnings.
Use the same outcome, coding, units, analysis model, and modifier groups for complete-case and imputed-data checks.
Compare exposure-outcome patterns across modifier levels, checking counts, uncertainty, and whether the shape changes in a particular region.
Review the complete-case interaction estimate alongside the pooled imputed estimate, its standard error, and imputation-specific estimates.
Check sparse support, missingness patterns, influential observations, and whether the imputation model can represent the analysis interaction. A discrepancy is a clue, not a causal attribution.
Complete-case analysis is a useful descriptive comparator, but it can lose precision and can be biased when completeness is related to the outcome or predictors. Imputation-based analysis uses more records and represents missing-data uncertainty under its assumptions, but it can distort an interaction if the conditional imputation models do not preserve that relationship. Neither method is a universal benchmark. Alternative checks include examining stratum-specific associations, comparing fitted values over the modifier range, and fitting a model with a justified flexible form when a straight-line interaction is doubtful.
Several limitations remain. Grouping a continuous modifier can mask nonlinear patterns and make results sensitive to cut points. A coefficient can change because of differing observed-data composition, model misspecification, sparse support, or random variability—not only because of incompatibility. Conversely, similar coefficients do not prove that the imputation model is adequate. These checks assess whether the imputed analysis relationship looks plausible; they cannot identify the true interaction from incomplete data or verify the missingness assumption.
Common Mistake
Wrong reasoning: “The complete-case interaction is near zero and the imputed interaction is larger, so imputation must have invented an effect.” This treats complete cases as an unbiased reference and ignores uncertainty and differences in who contributes information.
Correction: Describe the discrepancy, quantify uncertainty, and inspect where the grouped patterns differ. Then examine whether the imputation model represents the interaction and whether there is adequate data support in the relevant groups. The comparison can flag a possible distortion, but it cannot by itself determine which estimate is closer to the target.
Check Your Understanding
Answer each question, then compare with the solutions.
- With a binary modifier, the exposure differences are 4 points in the low group and 7 points in the high group. What is the interaction coefficient under the saturated linear model?
- Why should a complete-case interaction estimate not automatically be treated as the correct reference for an imputed estimate?
- Five imputations give \(\bar{Q}=0.50\), \(\bar{U}=0.09\), and \(B=0.02\). Calculate \(T\) and the pooled standard error.
- Name two reasons, other than imputation-model incompatibility, that complete-case and imputed interaction estimates might differ.
Solutions. (1) The interaction is the difference between exposure differences: \(7-4=3\) points. (2) Complete cases can be a selected subset and their estimate can be biased for the target population; it also has sampling uncertainty. (3) \(T=0.09+(1+1/5)(0.02)=0.09+0.024=0.114\), so \(SE=\sqrt{0.114}=0.338\). (4) Possible reasons include differing sample composition due to missingness, sampling variability, sparse data, influential observations, or misspecification of the analysis model.