Stratified Checks for Imputed Interactions

Why Check Interactions Within Strata?

An imputation model can reproduce the overall distribution of a predictor and still weaken the interaction that the analysis is meant to estimate. The problem is that a marginal comparison averages over covariate groups. If the predictor–outcome relationship differs across those groups, averaging can hide the difference that defines the interaction.

Suppose the planned analysis relates a continuous outcome \(Y\) to a predictor \(X\), a binary covariate \(Z\), and their product. A useful diagnostic is to compare the distribution of \(X\) among observed and imputed values within joint strata of \(Y\) and \(Z\). The comparisons ask whether the imputed values retain the way \(X\) varies with \(Y\) separately for each value of \(Z\). These checks are diagnostic evidence, not a replacement for fitting the analysis model or a guarantee of valid inference.

For a continuous outcome, strata can be formed using prespecified outcome bands, or the outcome can be examined continuously with stratified plots or smooth summaries. For a categorical outcome, its observed categories can define the outcome strata directly. The covariate strata should reflect the interaction in the analysis model; for a continuous covariate, broad bands or a plot across its range may be more informative than arbitrary fine categories.

Analysis Model and the Conditional Pattern

For the basic setup, assume the planned analysis is the linear model

The conditional slope of \(Y\) with respect to \(X\) is \(\beta_X\) when \(Z=0\) and \(\beta_X+\beta_{XZ}\) when \(Z=1\). Thus, the interaction coefficient is the difference between those two slopes. If \(X\) is incomplete, an imputation model for \(X\) should preserve the information in \(Y\) and \(Z\) that is needed to retain this changing relationship. The diagnostic does not require the linear model to be true; the model states the feature that the imputation is intended to support.

A short derivation under a simple joint normal model explains why comparing \(X\) across both \(Y\) and \(Z\) is relevant. Suppose, within each value \(z\), that \(X\mid Z=z\sim N(\mu_z,\tau_z^2)\), and that

By multiplying the normal density for \(X\mid Z=z\) by the normal likelihood for \(Y\mid X,Z=z\), the terms involving \(X\) in the exponent are quadratic. Completing the square gives a conditional normal distribution with variance and mean

The conditional mean of \(X\) changes with \(Y\), and the coefficient of that change depends on the group-specific slope \(\gamma_z\). When the outcome slope differs by \(Z\), the conditional pattern of \(X\) given \(Y\) can therefore differ by \(Z\) as well. This derivation assumes normality and a simple joint model; it motivates the diagnostic but does not make normality a requirement for comparing empirical distributions.

Worked Stratified Checks

A Practical Diagnostic Sequence

A useful check keeps the comparison tied to the analysis question while showing enough detail to expose patterns hidden by averages. The following sequence is appropriate when \(Y\) and the stratifying covariates are observed for records with missing \(X\).

Assumptions, Alternatives, and Failure Modes

These comparisons are most interpretable when the interaction-relevant covariates and outcome are observed for records with missing \(X\), the observed and imputed values can be meaningfully compared within the same strata, and there is adequate overlap: each cell needs enough observed values to serve as a useful reference. A common working assumption in missing-data analysis is that missingness is at random conditional on variables included in the imputation model. Stratified checks cannot verify that assumption. If missingness still depends on unobserved \(X\) after conditioning on observed information, observed values may not provide a representative reference for the missing values.

A table of means alone is not enough. Two distributions can have the same mean but differ in spread, skewness, or tails; those features may affect an analysis. In addition to cell summaries, inspect plots or quantiles when cell sizes allow. For continuous \(Y\), outcome bands are a simplification: results may change with band boundaries, and binning can obscure smooth changes. Use bands as a readable diagnostic, not as evidence that the outcome truly has discrete categories.

A large difference in one sparse cell can be unstable, while an overall average can hide a systematic difference across several cells. Check cell counts and the distribution of summaries across imputations. If the imputation model omits information needed to preserve an interaction, a pattern of reduced or reversed conditional contrasts may prompt model review. It does not identify one specific correction by itself.

Stratified checks complement, rather than replace, alternatives. A direct comparison of analysis-model coefficients across completed datasets addresses the fitted interaction on the analysis scale, but can be hard to interpret as a check of the conditional distributions that produced those estimates. Marginal comparisons are simpler, but can mask the group-specific pattern, as the second example showed. Predictive checks can compare broader features of the imputed distributions, while this diagnostic focuses on the conditional structure relevant to a particular interaction. None of these checks alone establishes that an imputation model is correct.

Common Mistake

A further limitation is that stratification becomes unreliable when it creates many small cells, especially when outcome bands and several covariates are crossed. In that situation, simplify the display, use plots that retain continuous information where possible, and report the limited precision honestly. Do not interpret noisy cell differences as established distortion or a good-looking summary as confirmation that none exists.