Why a Regression Plan Needs a Compatibility Review
A regression plan determines which relationships the completed data must support. If an analysis includes an interaction or a nonlinear term, preserving only the marginal distributions of the covariates is not enough: the imputation models must also retain the relevant conditional relationships. A useful review therefore starts with the proposed analysis formula and traces each term to the imputation models for the variables it involves.
This tutorial reviews a hypothetical plan with a fully observed continuous outcome, two incomplete continuous covariates, and one complete adjustment covariate. The purpose is not to select an imputation algorithm for every possible data structure. It is to make the compatibility requirements explicit, identify where a simple predictor list is inadequate, and state what should be checked before relying on the analysis.
The Hypothetical Regression Plan
Suppose a clinical team plans to model a continuous outcome \(Y\) using two continuous covariates \(X\) and \(Z\), plus a fully observed covariate \(C\). Some values of \(X\) and \(Z\) are missing, but \(Y\) and \(C\) are complete. The prespecified analysis is
The analysis assumes, for its conditional mean, that the effects are represented by the displayed terms. A complete outcome model also specifies an error distribution or another suitable outcome distribution, and its residual assumptions must be checked for the intended inference. For this review, the important features are the interaction \(XZ\) and the quadratic term \(X^2\). The conditional mean need not be linear in \(X\), and the association between \(X\) and \(Y\) need not be constant across values of \(Z\).
For the compatibility requirements below, assume that the missingness mechanism is plausibly missing at random given variables observed in the data, that there is adequate overlap in the covariate patterns, and that the imputation models are sufficiently well specified and estimated. These conditions do not follow from the regression formula. They are separate assumptions needed for valid inference from multiple imputation. If important predictors of missingness or of the incomplete values are unavailable, or some combinations of \(X\) and \(Z\) are poorly represented, a compatibility checklist cannot repair that problem.
Worked Example: Expanding the Analysis Formula Into Relationships
For a hypothetical participant with \(X=2\), \(Z=3\), and \(C=1\), the analysis model’s covariate terms are \(X\), \(Z\), \(XZ\), \(X^2\), and \(C\). Evaluate the terms that are not just the original covariate values.
Interaction: \(XZ=2\times3=6\). The interaction contribution to the fitted mean is \(6\beta_3\).
Quadratic term: \(X^2=2^2=4\). Its contribution is \(4\beta_4\).
Thus the fitted conditional mean for this covariate pattern is \(\beta_0+2\beta_1+3\beta_2+6\beta_3+4\beta_4+\beta_5\). The imputation review must account for the relationships represented by both \(XZ\) and \(X^2\), not merely include the raw values \(X\), \(Z\), and \(C\) somewhere in a predictor list.
What the Interaction and Curvature Require
The model’s conditional mean makes the role of each term clear. Holding \(Z\) and \(C\) fixed, the rate of change in the mean outcome with respect to \(X\) is
This derivative follows by differentiating each term: \(\beta_1X\) contributes \(\beta_1\), \(\beta_3XZ\) contributes \(\beta_3Z\), and \(\beta_4X^2\) contributes \(2\beta_4X\); the remaining terms do not depend on \(X\). The association of \(X\) with the conditional mean therefore varies with both \(Z\) and \(X\). Similarly,
So the \(XZ\) term also means that the association for \(Z\) varies with \(X\). An imputation model that only represents a constant, straight-line association between \(Y\) and each covariate may fail to retain these changing relationships. This is a requirement about the conditional distributions used for imputation, not a claim that adding a product column automatically makes the models compatible.
Audit the Conditional Models Separately
A chained-equation plan specifies a conditional model for each incomplete variable. The audit must therefore be variable-specific. A predictor that is available when imputing one covariate may be the target being imputed in another model; in the latter case, the corresponding functional form must be represented in the conditional distribution rather than naively added as a predictor containing an unknown target value.
| Variable being imputed | Relationships to retain from the analysis plan | Review question |
|---|---|---|
| \(X\) | Association with \(Y\); dependence on \(Z\); curvature in \(X\); and the way the \(X\)-outcome relationship varies with \(Z\). | Can the conditional model for \(X\) represent the nonlinear, \(Z\)-dependent relationship implied by the substantive model? |
| \(Z\) | Association with \(Y\); dependence on \(X\), including the \(X\)-by-\(Z\) relationship; and any relevant association with \(X^2\). | Can the conditional model for \(Z\) represent an association with the outcome that changes over \(X\), rather than forcing a constant effect? |
| Any other incomplete analysis covariate | Its analysis association with \(Y\), its links to \(X\) and \(Z\), and any interactions or nonlinear terms involving it. | Does its own conditional model preserve every relevant term in which it participates? |
For both \(X\) and \(Z\), \(Y\) is observed and should generally be available as an imputation predictor: omitting the outcome can weaken the covariate–outcome associations that the final analysis estimates. The observed covariate \(C\) should also be considered in both models because it is included in the analysis and may explain variation in the incomplete variables. For the \(X\) model, \(Z\) is an essential observed companion when available; for the \(Z\) model, \(X\) and the curvature represented by \(X^2\) are relevant. The interaction means that each target’s relationship with the outcome must be allowed to vary with the other covariate.
This audit does not prescribe that the \(X\) imputation equation contain \(X^2\) or \(XZ\) as ordinary predictors: those expressions require the unknown \(X\). Nor does it claim that placing \(Y\), \(Z\), and a product such as \(YZ\) in a standard regression automatically yields a conditional distribution compatible with the stated analysis model. Such terms may be useful in particular practical specifications, but compatibility is a distributional property. The conditional form must be capable of representing the relationships implied by the substantive model, and its adequacy must be evaluated.
Worked Example: Reviewing Each Incomplete Covariate
Suppose a draft plan proposes two ordinary linear imputation regressions: \(X\) on \(Y,Z,C\), and \(Z\) on \(Y,X,C\). It includes no nonlinear structure or interactions. Review it against the analysis formula.
For \(X\): The proposed model includes the outcome and the other covariates, which is a useful starting point. However, it assumes a simple conditional form unless expanded. The analysis requires the \(X\)-outcome association to vary with \(Z\) and to reflect curvature in \(X\). The draft does not show how the conditional distribution for \(X\) can represent either feature. Thus, the predictor list alone does not pass the compatibility review.
For \(Z\): The proposal includes \(Y\) and \(X\), but a constant-form regression can miss that the analysis allows the \(Z\)-outcome association to vary with \(X\). It also does not establish that the relationship involving \(X^2\) is represented. This model likewise needs a functional-form review.
Decision: Do not approve the plan solely because both equations contain \(Y\) and the other incomplete covariate. Specify conditional models capable of representing the interaction and curvature, and assess whether those models are plausible for the observed data. If a straightforward conditional specification cannot express the required structure, use an approach designed to respect the substantive model rather than treating a longer predictor list as proof of compatibility.
Worked Compatibility Review With Hypothetical Values
Worked Example: Reading the Implied Effect Across Covariate Values
Suppose, purely hypothetically, the fitted analysis coefficients are \(\beta_1=1.2\), \(\beta_3=0.5\), and \(\beta_4=-0.1\). These values are used only to illustrate the model’s structure. At \(X=2\), the slope in the conditional mean for \(X\), when \(Z=1\), is
\(\beta_1+\beta_3Z+2\beta_4X=1.2+(0.5)(1)+2(-0.1)(2)=1.2+0.5-0.4=1.3\) outcome units per unit of \(X\).
At the same \(X=2\) but \(Z=3\), the slope is \(1.2+(0.5)(3)+2(-0.1)(2)=1.2+1.5-0.4=2.3\) outcome units per unit of \(X\). The difference, \(2.3-1.3=1.0\), equals \(\beta_3(3-1)=0.5\times2=1.0\). It reflects the interaction. An imputation strategy that flattens the relationship across \(Z\) could distort the imputed values needed to estimate this difference.
At \(Z=1\), increasing \(X\) from 2 to 3 changes the slope because of the quadratic term. The slope at \(X=3\) is \(1.2+0.5+2(-0.1)(3)=1.1\), compared with \(1.3\) at \(X=2\). The decrease of \(0.2\) per unit of \(X\) matches \(2\beta_4=-0.2\). Consequently, a compatibility review must check both the variation across \(Z\) and the curvature across \(X\).
Diagnostics, Alternatives, and Limits
After imputation, examine whether the relationships that the analysis needs remain plausible. Compare observed and imputed values of \(X\) and \(Z\), including their ranges, spread, and conditional patterns across the other covariate and outcome. Inspect plots or summaries that expose curvature and interaction: for example, the association of \(Y\) with \(X\) across broad \(Z\) groups, and the association of \(Y\) with \(Z\) across \(X\) ranges. Check whether imputed values produce implausible covariate combinations or too little variation in cells important to the interaction. These checks can reveal problems, but they do not prove compatibility, establish the missingness mechanism, or guarantee valid inference.
The principal alternatives clarify what is at stake. Complete-case analysis avoids specifying these imputation conditionals, but discards records with missing covariates and can target a biased result under many missingness mechanisms. A simple chained-equation model with only main effects is easier to fit, but may attenuate or otherwise distort the interaction and curvature. A richer imputation model can retain more of the analysis structure, though adding many terms without adequate data can produce unstable estimates or overfitting. A method tailored to the substantive model may better address difficult nonlinear conditional relationships, but it requires its own assumptions and careful implementation.
No finite checklist guarantees exact compatibility. Chained-equation conditionals can be mutually incompatible as probability models, and good marginal distributions do not ensure that the joint or conditional relationships are preserved. Sparse data, extreme covariate values, poor overlap, misspecified outcome or imputation models, and departures from the missing-at-random assumption can all undermine the review. The appropriate conclusion is therefore calibrated: a compatibility audit identifies required relationships and weaknesses in the plan; it does not certify the assumptions as true.
Common Mistake
- Wrong reasoning: “Both imputation regressions include \(Y\), \(X\), \(Z\), and \(C\), so the analysis interaction and quadratic term are preserved.”
- Correction: Variable inclusion does not guarantee functional-form preservation. Check whether each conditional model can represent the \(X\)-by-\(Z\) variation and the curvature in \(X\). For the target being imputed, terms containing that target need a coherent conditional treatment; they cannot be assumed to work as ordinary predictors.
- Another warning: Finding similar marginal distributions for observed and imputed values is not enough. The analysis depends on conditional relationships that can be wrong even when each variable’s overall distribution looks reasonable.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- In the model \(E(Y\mid X,Z,C)=\beta_0+\beta_1X+\beta_2Z+\beta_3XZ+\beta_4X^2+\beta_5C\), what is the slope in the conditional mean for \(X\)?
- Why does including \(Y\), \(Z\), and \(C\) in an \(X\) imputation equation not by itself establish compatibility?
- At \(X=1\), \(Z=2\), with hypothetical \(\beta_1=0.8\), \(\beta_3=0.3\), and \(\beta_4=0.2\), calculate the slope in the conditional mean for \(X\).
- Name two diagnostics that target the relationships relevant to this analysis plan, and state one limitation of those diagnostics.
Solutions. (1) Differentiate the conditional mean with respect to \(X\): the slope is \(\beta_1+\beta_3Z+2\beta_4X\). (2) Those variables may be present while the conditional model still imposes an incorrect simple form; compatibility requires representing the interaction and nonlinear relationship as well as including relevant variables. Also, terms involving the target \(X\) cannot simply be added as ordinary predictors. (3) Substitute the values: \(0.8+(0.3)(2)+2(0.2)(1)=0.8+0.6+0.4=1.8\) outcome units per unit of \(X\). (4) Examples include comparing imputed and observed conditional patterns of \(Y\) against \(X\) across \(Z\) groups, and checking whether imputed values preserve plausible curvature across \(X\). Such diagnostics can flag distortions but cannot prove compatibility or verify the missingness assumption.