Preserving Interactions and Curvature in Imputation

Why the Shape of the Regression Model Matters

An imputation model is used to fill in plausible values for missing variables. It should do more than predict each missing value reasonably well on average: it should also preserve the relationships that the regression analysis is intended to estimate. If the analysis includes an interaction or a curved relationship, an imputation model that assumes only simple straight-line relationships can weaken or alter those effects.

For example, the association between a predictor \(X\) and an outcome \(Y\) may differ across groups defined by \(Z\). That is an interaction. Or the mean outcome may change along a curve as \(X\) changes, rather than along a straight line. That is curvature. In either case, imputing \(X\) without representing the relevant structure can make the completed data inconsistent with the regression model.

Definition: An imputation model is compatible with the analysis model when it represents the relationships needed by the planned regression, including its interactions and nonlinear terms. Compatibility does not mean that every imputation equation must have an identical form to the regression equation; it means the imputation procedure must preserve the relationships the analysis will estimate.

The standard multiple-imputation framework assumes that the imputation model includes information relevant to missingness and to the analysis, and that its form is adequate for the data. Under a missing-at-random assumption conditional on the variables included in the imputation model, a suitable model can support valid inference. Missingness diagnostics alone cannot establish that assumption. The earlier material on predictors for imputation and including the outcome when imputing predictors explains why relevant observed variables, including the outcome, generally belong in the imputation process.

Interactions Must Be Represented

Consider an analysis model with an interaction between \(X\) and \(Z\):

E(Y\mid X,Z)=\beta_0+\beta_1X+\beta_2Z+\beta_3XZ

The slope relating \(X\) to the mean of \(Y\) depends on \(Z\). For a fixed value of \(Z\), the slope is \(\beta_1+\beta_3Z\). Thus, if an imputation model for a missing \(X\) treats its relationship with \(Y\) as having the same form at every value of \(Z\), it may fail to preserve information about the interaction.

Worked Example: Different Slopes Across Two Groups

Suppose a hypothetical regression analysis uses the model \(E(Y\mid X,Z)=10+2X+4Z+3XZ\), where \(Z=0\) or \(Z=1\). Work out the fitted mean at \(X=-1\) and \(X=1\) in each group, and identify the interaction pattern.

For \(Z=0\), the model reduces to \(E(Y\mid X,Z=0)=10+2X\). Therefore, the fitted means are \(10+2(-1)=8\) at \(X=-1\) and \(10+2(1)=12\) at \(X=1\). The slope across these values is \((12-8)/(1-(-1))=2\) outcome units per unit of \(X\).

For \(Z=1\), the model reduces to \(E(Y\mid X,Z=1)=10+2X+4+3X=14+5X\). The fitted means are \(14+5(-1)=9\) and \(14+5(1)=19\). The slope is \((19-9)/2=5\) outcome units per unit of \(X\).

The slopes differ by \(5-2=3\), which is the coefficient of \(XZ\). An imputation model that ignores how the relationship changes with \(Z\) may blur this difference. For instance, the model implies \(X=(Y-10)/2\) when \(Z=0\), but \(X=(Y-14)/5\) when \(Z=1\), in this noise-free illustration. These distinct relationships show why a single simple relationship between \(X\) and \(Y\) need not be adequate for imputing \(X\).

In practice, a useful imputation model for a missing predictor should include the outcome and variables that define important interactions. Depending on the variables and the imputation method, this can mean including terms such as \(Y Z\) when imputing \(X\), or using an approach designed to make the imputation model compatible with the substantive regression. Simply entering \(Y\), \(Z\), and \(X\)'s other predictors as separate main effects may not capture the way the conditional relationship changes across values of \(Z\).

The exact conditional distribution for a missing variable can be more complicated than the substantive regression equation. The example above used a noise-free equation to make the pattern visible; real data include residual variation, and one algebraic rearrangement is not a general recipe for imputation. The important requirement is to use an imputation model capable of retaining the effect modification represented by the analysis.

Curvature Must Be Preserved Too

A nonlinear analysis may include a squared predictor, such as \(X^2\), in addition to \(X\). If \(X\) is missing, the imputed value determines both the linear term and its square. Replacing \(X\) with a conditional mean and then squaring that mean does not generally preserve the expected squared value.

Key result: For any variable \(X\) with finite variance, \(E(X^2)=\{E(X)\}^2+\operatorname{Var}(X)\). Therefore, unless \(X\) has no variation, squaring its mean gives a smaller value than the mean of its square.

This difference matters in a regression with curvature. Suppose the conditional mean is \(5+2X-0.5X^2\). If two plausible values of \(X\) are equally likely, replacing them with their average can change the fitted mean, even though the average value of \(X\) is unchanged.

Worked Example: Mean Imputation Can Flatten Curvature

Consider the hypothetical regression mean \(E(Y\mid X)=5+2X-0.5X^2\). Suppose two equally likely values for a missing predictor are \(X=-2\) and \(X=2\). Compare the average fitted outcome at those values with the fitted outcome after replacing both values by their mean.

At \(X=-2\), the fitted outcome is \(5+2(-2)-0.5(4)=5-4-2=-1\). At \(X=2\), it is \(5+2(2)-0.5(4)=5+4-2=7\). Their average is \((-1+7)/2=3\).

The mean of the two predictor values is \((-2+2)/2=0\). Substituting that mean into the regression gives \(5+2(0)-0.5(0^2)=5\), not 3. The difference occurs because the average of \(X^2\) is \((4+4)/2=4\), whereas the square of the average \(X\) is \(0^2=0\). The curvature term is lost when the two different values are replaced by their mean.

This calculation does not say that every imputation method will create exactly this numerical change. It shows why a single mean replacement, or an imputation model that fails to represent curvature, can misrepresent the relationship that the regression is meant to estimate.

Construct Terms From Imputed Values

When an analysis uses a product \(XZ\) or a square \(X^2\), those terms are calculated from the underlying variables. A practical principle is to impute the component variables and then calculate the derived terms within each completed data set. This is often called passive imputation: a derived quantity is updated from its components rather than imputed as though it were an unrelated variable.

Worked Example: Updating an Interaction and a Square

Suppose the analysis uses \(X\), \(Z\), \(XZ\), and \(X^2\). For one hypothetical record, \(Z=3\) is observed but \(X\) is missing. In one imputed data set, the imputed value is \(X=1.5\). Calculate the derived terms for that record.

The interaction is \(XZ=1.5(3)=4.5\). The squared term is \(X^2=(1.5)^2=2.25\). Thus, this completed record contributes \(X=1.5\), \(Z=3\), \(XZ=4.5\), and \(X^2=2.25\) to the analysis.

The values \(XZ\) and \(X^2\) are not independent unknown measurements in this example: each is determined by the imputed \(X\) and observed \(Z\). If an interaction were separately imputed as, say, \(XZ=5\), it would contradict the component values because \(1.5(3)=4.5\), not 5. Recomputing the terms keeps the completed records internally consistent.

Passive calculation alone is not enough if the model used to impute \(X\) ignores the outcome relationship or the interaction pattern. The imputation equation for the missing component must also be suitable for the analysis structure.

A practical workflow is to identify the terms in the substantive regression first, then decide how the imputation process can preserve them. If several variables are incomplete, this decision applies to each relevant conditional imputation model. For example, if \(X\) is missing and the analysis contains \(XZ\), the imputation for \(X\) needs to account for the outcome and the role of \(Z\). If \(Z\) is also missing, the imputation strategy must respect the same substantive interaction rather than creating unrelated, contradictory values for \(X\), \(Z\), and \(XZ\).

Write the analysis model.
List every interaction and nonlinear term, such as \(XZ\) or \(X^2\), that will appear in the regression.

Identify incomplete components.
Determine which underlying variables used in those terms have missing values; do not treat a product or power as a separate measured variable when it is determined by its components.

Choose a compatible imputation approach.
Include relevant outcomes and interaction or nonlinear structure in the imputation strategy, or use an approach specifically designed to preserve the substantive model.

Recalculate derived terms in each completion.
After imputing component values, calculate products and powers from those values before fitting the planned regression.

Common Pitfalls and Why They Matter

Imputing every variable with main effects only. This can miss a relationship that changes across groups or bends with a predictor. The correction is to make the imputation strategy capable of representing the interactions and curvature required by the analysis.
Imputing a derived term independently of its components. An independently imputed \(XZ\) may not equal the product of the completed \(X\) and \(Z\). Recalculate such terms from their components in every completed data set.
Assuming that recomputing terms solves every problem. Passive calculation ensures algebraic consistency, but it does not repair an imputation model for \(X\) that fails to preserve the relationship between \(X\), \(Y\), and \(Z\).
Treating a single conditional mean as a plausible completed data set. A mean can suppress real variation and, with curvature, need not preserve the average fitted outcome. Multiple imputation represents uncertainty by creating multiple plausible completions rather than substituting one fixed mean for every missing value.
Assuming that an interaction is automatically handled because both variables appear in the imputation model. Including \(Y\), \(X\), and \(Z\) as main effects does not necessarily preserve the way the \(X\)-outcome relationship differs by \(Z\). The interaction structure itself must be considered.

Preserving interactions and curvature matters because regression coefficients describe the model actually fitted. If imputation has weakened an interaction or smoothed away a curve, the fitted coefficient may describe a distorted relationship even when the imputed values look individually plausible. A careful analysis therefore checks compatibility between the imputation strategy and the intended regression before interpreting its effects.

Check Your Understanding

Use the ideas in this tutorial to answer the following questions.

In a regression with \(XZ\), what does it mean for the slope of \(X\) to depend on \(Z\)?
Why might imputing \(X\) from a model containing only main effects fail when the analysis includes an \(XZ\) interaction?
For \(X=-2\) and \(X=2\) with equal probability, what are \(E(X^2)\) and \(\{E(X)\}^2\), and why are they different?
If a completed record has imputed \(X=1.5\) and observed \(Z=3\), what values should be used for \(XZ\) and \(X^2\)?
Why does passive recalculation of a product not by itself guarantee that the imputation model preserves the substantive regression relationship?