Imputation-Model Compatibility With the Analysis Model · Tutorial 274 of 497

Deriving an Imputation Conditional With an Interaction

Learn how a product term in an outcome regression changes the conditional distribution used to impute a predictor, and how to identify the resulting terms.

Advanced 10 min read

What You'll Learn

State the joint-model assumptions needed to derive a predictor’s conditional distribution.
Complete the square to obtain the conditional mean and variance.
Identify the outcome-by-predictor and squared-covariate terms induced by a product term.
Calculate conditional imputation distributions from hypothetical model coefficients.
Recognize why a simple additive imputation regression may not reproduce the derived conditional distribution.

Why Derive the Conditional Distribution?

When an analysis regression includes a product term, the relationship of the outcome to one predictor depends on another predictor. If that first predictor has missing values, its imputation conditional should reflect the relationship represented by the analysis model. Deriving the conditional distribution makes the consequences precise: the product term can produce an outcome-by-covariate term, a squared covariate term, and a conditional variance that changes with the covariate.

A regression for the outcome alone is not enough to determine the distribution of a predictor given the outcome. The predictor’s distribution before observing the outcome is also needed. The derivation below therefore specifies a model for the predictor conditional on another covariate and a model for the outcome conditional on both predictors.

Definition: Let $X$ be a continuous predictor that may be missing, $Z$ an observed covariate, and $Y$ a continuous outcome. Consider the joint model specified by $$ X\mid Z=z\sim N\bigl(m(z),\tau^2\bigr), \qquad m(z)=\alpha_0+\alpha_1z, $$ and $$ Y\mid X=x,Z=z\sim N\bigl(\beta_0+\beta_1x+\beta_2z+\beta_3xz,\sigma^2\bigr). $$ Here $\tau^2>0$ and $\sigma^2>0$. The coefficient $\beta_3$ represents the product-term effect in the outcome regression.

These assumptions say that $X$, conditional on $Z$, has a normal distribution with a linear mean and constant variance, and that $Y$, conditional on $X$ and $Z$, has a normal distribution with the stated linear predictor and constant variance. The model treats the parameters as fixed while deriving the conditional distribution. In an applied imputation procedure, parameter uncertainty must also be represented; the algebra here gives the conditional form implied by the specified model.

The factorization $p(x,y\mid z)=p(x\mid z)p(y\mid x,z)$ is the chain rule for a joint distribution. It does not assert that $X$ and $Y$ are independent given $Z$. In fact, the outcome model explicitly links them.

Deriving $X\mid Y,Z$

For fixed observed values $Y=y$ and $Z=z$, define $$ c(z)=\beta_1+\beta_3z,\qquad d(y,z)=y-\beta_0-\beta_2z. $$ Then the outcome model can be written as $Y\mid X=x,Z=z\sim N(\beta_0+\beta_2z+c(z)x,\sigma^2)$. As a function of $x$, its likelihood is proportional to $\exp\{-[d(y,z)-c(z)x]^2/(2\sigma^2)\}$. The predictor model contributes a factor proportional to $\exp\{-[x-m(z)]^2/(2\tau^2)\}$.

Multiplying these two densities gives the conditional density of $X$ given $Y=y,Z=z$, up to a normalizing constant. The exponent’s terms involving $x$ are $$ -\frac{1}{2}\left[\frac{(x-m(z))^2}{\tau^2}+\frac{(d(y,z)-c(z)x)^2}{\sigma^2}\right]. $$

Expanding the squares, the coefficient of $x^2$ inside the brackets is $A(z)=1/\tau^2+c(z)^2/\sigma^2$, and the coefficient of the linear term $x$ is $-2B(y,z)$, where $B(y,z)=m(z)/\tau^2+c(z)d(y,z)/\sigma^2$. Thus the expression inside the brackets is $A(z)x^2-2B(y,z)x$ plus terms not involving $x$. Since $A(z)>0$, completing the square gives $$ A(z)\left(x-\frac{B(y,z)}{A(z)}\right)^2 $$ plus a term independent of $x$. This is the kernel of a normal density, with variance $1/A(z)$ and mean $B(y,z)/A(z)$.

Result: Under the stated joint model, $$ X\mid Y=y,Z=z\sim N\left( \frac{\dfrac{m(z)}{\tau^2}+\dfrac{c(z)d(y,z)}{\sigma^2}} {\dfrac{1}{\tau^2}+\dfrac{c(z)^2}{\sigma^2}}, \quad \frac{1}{\dfrac{1}{\tau^2}+\dfrac{c(z)^2}{\sigma^2}} \right). $$ The first argument of $N$ is the conditional mean; the second is the conditional variance. The outcome informs the conditional distribution through $d(y,z)$, while the strength of that information depends on $c(z)$.

The conditional mean is not generally a linear function of $y$ and $z$, and the conditional variance is not generally constant in $z$. Those features follow from the product term; they are not optional embellishments to the derivation.

Worked Examples

Worked Example: Calculating a Conditional Imputation Distribution

Suppose the hypothetical model has $m(z)=1+0.5z$, $\tau^2=4$, $\beta_0=2$, $\beta_1=1$, $\beta_2=0.4$, $\beta_3=0.6$, and $\sigma^2=9$. Find the conditional distribution of $X$ when $Y=8$ and $Z=2$.

First, $m(2)=1+0.5(2)=2$. The outcome slope for $X$ at this value of $Z$ is $c(2)=1+0.6(2)=2.2$, and the adjusted outcome value is $d(8,2)=8-2-0.4(2)=5.2$.

The conditional precision, the reciprocal of the conditional variance, is $1/4+2.2^2/9=0.25+4.84/9=0.7878$. Therefore, the conditional variance is $1/0.7878=1.269$. The conditional mean is $$ \frac{2/4+(2.2)(5.2)/9}{0.25+2.2^2/9} = \frac{0.5+1.2711}{0.7878} =2.248. $$

Thus $X\mid Y=8,Z=2\sim N(2.248,1.269)$. The conditional standard deviation is $\sqrt{1.269}=1.127$, in the units of $X$. These are model-implied conditional values for the hypothetical coefficients, not estimates from a real study.

Worked Example: Locating the Induced Interaction Terms

Let $X\mid Z=z\sim N(z,1)$, and let the outcome model have $\beta_0=1$, $\beta_1=2$, $\beta_2=1$, $\beta_3=0.5$, and $\sigma^2=1$. Write the conditional mean and variance of $X\mid Y=y,Z=z$, identifying the terms created by the product term.

Here $m(z)=z$, $c(z)=2+0.5z$, and $d(y,z)=y-1-z$. The numerator of the conditional mean is $$ z+(2+0.5z)(y-1-z) =2y-2+0.5yz-1.5z-0.5z^2. $$ The denominator, also the conditional precision, is $$ 1+(2+0.5z)^2=5+2z+0.25z^2. $$

Consequently, the conditional mean is $(2y-2+0.5yz-1.5z-0.5z^2)/(5+2z+0.25z^2)$, and the conditional variance is $1/(5+2z+0.25z^2)$. The numerator contains both $yz$ and $z^2$; the denominator also depends quadratically on $z$. For $y=3,z=2$, the numerator is $6-2+3-3-2=2$, and the denominator is $5+4+1=10$. The conditional mean is therefore $0.2$, and the variance is $0.1$.

This calculation shows why simply placing $Y$, $Z$, and $YZ$ in a linear predictor does not, by itself, reproduce the exact conditional distribution: the derived mean is a ratio involving $z$, and its variance changes with $z$.

Worked Example: Comparing the Interaction and Additive Cases

Suppose $X\mid Z=z\sim N(0,1)$, and $Y\mid X=x,Z=z\sim N(x+xz,1)$. At $Y=2$, compare the conditional distribution of $X$ for $Z=0$ and $Z=1$. Then compare the result at $Z=1$ with an additive outcome model that omits $xz$.

With the product term, $\beta_1=1,\beta_3=1,\beta_0=\beta_2=0$, so $c(z)=1+z$, $d=2$, and $m(z)=0$. At $z=0$, the conditional precision is $1+(1)^2=2$, so the variance is $1/2=0.5$; the mean is $(1)(2)/2=1$. At $z=1$, the precision is $1+(2)^2=5$, so the variance is $1/5=0.2$; the mean is $(2)(2)/5=0.8$.

If the product term were omitted, then at $z=1$ the slope would instead be $c=1$. The conditional variance would be $1/(1+1)=0.5$, and the conditional mean would be $2/2=1$. Thus, for these hypothetical values, omitting the product term changes both the conditional mean and variance used to impute $X$ at $Z=1$. This comparison illustrates a model consequence; it does not, by itself, quantify bias in any particular analysis.

What the Derivation Implies for Imputation

The derived conditional distribution provides a benchmark for an imputation model intended to reflect this joint regression structure. Its main implications are:

The outcome value $Y$ enters the conditional distribution for the missing predictor $X$. Its contribution is scaled by $c(z)$, the outcome slope for $X$ at the observed $Z$.
The product term can induce an outcome-by-covariate term $yz$ and a squared-covariate term $z^2$ in the numerator of the conditional mean.
The conditional variance depends on $z$ whenever $c(z)^2$ varies with $z$. A constant-variance conditional regression may therefore be an approximation.
If $\beta_3=0$, then $c(z)=\beta_1$ is constant. With the assumed linear $m(z)$, the conditional mean is then additive in $y$ and $z$, and the conditional variance is constant.

The derivation uses a continuous $X$, a continuous normally distributed outcome, and the particular linear-normal models stated above. It does not establish the correct conditional distribution for binary or ordinal variables, for nonnormal continuous outcomes, or for a different predictor model. Those settings require their own model assumptions and derivations.

Diagnostics, Alternatives, and a Common Pitfall

A useful diagnostic is to compare the conditional form implied by the joint model with the form used to generate imputations. Check whether the imputation model allows the outcome to inform $X$, whether the product-term consequences are represented, and whether the assumed variance behavior is plausible. Residual patterns, marked variation in spread across values of $Z$, or imputed values inconsistent with the observed support of $X$ can signal that the chosen conditional model is inadequate. Diagnostics cannot prove compatibility, but they can reveal clear mismatches.

An alternative is to specify a conditional imputation regression directly, rather than derive it from a joint distribution. Such a model can be convenient, but it is not automatically equivalent to the conditional distribution derived here. A simple normal regression with only $Y$ and $Z$ as linear predictors omits the displayed $yz$ and $z^2$ terms; even adding those terms does not automatically reproduce the changing conditional variance or the ratio form of the mean. Conversely, the joint-model derivation is exact only to the extent that its distributional assumptions are defensible.

Common mistake: “The outcome model contains $XZ$, so the imputation model only needs to add $YZ$ to a linear regression for $X$.” This reasoning notices one induced term but treats the conditional mean as linear and ignores its denominator. The correction is to derive $X\mid Y,Z$ from both parts of the joint model, then examine the entire mean and variance. The exact conditional may have a nonlinear mean and a predictor-dependent variance, not just one extra product term.

Key takeaway: A product term in the outcome regression changes the slope $c(z)$ used in the conditional distribution of $X$. Bayes’ rule combines that $z$-dependent likelihood with the predictor model, producing a conditional mean and variance that generally depend nonlinearly on $z$.

Check Your Understanding

Use the derivation and assumptions in this tutorial to answer the questions. Solutions follow each question.

Why is the outcome regression alone insufficient to derive $X\mid Y,Z$?
Solution: It specifies $p(y\mid x,z)$, but the conditional distribution also depends on how plausible different values of $X$ are before observing $Y$. A model for $p(x\mid z)$ is also required.
In the stated model, what are $c(z)$ and $d(y,z)$, and what roles do they play?
Solution: $c(z)=\beta_1+\beta_3z$ is the outcome slope for $X$ at $Z=z$, and $d(y,z)=y-\beta_0-\beta_2z$ is the outcome after subtracting the terms not involving $X$. They determine the likelihood contribution to the conditional mean and precision.
If $\beta_3=0$, $m(z)$ is linear, and the stated normal models hold, what happens to the conditional variance?
Solution: Then $c(z)=\beta_1$, so the variance is $1/(1/\tau^2+\beta_1^2/\sigma^2)$, which is constant in $z$ and $y$.
For $m(z)=0$, $\tau^2=1$, $c(z)=2$, $\sigma^2=4$, and $d(y,z)=3$, calculate the conditional mean and variance.
Solution: The precision is $1+2^2/4=2$, so the variance is $1/2=0.5$. The mean is $(0+2(3)/4)/2=1.5/2=0.75$.
Why may adding $YZ$ to a linear imputation regression still fail to match the exact conditional distribution?
Solution: The derived conditional mean is generally a ratio whose denominator depends on $z$, and the conditional variance also depends on $z$. A linear mean term $YZ$ alone does not represent those features.

Back to Tutorials