Worked Example With a Continuous Outcome

Set-up: Two Incomplete Predictors and a Continuous Outcome

Suppose the analysis of interest is a linear regression of a continuous outcome \(Y\) on two predictors, \(X_1\) and \(X_2\). Both predictors have some missing values, while \(Y\) is observed for the records considered here. The analysis model is

Y\mid X_1,X_2 \sim N(\beta_0+\beta_1X_1+\beta_2X_2,\sigma^2).

This specifies the conditional distribution of the outcome given the predictors. It does not, by itself, specify how the predictors are distributed. To construct a transparent pair of compatible imputation models, we add a joint distributional assumption for the predictors and an assumption about the outcome error.

Assumptions for the worked construction: The predictors are jointly Gaussian; conditional on them, \(Y\) follows the stated linear model with constant error variance; and the outcome error is independent of the predictors. The missingness mechanism must also be ignorable for the intended imputation analysis, given the variables included in the imputation model. These assumptions are a sufficient construction for deriving compatible conditionals, not a requirement that every practical MICE analysis use a joint Gaussian model.

In chained equations, the two predictor models update \(X_1\) conditional on \(Y\) and the current value of \(X_2\), then \(X_2\) conditional on \(Y\) and the current value of \(X_1\). Including \(Y\) helps preserve the association with the outcome represented in the analysis model. The key question here is more specific: what conditional models follow from one common joint model, and how can their numerical form be checked?

A Joint Model and Its Conditional Imputation Models

Consider the following hypothetical data-generating model. The predictors have mean zero, unit variances, and correlation \(0.4\). The outcome depends additively on both predictors, with an independent error having variance \(4\):

\begin{pmatrix}X_1\\X_2\end{pmatrix} \sim N\left[ \begin{pmatrix}0\\0\end{pmatrix}, \begin{pmatrix}1&0.4\\0.4&1\end{pmatrix} \right], \qquad Y=10+2X_1-1.5X_2+\varepsilon, \qquad \varepsilon\sim N(0,4).

The error \(\varepsilon\) is independent of \((X_1,X_2)\). Thus the analysis model has intercept \(10\), slopes \(2\) and \(-1.5\), and residual variance \(4\). The assumptions make \((X_1,X_2,Y)\) jointly Gaussian, so its conditional distributions are Gaussian linear regressions.

For a jointly Gaussian pair of variables \(A\) and a vector \(B\), the conditional mean of \(A\) given \(B\) is linear, with coefficient \(\operatorname{Cov}(A,B)\operatorname{Var}(B)^{-1}\); its conditional variance is \(\operatorname{Var}(A)-\operatorname{Cov}(A,B)\operatorname{Var}(B)^{-1}\operatorname{Cov}(B,A)\). Applying this result after conditioning first on the other predictor gives the chained-equation models.

Worked Example: Deriving Both Compatible Conditionals

First calculate the covariances needed for \(X_1\mid X_2,Y\). From the model, \(\operatorname{Cov}(X_1,Y)=2(1)-1.5(0.4)=1.4\), \(\operatorname{Cov}(X_2,Y)=2(0.4)-1.5(1)=-0.7\), and \(\operatorname{Var}(Y)=2^2+(-1.5)^2+2(2)(-1.5)(0.4)+4=7.85\).

Given \(X_2\), the conditional mean of \(X_1\) is \(0.4X_2\), and its variance is \(1-0.4^2=0.84\). The conditional mean of \(Y\) is \(10-0.7X_2\), and its conditional variance is \(7.85-(-0.7)^2=7.36\). The conditional covariance between \(X_1\) and \(Y\), given \(X_2\), is \(1.4-(0.4)(-0.7)=1.68\). Therefore,

X_1\mid X_2,Y \sim N\left( 0.4X_2+\frac{1.68}{7.36}(Y-10+0.7X_2), \;0.84-\frac{1.68^2}{7.36} \right).

Since \(1.68/7.36=0.2283\) and \(0.84-1.68^2/7.36=0.4565\), the equivalent regression form is \(X_1\mid X_2,Y\sim N(-2.283+0.2283Y+0.5598X_2,\;0.4565)\).

For the other conditional, given \(X_1\), the mean of \(X_2\) is \(0.4X_1\), and its variance is \(0.84\). The conditional mean of \(Y\) is \(10+1.4X_1\), with variance \(7.85-1.4^2=5.89\). The conditional covariance of \(X_2\) and \(Y\), given \(X_1\), is \(-0.7-(0.4)(1.4)=-1.26\). Hence,

X_2\mid X_1,Y \sim N\left( 0.4X_1-\frac{1.26}{5.89}(Y-10-1.4X_1), \;0.84-\frac{1.26^2}{5.89} \right).

Here \(1.26/5.89=0.2139\), so the conditional mean is \(2.139+0.6995X_1-0.2139Y\); the conditional variance is \(0.5705\). Both equations are derived from the same joint distribution. They are therefore compatible with each other and with the specified outcome model under the stated Gaussian assumptions.

Using the Conditional Models in an Imputation Cycle

The equations above describe distributions, not deterministic replacement rules. For an incomplete \(X_1\), the model supplies a conditional mean and a conditional variance given the observed \(Y\) and the current \(X_2\). A value is drawn from that conditional distribution. The update for a missing \(X_2\) uses the corresponding conditional distribution given \(Y\) and the current \(X_1\). If both predictors are missing in a record, the current value of the other predictor is used at each step of the cycle.

Worked Example: Imputing One Missing Predictor

Suppose a hypothetical record has \(X_2=1\), \(Y=12\), and a missing \(X_1\). The compatible conditional mean is

$$ -2.283+0.2283(12)+0.5598(1)=1.016. $$

The conditional variance is \(0.4565\), so the conditional standard deviation is \(\sqrt{0.4565}=0.6757\). Thus the model for this missing value is approximately \(N(1.016,0.4565)\), conditional on the values supplied. For illustration, if a standard-normal draw were \(z=0.5\), the corresponding draw with fixed parameters would be \(1.016+0.5(0.6757)=1.354\). A real multiple-imputation procedure must also account for uncertainty in estimated model parameters; repeatedly inserting the conditional mean alone would understate uncertainty.

The same reasoning applies when the missing value is \(X_2\), using its conditional model. In an estimated MICE procedure, the regression parameters and residual variance are estimated from available cases at each update, and appropriate parameter uncertainty is propagated when producing imputations. The numerical conditional distributions above are the population targets in this hypothetical construction; they are not claimed to be known in an applied data set.

Why the Outcome and Compatibility Matter

The outcome is part of both predictor conditionals because the target analysis relates each predictor to \(Y\). In the example, at \(X_2=1\), the conditional mean of \(X_1\) changes with \(Y\): it is \(-2.283+0.5598+0.2283Y\). Omitting \(Y\) would instead use the predictor-only conditional mean \(E(X_1\mid X_2)=0.4X_2\), which does not preserve this outcome-specific information.

Compatibility is not guaranteed merely because each imputation equation is a regression. A pair of regressions is compatible when both can be obtained as conditionals of one joint distribution. Here, the shared Gaussian model supplies that distribution. Independently selecting slopes and residual variances for the two conditionals may fail to correspond to any common joint model, even if each equation looks plausible on its own.

Key result: Under the stated joint Gaussian predictor model and homoscedastic Gaussian linear outcome model, the conditional models \(X_1\mid Y,X_2\) and \(X_2\mid Y,X_1\) are Gaussian linear regressions. Their coefficients and variances are constrained by the same joint covariance structure; they should not be treated as unrelated regressions.

Diagnostics, Alternatives, and Limitations

Diagnostics should ask whether the conditional models are plausible for the data, whether the observed and imputed values are compatible in scale and distribution, and whether the chained updates have stabilized. Useful checks include comparing observed and imputed predictor distributions, examining conditional relationships with \(Y\) and the other predictor, and inspecting trace plots or summaries across iterations and imputations. Extreme imputed values, implausible conditional slopes, poor mixing, or a mismatch between observed and imputed distributions are reasons to investigate the model rather than simply accept the output.

Joint modeling: A multivariate Gaussian joint model makes the common distribution explicit and directly produces the compatible conditionals. It can be restrictive when predictors are strongly non-Gaussian or include categorical variables.
Chained equations: Separate conditional models allow flexibility in choosing a model suited to each variable. But flexibility does not ensure compatibility; the equations and their implied relationships need scrutiny.
Omitting the outcome: This may simplify a predictor model, but it can discard information about the incomplete predictor that is relevant to the analysis relationship. The example shows the difference between \(E(X_1\mid X_2)\) and \(E(X_1\mid X_2,Y)\) directly.

This construction is intentionally narrow. Joint normality and linearity may be poor approximations, and real data can require different conditional distributions. It also does not establish that missingness assumptions hold, that predictors have adequate observed support, or that an imputation model is well estimated. The earlier material on compatibility and congeniality distinguishes the mathematical existence of a shared joint distribution from the broader question of whether imputation and analysis procedures target aligned inferential quantities.

Common Mistake

A common mistake is to assume that the analysis equation \(Y\mid X_1,X_2\) automatically determines the imputation models, or that using a linear regression for each missing predictor automatically makes the equations compatible. The correction is to state the additional joint assumptions, derive the conditionals from them, and check whether the chosen conditional models agree with that common structure. For the numerical model here, the outcome regression alone is not enough; the bivariate Gaussian distribution of the predictors and the independent error assumption are also used.

Worked Example: Detecting the Consequence of Omitting the Outcome

Return to the hypothetical record with \(X_2=1\) and \(Y=12\). The compatible model gives a conditional mean for \(X_1\) of \(1.016\), as calculated above. If \(Y\) were omitted and the predictor-only conditional \(X_1\mid X_2\) were used, its mean would be \(0.4(1)=0.400\), with variance \(0.84\). The difference in conditional means is \(1.016-0.400=0.616\).

This calculation does not prove that omitting \(Y\) always causes a particular amount of bias. It demonstrates that the two models encode different conditional distributions in this specified example. The outcome-aware model retains the relationship required by the common joint model; the predictor-only model does not reproduce that conditional information.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions given after it.

Under the worked model, what is \(\operatorname{Cov}(X_1,Y)\)? Solution: \(Y=10+2X_1-1.5X_2+\varepsilon\), so it is \(2\operatorname{Var}(X_1)-1.5\operatorname{Cov}(X_1,X_2)=2-1.5(0.4)=1.400\).
For \(X_2=0\) and \(Y=10\), calculate the conditional mean of missing \(X_1\). Solution: Substitute into \(-2.283+0.2283Y+0.5598X_2\): \(-2.283+0.2283(10)+0.5598(0)=0.000\), up to rounding.
What is the conditional variance for imputing \(X_2\) in the worked model? Solution: It is \(0.84-1.26^2/5.89=0.5705\). The conditional standard deviation is approximately \(0.7553\).
Why is a deterministic replacement by the conditional mean inadequate for multiple imputation? Solution: It supplies no residual variation and does not by itself propagate parameter uncertainty. The resulting completed data sets would not reflect the predictive uncertainty represented by the conditional distribution.
Does a pair of separately fitted linear regressions necessarily define compatible conditionals? Solution: No. Compatibility requires that both conditional distributions arise from a common joint distribution. Separate linear forms alone do not establish that such a distribution exists.