Stochastic Regression Imputation and Marginal Variance

Why Add a Residual Draw?

Regression predictions can preserve the relationship between an outcome and its predictors, but a prediction is a conditional average, not an individual outcome. If every missing outcome is replaced by its predicted average, all residual variation among those outcomes disappears. Adding a random residual draw can restore that part of the spread, provided the regression and residual model describe the data adequately.

The focus here is the variance of the completed outcome. The key comparison is between deterministic regression imputation, which fills in the fitted prediction, and stochastic regression imputation, which adds a draw representing the outcome's residual variation. This comparison concerns residual spread; it does not imply that one random completed dataset captures all uncertainty about missing values or fitted model parameters.

Variance Under the Idealized Model

First consider an idealized setting in which the conditional mean \(m(X)\) and residual variance \(\sigma^2(X)=\operatorname{Var}(Y\mid X)\) are known. Let \(R=1\) indicate that \(Y\) is observed and \(R=0\) that it is missing. Define \(q(X)=P(R=1\mid X)\), the probability of observation at a given predictor value. Assume missingness is conditionally independent of \(Y\) given \(X\), the MAR condition for this setup, and that \(q(X)>0\) in the predictor ranges of interest.

For stochastic imputation to restore the intended spread, the residual draw must represent the correct conditional residual distribution. For example, under a homoscedastic normal regression model, one could draw \(\varepsilon^*\sim N(0,\sigma^2)\) independently for each missing case, conditional on its predictors. If the residual variance changes with \(X\), the draw must reflect that changing variance rather than use one common variance without justification.

For a missing case, deterministic imputation puts all its completed values at \(m(X)\). Among cases with the same \(X\), only the observed fraction \(q(X)\) retains residual variation. Stochastic imputation instead gives missing cases residual variation as well. The law of total variance makes the comparison precise.

For either method, the conditional mean is \(m(X)\). With deterministic imputation, the conditional variance is \(q(X)\sigma^2(X)\): observed cases retain their residual variation, while missing cases contribute none. With correctly drawn residuals, the conditional variance is \(\sigma^2(X)\), since both observed and imputed cases have the modeled residual distribution. Thus the idealized marginal variances are

Subtracting the first expression from the second gives the marginal variance difference attributable to restoring residual spread:

The difference is nonnegative. It is larger where missingness is more common and where the conditional residual variance is greater. Under these ideal assumptions, the stochastic completed outcome has the same marginal variance as the full-data outcome, while the deterministic completed outcome is short by the residual variance removed among missing cases. The result is about the population distribution under the model; finite samples and estimated regressions need not reproduce it exactly.

Worked Variance Comparisons

What One Random Draw Looks Like in Data

The variance result describes a distribution under specified assumptions. In a finite dataset, a particular set of residual draws can make the completed sample variance larger or smaller than the deterministic sample variance. Random imputation restores residual variability in expectation under the model; it does not force every completed dataset to have the exact full-data variance.

Assumptions, Diagnostics, and Limits

The variance restoration argument depends on the conditional model, not merely on adding random numbers. In practice, regression parameters and residual variance are estimated from observed cases, so the ideal formulas are a reference rather than an exact description of every dataset. Before interpreting a stochastic imputation as restoring spread, check the following:

• Conditional mean: the regression should represent the relationship between the outcome and relevant predictors. Residual patterns against fitted values or predictors can reveal curvature, omitted structure, or unequal variance.
• Residual distribution: the draw method should reflect the residual distribution, including heteroscedasticity if present. A normal draw with a single variance can be inappropriate when residual spread changes across predictor values.
• Missingness and support: the conditional independence assumption and adequate observed cases across the predictor range matter. If some predictor patterns have no observed outcomes, their residual behavior cannot be learned directly from those data.
• Draw implementation: every missing outcome needs a residual draw with the intended variability. Replacing draws by zero, or adding noise with a variance chosen without regard to the residual model, does not have the stated justification.

A common mistake is to conclude that because residual draws restore some marginal spread, stochastic regression imputation has solved the uncertainty problem. It has not: this calculation restores residual variation under a specified model. If regression coefficients are treated as fixed, the procedure does not also draw alternative plausible coefficients. Nor does random variation alone correct a misspecified conditional mean, a poor residual distribution, or unsupported predictor patterns. A single random completed dataset can therefore have an imperfect variance and cannot represent all uncertainty about the missing values.

The comparison is about marginal variance, not a guarantee that every other feature of the completed data is correct. Stochastic residual draws can address the particular loss of conditional residual spread caused by deterministic predictions, but the model and missingness assumptions still require scrutiny.