Parameter And Residual Uncertainty In Continuous Imputation

Why Two Kinds of Uncertainty Matter

A continuous imputation from a regression model is not just a predicted mean. For a case with predictors \(x\), the regression mean \(x^\mathsf{T}\beta\) is uncertain because \(\beta\) is estimated from a finite observed sample. Even if that mean were known exactly, an individual outcome would still vary around it. A proper predictive draw represents both sources: uncertainty about the regression parameters and residual variation among individuals.

These sources have different consequences. Coefficient uncertainty moves the fitted regression surface; it can shift predictions for many missing cases together. Residual uncertainty represents case-specific deviations around that surface. A procedure that includes one source but not the other generally produces imputations that are too concentrated in some respect.

Model and Predictive Draws

Suppose \(y_o\) denotes the observed values of the target variable in the current MICE update, and \(X_o\) is the corresponding design matrix. For a case to be imputed, let \(x_m\) be its predictor vector, including an intercept if the model has one. The normal linear model is

Here \(\beta\) is a \(p\)-component coefficient vector and \(\sigma^2\) is the residual variance. The model assumes a correctly specified linear conditional mean, independent errors with constant variance, and normally distributed errors. The observed design matrix must have full column rank, and there must be more observed outcomes than coefficients. For this imputation model to be used without modeling the missingness process directly, the usual missing-data framework also requires an appropriate missing-at-random assumption and distinct model parameters.

One standard Bayesian draw uses the reference prior \(p(\beta,\sigma^2)\propto 1/\sigma^2\). Write \(\hat{\beta}\) for the least-squares estimate, \(SSE\) for the residual sum of squares, and \(\nu=n_o-p\), where \(n_o\) is the number of observed outcomes used in the fit. Then draw

The asterisk marks a draw, not a new fixed parameter. For each missing case, make the predictive draw

The coefficient draw and variance draw are shared across cases in the same model update; the residual draws are case-specific and independent conditional on those parameter draws. This distinction matters when imputing several values at once.

Why the Predictive Variance Has Two Parts

The separation follows directly from the law of total variance. Conditional on a coefficient and residual variance, a new response has mean \(x_m^\mathsf{T}\beta\) and variance \(\sigma^2\). Averaging over parameter uncertainty gives

The first term is residual uncertainty. The second is uncertainty in the fitted conditional mean. Under the reference-prior model above, let \(h_m=x_m^\mathsf{T}(X_o^\mathsf{T}X_o)^{-1}x_m\). Then \(\operatorname{Var}(\beta\mid y_o)=E(\sigma^2\mid y_o)(X_o^\mathsf{T}X_o)^{-1}\), so

The “1” represents residual variation; \(h_m\) scales coefficient uncertainty at the particular predictor values. Cases with predictor values far from the observed design can have larger \(h_m\), and therefore greater parameter uncertainty. If \(\nu>2\), then \(E(\sigma^2\mid y_o)=SSE/(\nu-2)\). The full predictive distribution is a Student \(t\) distribution with \(\nu\) degrees of freedom, center \(x_m^\mathsf{T}\hat{\beta}\), and scale \(s\sqrt{1+h_m}\), where \(s^2=SSE/\nu\). Its variance, when \(\nu>2\), is \(s^2(1+h_m)\nu/(\nu-2)\), consistent with the decomposition above.

What Is Lost When a Source Is Omitted?

The following comparisons describe incomplete procedures, not interchangeable versions of the same predictive draw. A deterministic fitted-mean imputation sets every imputed value to \(x_m^\mathsf{T}\hat{\beta}\). It omits both sources and places cases exactly on the fitted regression surface. Adding residual draws while keeping \(\hat{\beta}\) fixed restores individual scatter, but still treats the estimated surface as known. Drawing coefficients but setting the residual to zero allows the surface to vary across imputations, but places each value exactly on whichever surface was drawn.

A coefficient draw is not a replacement for a residual draw: it describes uncertainty in the estimated conditional mean, not the spread of individual outcomes around that mean. Conversely, a residual draw around a fixed estimate does not account for how the fitted mean could change if a different plausible parameter value were used. In the standard normal model, both draws are needed for the predictive distribution.

A further distinction appears when several missing cases are imputed using the same model update. Their coefficient uncertainty is generally shared, because one \(\beta^*\) is drawn for the update. Their residual errors are separate. Treating every case as if it had an independent coefficient draw would remove this shared component and give the wrong joint predictive behavior.

Checks, Alternatives, and Limitations

After fitting a continuous imputation model, check that the fitted model is numerically stable and that the draws behave as intended. The design matrix must not be rank deficient, and the residual degrees of freedom must be positive. Extremely large coefficient draws, implausibly large or small residual-variance draws, or unstable chains can signal weak information, influential observations, or model problems. Such draws should be investigated rather than silently discarded.

Also compare the conditional spread of imputed values with the observed data in relevant predictor regions. A cloud of imputations that is much narrower than observed residual scatter can indicate that residual variation was omitted or understated. Imputations collapsing onto the regression line are a direct warning that predictive residuals were not added. Conversely, excessive spread can arise from an inappropriate model or poorly estimated variance. These checks do not prove that the conditional model is correct: the assumptions of linearity, normal errors, and constant variance still require substantive assessment.

The main alternatives differ in which uncertainty they represent. Deterministic regression imputation is simple but omits both predictive components. Fixed-coefficient regression with residual draws represents individual variation but omits coefficient uncertainty. Parameter-draw-only procedures represent uncertainty in the fitted surface but omit residual variation. A bootstrap-based approach can represent parameter uncertainty by refitting to resampled observed cases, but it still needs predictive residual variation for individual outcomes. Predictive mean matching is another commonly used approach for continuous variables; it does not use this same normal residual-draw construction. The appropriate choice depends on the conditional model and the imputation goal.

The normal linear predictive draw has limitations. It can produce values outside a variable’s meaningful range, and its linear, homoscedastic, normal-error assumptions may be unsuitable for a particular outcome. Transformations, bounded-variable models, two-part models, or models allowing changing variance address different structures and require their own assumptions. Do not try to correct a poor model merely by changing how its coefficient and residual draws are combined.

Common Mistake