Deterministic Regression Imputation Removes Residual Spread

Why a Regression Trend Is Not the Whole Outcome

A regression model can use observed predictors to estimate a missing outcome in a way that follows a systematic trend. For example, if an outcome tends to increase with a measured baseline value, regression predictions can reflect that increase. But an individual outcome is generally not just its predicted value: it also varies around the regression trend. Replacing a missing outcome with its fitted prediction preserves the trend while assigning that case no deviation from it.

This is the defining limitation of deterministic regression imputation. It is important to separate two questions: whether the regression describes how the outcome changes with predictors, and whether a single fitted value represents the variation of individual outcomes around that trend. A model may describe the trend adequately while deterministic replacement still removes residual spread from the imputed cases.

Definition: Let $Y_i$ be an outcome and $X_i$ a set of observed predictors for individual $i$. Fit a regression model to cases whose outcomes are observed, and let $\hat{m}(X_i)$ denote the fitted conditional mean for case $i$. Deterministic regression imputation replaces a missing $Y_i$ by $\hat{m}(X_i)$, with no random deviation added: $$ Y_i^{\mathrm{imp}}=\hat{m}(X_i). $$ For an observed case, the regression residual is $e_i=Y_i-\hat{m}(X_i)$. For an imputed case, the imputed value minus its fitted value is exactly zero by construction; this is not evidence that the case's unobserved true residual was zero.

Model and Conditions for Using the Regression

For a single quantitative predictor, a common model is

Y_i=\beta_0+\beta_1X_i+\varepsilon_i, \qquad E(\varepsilon_i\mid X_i)=0.

Here $\beta_0+\beta_1X_i$ is the conditional mean of the outcome given the predictor, and $\varepsilon_i$ is the individual deviation from that mean. The conditional mean-zero assumption says that, at a given predictor value, the deviations average to zero. A constant conditional variance, $Var(\varepsilon_i\mid X_i)=\sigma^2$, is often used in the standard linear regression model, but it is not necessary to define deterministic imputation. Normal errors are also not required simply to calculate fitted means.

To use a regression fitted among observed outcomes to predict missing outcomes, the fitted relationship must be relevant to the missing cases. In particular, the chosen predictors and model form should adequately describe the conditional mean, and the cases with observed outcomes should provide information across the predictor values of the missing cases. A usual missingness condition supporting this approach is that, given the included observed predictors, outcome observation does not depend further on the unobserved outcome. If missingness still depends on the missing outcome after conditioning on those predictors, the observed cases may not represent the missing cases, and the fitted mean can be biased for them.

Even when these conditions are reasonable and the conditional mean is estimated well, deterministic replacement does not include an individual residual term for a missing case. The missingness assumptions may support estimating the trend; they do not make each case's true deviation from that trend disappear.

Worked Example: Fitting a Line and Imputing Two Outcomes

Consider a hypothetical data set with four observed outcomes and two missing outcomes. The predictor and observed outcome pairs are $(0,10)$, $(1,13)$, $(2,13)$, and $(3,16)$. The missing outcomes belong to cases with predictor values $1.5$ and $2.5$. Fit an ordinary least-squares line using the four observed pairs, then calculate the deterministic imputations.

The observed predictor mean is $\bar{x}=1.5$, and the observed outcome mean is $\bar{y}=13$. The slope and intercept are

\hat{\beta}_1= \frac{\sum (x_i-\bar{x})(y_i-\bar{y})} {\sum (x_i-\bar{x})^2} = \frac{4.5+0+0+4.5}{2.25+0.25+0.25+2.25} = \frac{9}{5} =1.8,

\hat{\beta}_0=\bar{y}-\hat{\beta}_1\bar{x} =13-(1.8)(1.5)=10.3.

Thus the fitted line is $\hat{y}=10.3+1.8x$. The missing outcome at $x=1.5$ is replaced by

\hat{y}=10.3+(1.8)(1.5)=13.0,

and the missing outcome at $x=2.5$ is replaced by

\hat{y}=10.3+(1.8)(2.5)=14.8.

The two replacements follow the predictor-based trend: the larger predictor value receives the larger predicted outcome. Now inspect the observed residuals. The fitted values at $x=0,1,2,3$ are $10.3,12.1,13.9,15.7$, so the residuals are $-0.3,0.9,-0.9,0.3$. Their squared sum is

$$ SSE=(-0.3)^2+(0.9)^2+(-0.9)^2+(0.3)^2=1.8. $$

The fitted regression therefore has observed residual spread. By contrast, each imputed value equals its fitted value exactly, so each imputed value contributes zero deviation from the line. The actual missing outcomes could have been above or below those predictions; the data do not reveal their residuals.

Why the Residual Spread Is Removed

The loss follows directly from the imputation rule, not from a special property of one data set. Suppose a missing case has predictor value $X_i=x$. Its fitted conditional mean is $\hat{m}(x)$, and deterministic imputation sets $Y_i^{\mathrm{imp}}=\hat{m}(x)$. Therefore,

Y_i^{\mathrm{imp}}-\hat{m}(x) = \hat{m}(x)-\hat{m}(x) =0.

If the outcome model is $Y_i=m(x)+\varepsilon_i$, the unobserved outcome would instead be $m(x)+\varepsilon_i$. The fitted mean targets the systematic part $m(x)$; it does not reveal or reproduce the case-specific deviation $\varepsilon_i$. Replacing the unknown outcome by the fitted mean therefore substitutes a central prediction for a variable outcome.

The distinction is especially clear when several cases have the same predictor values. They share the same fitted mean under the model, but their outcomes need not be identical. Deterministic imputation gives them identical replacements. It retains differences in fitted values across predictor patterns, but not deviations among individuals with the same predictors.

Worked Example: Identical Predictors, Different Possible Outcomes

Suppose a hypothetical regression model predicts a clinical outcome in millimetres of mercury (mmHg) using a baseline predictor $X$, and at $X=4$ the fitted value is $18$ mmHg. Two observed cases with $X=4$ have outcomes $15$ and $21$ mmHg. Their residuals are

15-18=-3\text{ mmHg}, \qquad 21-18=3\text{ mmHg}.

Both outcomes have the same fitted mean, but their observed deviations from it differ. If a third case at $X=4$ has a missing outcome, deterministic regression imputation assigns $18$ mmHg. Its imputed deviation is $18-18=0$ mmHg. That zero is imposed by the rule; it does not establish that the missing case's true outcome would have equalled $18$ mmHg. The replacement preserves the model's prediction at $X=4$, while representing none of the case-to-case residual variation visible in the two observed outcomes.

What the Completed Data Can and Cannot Show

A completed data set from deterministic regression imputation can look orderly: values track the fitted relationship, and missing cases have plausible predictions given their predictors. That appearance should not be mistaken for a full representation of the outcome distribution. Imputed outcomes are constrained to the fitted surface (the line in a one-predictor model), whereas observed outcomes can lie above or below it.

This method does not guarantee that every overall summary, such as the completed-data variance, moves in one fixed direction by a particular amount. The effect on an overall summary also depends on the predictor values and the observed cases. The narrower conclusion here is exact: the imputed cases have no residual deviation from their fitted means. How this combines with variation across predictor values is a separate calculation.

This matters when the analysis depends on outcome variability, relationships among variables, or uncertainty in estimates. A completed data set treats each fitted prediction as though it were the missing value itself. It therefore fails to carry forward the unresolved variation in those outcomes. A regression trend can be useful for prediction, but that alone does not make a single deterministic completed data set a faithful representation of all the variation relevant to statistical analysis.

Worked Example: Checking Predictions and Residual Spread in Context

In a hypothetical clinical analysis, the outcome is change in a measure, in mmHg, and $X$ is a baseline score. A regression fitted to observed outcomes gives $\hat{y}=5+0.6x$. Three cases have missing outcomes and baseline scores $20,30,$ and $40$. The observed-case regression residual standard deviation is reported as $8$ mmHg.

The deterministic imputed values are

\hat{y}_{20}=5+(0.6)(20)=17\text{ mmHg}, \quad \hat{y}_{30}=5+(0.6)(30)=23\text{ mmHg}, \quad \hat{y}_{40}=5+(0.6)(40)=29\text{ mmHg}.

These predictions increase by $6$ mmHg for each $10$-point increase in the baseline score, exactly following the fitted slope. But the observed residual standard deviation of $8$ mmHg describes substantial spread around the fitted relationship among cases with observed outcomes. Each of the three imputed outcomes has zero deviation from its own fitted value in the completed data. The $8$ mmHg figure is not an estimate of each missing case's residual, nor does it tell us which direction that case would differ from its prediction. It does, however, show why treating the three predictions as if they had no unresolved variation is a limitation.

Diagnostics, Alternatives, and Common Pitfalls

Diagnostics should examine the regression model and the consequences of using its fitted means, while recognizing that the true residuals for missing outcomes cannot be directly inspected. Useful checks include:

Plot observed outcomes against predictors and inspect residuals for curvature, changing spread, or influential cases. A poorly specified conditional-mean model can produce poor predictions in addition to the residual-spread problem.
Compare predictor distributions for cases with observed and missing outcomes. Missing cases far outside the observed predictor range require extrapolation, where the fitted means may be unreliable.
Check whether predictors thought to account for outcome observation are available and included. If outcome observation may still depend on the unobserved outcome after conditioning on them, a regression fitted to observed cases may not transfer appropriately.
Notice whether many imputed values repeat or fall exactly on a fitted line or surface. This is an expected signature of deterministic replacement, not evidence that the missing values were measured precisely.

Mean imputation ignores predictor-based trends, whereas deterministic regression imputation uses them. Neither method, when it supplies one fixed value for each missing outcome, represents that case's unobserved residual. Approaches that add a residual or otherwise reflect uncertainty are alternatives in principle, but they require additional choices and have their own limitations; a single random draw also does not by itself account for all uncertainty about missing values and model parameters.

Common mistake: “The regression model has residual variation among observed cases, so that variation is automatically present in the imputed outcomes.” The correction is that residual spread in the fitted model describes observed deviations, while deterministic imputation sets every missing outcome equal to its prediction. The estimated spread is not added to those replacements.

A related mistake is to say that a zero imputed residual means the missing person's outcome is exactly known. It means only that the completed value was defined to equal the prediction. The actual outcome remains unobserved, so its deviation from the conditional mean is unknown.

Key takeaway: Deterministic regression imputation retains the fitted predictor-based trend by assigning each missing case its conditional-mean prediction. It removes residual spread from those cases because each imputed value is exactly on the fitted regression surface.

Check Your Understanding

Answer each question, then compare with the solutions below.

A fitted model gives $\hat{y}=4+2x$. What deterministic value is imputed when $x=3$, and what is its deviation from its fitted value?
At a fixed predictor value, two observed outcomes are $12$ and $18$, and the fitted mean is $15$. Give their residuals. If another case at that predictor value is imputed deterministically, what residual does the completed data assign it?
Does a missing case's zero imputed residual show that its unobserved outcome was equal to the conditional mean? Explain.
Name one model diagnostic and one missingness-related concern that can make regression imputation problematic, apart from its removal of residual spread.

Solutions. 1. The imputed value is $4+2(3)=10$; its deviation from its fitted value is $10-10=0$. 2. The observed residuals are $12-15=-3$ and $18-15=3$. The deterministic replacement is $15$, so its imputed deviation is $15-15=0$. 3. No. The zero follows from assigning the fitted mean as the completed value; the actual outcome and its residual remain unknown. 4. For example, residual plots can reveal curvature or changing spread, indicating a poorly specified regression. A missingness concern is that outcome observation may still depend on the unobserved outcome after conditioning on the included predictors, so the observed-case relationship may not represent the missing cases.