Why Mean Imputation Distorts Regression

Why Replacing a Missing Value Is Not a Neutral Step

A regression analysis uses the values in the data set to calculate quantities such as variation in a predictor, its covariance with the outcome, and the residual variation around the fitted line. Replacing a missing value with a mean inserts a number, but it does not restore the value that was missing. The replacement affects those calculations in a systematic way: it places the imputed record exactly at the center of the observed values for that variable.

That central placement often makes the completed data look more concentrated than the original data would have been. It can also change the fitted regression coefficient and the standard error. The direction and size of coefficient changes are not universal; they depend on which variable is missing and on the pattern of missingness and observed values.

The numerical examples below use ordinary least squares (OLS) in a simple linear regression, where the fitted slope is the sum of cross-products of deviations divided by the sum of squared predictor deviations. These calculations describe how the completed data determine a line. To use the usual model-based standard errors and tests, one also needs the regression model’s error assumptions, such as independent errors with constant variance; normal errors are additionally used for exact small-sample \(t\)-inference. A substitution by itself does not establish those assumptions.

Mean Imputation Compresses a Variable’s Spread

Suppose \(r\) values of a variable are observed, with observed mean \(\bar{x}_{\mathrm{obs}}\), and \(m\) values are missing. After mean imputation, all \(m\) missing entries equal \(\bar{x}_{\mathrm{obs}}\). The mean of the completed variable is therefore still \(\bar{x}_{\mathrm{obs}}\), and the imputed entries contribute zero to the sum of squared deviations around that mean.

The reduction can be stated precisely by comparing the imputed data with the data that would have been available if the missing values were known. Let \(S_{\mathrm{obs}}\) be the observed values’ sum of squared deviations around their observed mean. Let \(\bar{x}_{\mathrm{mis}}\) be the mean of the actual missing values, and let \(S_{\mathrm{mis}}\) be their sum of squared deviations around their own mean. The completed-data sum of squares after mean imputation is \(S_{\mathrm{obs}}\). With the actual values included, the sum of squares is:

The last two terms are nonnegative. Thus mean imputation cannot increase the completed-data sum of squared deviations; it can leave it unchanged, or reduce it. It is strictly reduced if at least one actual missing value differs from the observed mean. The same comparison applies to sample variance, since both sums of squares are divided by the same completed sample size minus one.

What Happens to a Regression Slope?

For a simple regression of outcome \(Y\) on predictor \(X\), the OLS slope is the predictor-outcome cross-product sum divided by the predictor sum of squares:

Mean-imputing \(X\) changes the denominator by compressing predictor variation. It can also change the numerator, because the imputed records no longer have their actual predictor deviations from the mean. Since both parts of the ratio can change, there is no universal rule that predictor mean imputation must increase a slope, decrease it, or pull it toward zero.

Outcome Imputation Can Weaken the Predictor-Outcome Relationship

For a missing outcome, there is a useful population result under a specific condition. Suppose a proportion \(q\) of outcomes is observed, missingness is independent of both \(X\) and \(Y\) (the MCAR condition), and each missing outcome is replaced by the population outcome mean \(\mu_Y\). Let \(R\) equal 1 when \(Y\) is observed and 0 otherwise, and define \(Y^*=RY+(1-R)\mu_Y\). Since the replacement is the population mean, \(E(Y^*)=\mu_Y\). Also, because \(R\) is independent of both variables:

The variance of \(X\) has not been changed by imputing the outcome. Therefore, in this population idealization, the regression slope after outcome mean imputation is \(q\) times the slope before imputation. This result depends on the stated assumptions; missingness independent of \(X\) alone is not enough. In actual samples, the observed sample mean is used in place of \(\mu_Y\), so the exact finite-sample relationship need not equal this population result.

Why Conventional Standard Errors Can Mislead

In a simple regression with an intercept, the conventional OLS standard error for the slope is calculated as:

This formula is used under the usual linear-model conditions, including independent errors with constant variance; the standard \(t\)-based inference also uses normal errors for exact small-sample results. Mean imputation changes \(S_{xx}\), the fitted values, and hence the residual sum of squares \(SSE\). Moreover, a conventional calculation on the completed data treats substituted values as though they had been observed. It does not account for the uncertainty about what the missing values could have been. The resulting standard error can be distorted; it is often too small in practice, but its direction is not guaranteed in every data set.

The predictor example also shows why “mean imputation always makes the standard error smaller” is not a valid rule. For its four actual predictor values, the fitted line has \(SSE=2.30\), so \(MSE=2.30/2=1.15\) and the conventional slope standard error is \(\sqrt{1.15/5}=0.4796\). After imputation, the residuals are \(-1.25,-1.75,-1.25,4.25\); their squared sum is \(24.25\). Thus \(MSE=24.25/2=12.125\), and the conventional standard error is \(\sqrt{12.125/2}=2.462\). The numbers are calculations from a deliberately small example, not a basis for inference: the missing predictor is unknown in practice, and the model assumptions have not been established by this constructed data set.

If the missing values are outcomes, the imputed values can also make the residual pattern unusual, while the population result above shows that the fitted slope can be attenuated under MCAR. Neither effect is repaired just by reporting the standard OLS standard error calculated from the completed table.

Common Mistakes and a Practical Conclusion

• Claiming the variance must strictly decrease. The completed-data sum of squares cannot exceed the sum of squares using the actual missing values, but it can be equal when every missing value equals the observed mean.
• Claiming every slope moves toward zero. Predictor imputation can alter both the predictor spread and its cross-products with the outcome. The resulting change depends on the data and missingness pattern.
• Applying the outcome-slope result too broadly. Multiplication by the observed fraction \(q\) was derived for outcome missingness independent of both \(X\) and \(Y\), with the population outcome mean used for imputation. It is not a general consequence of missingness being independent of \(X\) alone.
• Treating imputed values as observed for uncertainty calculations. The usual standard error from the completed data does not include uncertainty about the missing values. A plausible numerical replacement is not the same as observing the value.

Mean imputation is easy to carry out, but it changes the distributional information and associations that regression uses. It can distort predictor variability, predictor-outcome relationships, fitted coefficients, and standard errors. The examples show why those changes should be examined separately rather than summarized by a single claim about bias or shrinkage.