Why a Regression Model for a Continuous Imputation?
When a continuous variable has missing values, a regression model can express how its observed values vary with other information in the data. It gives a conditional distribution for the missing values rather than assigning every case the same overall average. This matters when the variable is related to other recorded characteristics: a single mean would erase those relationships and understate the variation among plausible values.
In a MICE update, the incomplete variable is the response for its conditional model. Other variables can serve as predictors, including variables whose missing values have already been filled in during the current iteration. The update estimates the relationship using cases where the response is observed, then uses the model to generate values for cases where it is missing. Observed response values are retained; the update does not replace them.
Model Specification and Assumptions
The model states that the conditional mean of the response is linear in the included predictors and that residual variation around that mean is normally distributed with a common variance. The usual regression assumptions also treat errors for different individuals as independent, conditional on the predictors. The conditional mean and residual distribution are assumptions about the data-generating relationship, not guarantees produced by fitting a regression.
Predictors should be selected to represent useful information about the incomplete variable and, for an analysis model, to preserve relationships relevant to the intended estimates. Include suitable auxiliary variables when they help predict missing values or missingness. A categorical predictor requires a coding scheme: for a two-category variable, an indicator coded 0 or 1 makes its coefficient a difference from the category coded 0. Nonlinear relationships may require transformations, polynomial terms, or other appropriate model terms; simply writing down a straight line does not make a curved relationship linear.
The predictors must have usable values for every case receiving an imputation. In chained equations, a predictor that is itself incomplete can contribute its current imputed value. That updating feature does not remove the need to assess whether the conditional model is sensible. Nor does a normal regression specification establish assumptions about the missingness process; conclusions from multiple imputation still depend on the broader missing-data assumptions and the relevance of variables included in the imputation models.
Worked Example: Calculating a Simple Fitted Regression
Suppose a hypothetical pilot data set contains four complete pairs of a predictor \(X\) and continuous response \(Y\), measured in arbitrary laboratory units: \((0,10),(1,12),(2,13),(3,15)\). Fit the simple linear regression by least squares to illustrate how the intercept and slope are determined.
The sample means are \(\bar{x}=(0+1+2+3)/4=1.5\) and \(\bar{y}=(10+12+13+15)/4=12.5\). The least-squares slope is the cross-product sum divided by the predictor sum of squares:
The intercept is \(\hat{\beta}_0=\bar{y}-\hat{\beta}_1\bar{x}=12.5-(1.6)(1.5)=10.1\). Thus the fitted conditional mean is \(\hat{Y}=10.1+1.6X\). The fitted values for the four observations are \(10.1,11.7,13.3,14.9\), giving residuals \(-0.1,0.3,-0.3,0.1\). Their squared sum is \(0.01+0.09+0.09+0.01=0.20\). The usual residual standard deviation estimate is \(s=\sqrt{0.20/(4-2)}=\sqrt{0.10}=0.316\) laboratory units.
This calculation illustrates the model components, not a reliable model for real imputation: four observations are far too few to assess the assumptions or estimate a stable relationship. A fitted straight line can be calculated even when the sample is inadequate or the model is unsuitable.
From a Conditional Mean to a Plausible Imputed Value
For a case with predictor values \(x_{i1},\ldots,x_{ip}\), the fitted conditional mean is \(\hat{y}_i=\hat{\beta}_0+\sum_j\hat{\beta}_jx_{ij}\). This is a center of the model's conditional distribution, not generally a suitable imputed value by itself. A value drawn from a normal distribution centered at that mean also reflects residual variation, so cases with the same predictors need not receive identical imputations.
The numerical example below isolates residual variation using fixed fitted coefficients to make the calculation transparent. It is not the full multiple-imputation procedure: proper multiple imputation also accounts for uncertainty in the fitted model parameters. That uncertainty is treated in the subsequent material on predictive draws and parameter and residual uncertainty.
Worked Example: Adding Residual Variation to Predictions
In a hypothetical clinical data set, let \(Y\) be a continuous biomarker measured in units. Suppose a fitted conditional model is \(Y=50+0.30(\text{age})+2(\text{treatment})+\varepsilon\), where treatment is coded 0 for standard care and 1 for the new treatment, and the estimated residual standard deviation is 4 units. Consider two records with missing biomarker values. For illustration, use standardized residual draws \(z=-0.50\) and \(z=1.00\), so the residual contributions \(z(4)\) are \(-2\) and \(4\) units.
For a 40-year-old receiving the new treatment, the fitted mean is \(50+(0.30)(40)+(2)(1)=64\) units. The illustrative draw is \(64+(-0.50)(4)=62\) units.
For a 60-year-old receiving standard care, the fitted mean is \(50+(0.30)(60)+(2)(0)=68\) units. The illustrative draw is \(68+(1.00)(4)=72\) units. Both values differ from their fitted means because residual variation is part of the conditional model.
The age coefficient describes a 0.30-unit difference in conditional mean per year, holding treatment status fixed. The treatment coefficient describes a 2-unit difference in conditional mean between the coded treatment groups at the same age. These are model-based associations, not causal effects merely because treatment appears in the equation.
In an actual MICE update, the model is fitted using observed values of the target response and the current available predictor values. The procedure then generates values for the missing target entries and passes the updated variable to later conditional updates. Parameter uncertainty must be represented across imputations as well as variation around a regression line; repeatedly using fixed coefficients and only adding residual noise does not, by itself, deliver proper multiple imputation.
Choose the continuous target, predictors, coding, and any needed nonlinear or interaction terms. State the linear-mean and residual assumptions.
Estimate the regression relationship from cases with observed target values, using the current predictor values available in the chain.
Use the fitted conditional relationship and variation in the predictive distribution to produce plausible values for missing target entries; leave observed target values unchanged.
Inspect model fit and generated-value behavior, and assess chain diagnostics separately. A stable chain can still repeatedly use a misspecified conditional model.
Checking the Model and Recognizing Failure
Diagnostics ask whether the conditional model is a reasonable description of the observed response and whether its imputations behave plausibly. Residual-versus-fitted plots can reveal curvature, unequal spread, or outliers. A normal probability plot can reveal severe departures from normal residuals. These checks are imperfect, especially when the observed response sample is small, but they can identify obvious problems. Also compare imputed and observed values in context: implausible ranges, excessive clustering at the fitted means, or patterns inconsistent with known constraints deserve investigation.
Check predictor support as well. A model may be asked to impute for ages or covariate combinations scarcely represented among observed-response cases. Such extrapolation can produce unreliable predictions even if the fitted line looks reasonable within the observed range. Sparse combinations of categorical predictors can cause a similar problem. In each case, the issue is not fixed simply by retaining a normal distribution.
Chain diagnostics address a different question: whether the sequence of updates appears to have settled and explored its distribution. Trace behavior and between-chain comparisons can flag drift or poor exploration, but they cannot establish linearity, homoscedasticity, or a correct predictor set. Conversely, satisfactory residual plots do not establish that an FCS chain has mixed well. Both model fit and chain behavior matter.
Worked Example: Spotting a Missing Nonlinear Pattern
Suppose a hypothetical study models a continuous outcome in millimeters using age as the sole predictor. The observed-response residual plot shows that residuals are mostly positive at ages 20 and 60, but mostly negative around age 40. For a simple numerical illustration, suppose the fitted line predicts 20 mm at ages 20, 40, and 60, while observed conditional averages in those age bands are approximately 24, 16, and 24 mm.
At age 20, the average residual is \(24-20=4\) mm; at age 40 it is \(16-20=-4\) mm; at age 60 it is \(24-20=4\) mm. The systematic \(+,-,+\) pattern is inconsistent with a linear conditional mean: the model underpredicts at the ends and overpredicts in the middle. The residual pattern indicates that age may have a curved relationship with the outcome. A quadratic term or another suitable mean specification could be considered, then checked against the observed data and scientific context.
It would be a mistake to respond by merely increasing the residual standard deviation. A larger common spread could add noise, but it would not correct the systematic mean errors shown by \(4,-4,4\) mm. The conditional mean specification needs attention.
Alternatives, Limitations, and Common Mistakes
A normal linear regression is attractive because coefficients are interpretable, conditional predictions are direct, and residual variation has a familiar form. It is not automatically the right model for every continuous variable. A skewed response, a bounded measurement, a point mass at zero, or variance that changes with predictors may call for a different conditional model. Those data structures should not be forced into a normal model merely because the variable is numeric.
| Approach | What it does | Main limitation for imputation |
|---|---|---|
| Normal linear regression | Models a conditional mean and normal residual variation | Can misrepresent curvature, unequal spread, skewness, or bounds |
| Mean imputation | Fills each missing value with one common average | Ignores predictor relationships and suppresses variation among completed values |
| Deterministic regression prediction | Fills missing values with fitted conditional means | Assigns the same prediction to identical predictors and omits residual variation |
A frequent mistake is to say, “The regression predicts 64, so 64 is the imputed value.” That confuses a conditional mean with a draw from a conditional distribution. Another mistake is to treat a residual perturbation around fixed estimated coefficients as the whole multiple-imputation procedure. The correction is to distinguish the regression model, the predictive variation around its mean, and uncertainty in the fitted parameters. A third mistake is to assume that good chain traces validate the regression form; chain stability and model adequacy are separate checks.
No single conditional model guarantees that all substantive relationships are preserved, and separate conditional regressions in FCS need not correspond to one coherent joint distribution. Model choice, predictor inclusion, missingness assumptions, and convergence all affect the usefulness of the completed data. Normal regression is a model to assess, not a universal default that makes these concerns disappear.
Check Your Understanding
Answer each question, then compare with the short solutions below.
- A fitted model is \(Y=8+1.5X+\varepsilon\), with residual standard deviation \(2\). What is the conditional mean when \(X=4\), and what is one illustrative value if the standardized residual draw is \(-1\)?
- In a model with treatment coded 0 for control and 1 for active treatment, what does a treatment coefficient of \(-3\) mean, conditional on the other predictors?
- Why is deterministic regression prediction not equivalent to generating imputed values from a normal linear model?
- A residual plot shows increasing vertical spread as fitted values increase. Which model assumption is in question, and why might a normal model with one common residual standard deviation be inadequate?
Solutions. (1) The mean is \(8+(1.5)(4)=14\). The illustrative value is \(14+(-1)(2)=12\). (2) The fitted conditional mean is 3 response units lower for active treatment than control at the same values of the other predictors; this is an association in the model, not automatically a causal effect. (3) Deterministic prediction assigns only the fitted mean, omitting residual variation and giving identical predictions to cases with identical predictors. It also does not represent parameter uncertainty. (4) Constant residual variance is in question. If spread changes with fitted values, one common \(\sigma\) may understate variation in some regions and overstate it in others.