Modeling Heteroscedastic Continuous Variables

Why Conditional Spread Matters

A continuous imputation model must describe more than the expected value of the missing variable. It must also represent how much individual values vary around that expectation. In a normal linear regression imputation model, the usual assumption is that this residual spread is the same at every predictor pattern. When the spread changes with the predictors, the conditional distribution is heteroscedastic.

This matters because an imputed value is drawn from a predictive distribution, not simply set equal to a fitted mean. If high-predictor cases naturally have more variability than low-predictor cases, a constant-variance model tends to make the former imputations too concentrated and the latter too dispersed. The conditional mean model can be reasonable while the imputed values and their uncertainty are still poorly represented.

In a MICE update, \(Y\) is the variable currently being imputed and \(X\) denotes the other variables used in its conditional model. The model is fitted using cases with observed \(Y\), then used to generate replacements for missing \(Y\). The normal linear model and predictive parameter and residual draws were introduced in the earlier material on normal linear regression and parameter and residual uncertainty. Here the key change is that the residual variance is allowed to depend on \(X\).

A Location-Scale Model

A standard variance-aware specification gives the conditional mean a regression form and models the logarithm of the conditional variance:

Here \(x_i\) contains predictors for the mean, \(z_i\) contains predictors for the variance, \(\beta\) and \(\gamma\) are model parameters, and \(N(\mu_i,\sigma_i^2)\) uses \(\sigma_i^2\) as the variance. The predictors in the two parts need not be identical. Exponentiating the variance equation gives \(\sigma_i^2=\exp(z_i^\mathsf{T}\gamma)>0\), so the model cannot imply a negative variance.

The model assumes that, conditional on the predictors, observations are independent and normally distributed with the stated mean and variance functions. The functional forms must be adequate for the data: a linear mean and a log-linear variance are modeling choices, not automatic truths. For an imputation analysis, the fitted conditional model should also be appropriate for the analysis and include relevant predictors and relationships. As with other imputation models, accounting for heteroscedasticity does not by itself resolve a nonignorable missingness mechanism.

Likelihood and a Short Derivation

For observed outcomes \(y_i\), define the residual \(r_i=y_i-\mu_i\) and variance \(v_i=\sigma_i^2\). Under the conditional normal model, the log likelihood, apart from no omitted data-dependent terms, is

This expression explains why the variance model affects both the fit and the uncertainty. A large squared residual contributes less when the model assigns that case a larger variance, but assigning very large variances everywhere is penalized by the \(\log(v_i)\) term. To see how the log-variance parameters respond, let \(\log(v_i)=z_i^\mathsf{T}\gamma\). Differentiating the \(i\)th likelihood contribution with respect to \(\gamma\) gives

Thus, cases whose squared residuals tend to exceed their modeled variances push the variance parameters upward in the corresponding predictor directions; cases whose squared residuals tend to be smaller push them downward. This is a justification for estimating a variance function from residual behavior, rather than merely visually labeling a plot. In practice, joint estimation or a suitable likelihood-based fitting procedure must account for uncertainty in both the mean and variance parameters.

For proper multiple imputation, the prediction step must represent uncertainty in the fitted model parameters as well as individual residual variation. For a given posterior or other valid parameter draw \((\beta^{(m)},\gamma^{(m)})\), a missing value at predictors \(x,z\) can be drawn as

The square root appears because the residual draw is scaled by the standard deviation, not the variance. Using a single fitted parameter estimate and only drawing residuals generally omits parameter uncertainty; replacing the residual draw with zero would omit individual residual uncertainty as well.

Working Through a Variance-Aware Imputation

The next example uses a deliberately specified log-variance function. In an analysis, its coefficients would be estimated from observed outcomes and uncertainty in those coefficients would be included in imputation.

What a Constant-Variance Model Can Miss

A constant-variance model may still estimate a useful average trend, but it assigns the same conditional residual spread across predictor values. That can create systematically miscalibrated predictive distributions. In the hypothetical data below, use the residual variances calculated above as illustrative estimates of the two group variances. If the two groups have equal sizes, pooling them under a common variance gives

The pooled spread is larger than the low-group spread and smaller than the high-group spread. The following comparison treats the two group means as fitted correctly and uses normal predictive intervals to illustrate the consequence of the different variance assumptions.

Diagnostics and Alternative Approaches

A variance model is only useful when its assumptions are reasonably compatible with observed data. Examine residuals after fitting the mean model and, where applicable, standardized residuals \(r_i/\hat{\sigma}_i\) after fitting the variance model. A plot of residuals against fitted means or important predictors can reveal a fan shape, changing band width, or subgroup-specific spread. After variance adjustment, standardized residuals should not retain a clear spread pattern; their distribution should also be checked for pronounced nonnormality or outliers.

A fan pattern can also result from a misspecified mean, omitted interactions, influential observations, or mixtures of groups. Check whether the conditional mean has a systematic residual trend before attributing all structure to the variance. A model with a suitable variance function should reduce predictor-related spread while not leaving a strong mean pattern. Where data permit, compare predicted and observed residual dispersion across predictor ranges. The variance model can be unstable when observed cases are sparse in some ranges or when the variance predictors are highly collinear.

A constant-variance normal regression is simpler and can be adequate when residual spread is roughly stable. Transforming a skewed response may sometimes stabilize variance, but a transformation changes the scale of the conditional model and can complicate interpretation after returning to the original scale; the earlier material on transforming skewed continuous variables addresses that distinct issue. If observations form meaningful groups, group-specific variances may be an alternative to a smooth variance function, but estimates can be imprecise when groups are small. Robust methods may reduce sensitivity to outliers, but they are not automatically a substitute for modeling a genuine systematic variance pattern.

Heteroscedasticity is not itself evidence of skewness, a point mass at zero, or a bounded outcome. Those features call for different considerations, as in the earlier material on two-part models and truncated continuous models. Conversely, allowing a changing variance does not guarantee normal tails or protect against influential outliers. These are model limitations to assess, not problems that the variance equation alone solves.