Why Omitting Curvature Can Matter
Suppose the analysis relates an outcome \(Y\) to a predictor \(X\) using a quadratic regression. If some values of \(X\) are missing, the imputation model must generate plausible values of \(X\) while retaining the relationship that the analysis will estimate. A linear imputation model for \(X\) can fail at this task, even if it includes the observed outcome as a predictor: the conditional distribution of \(X\) given \(Y\) need not be linear, symmetric, or even unimodal.
The issue is not that every linear imputation model must produce bias. The distortion depends on the data-generating relationship, the distribution of \(X\), the missingness pattern, and how much information the observed cases provide. But if the analysis relationship is substantially curved and imputation ignores that curvature, the completed data may imply a different quadratic relationship from the one supported by the observed data.
Consider an analysis model
Here \(\beta_2\) measures curvature in the conditional mean of \(Y\) as a function of \(X\). When \(X_i\) is missing, an imputation model might regress \(X\) on \(Y\) and other observed variables. The direction of this conditional model is reversed from the analysis regression. Reversing the variables does not preserve a quadratic relationship by simply using a linear regression: in general, \(X\mid Y\) is not adequately represented by a linear mean and a single constant-variance normal error distribution.
Why a Quadratic Relationship Can Produce a Complicated Conditional Distribution
A short derivation shows the source of the problem. Suppose, for illustration, that \(Y=\beta_0+\beta_2X^2+\varepsilon\), where \(\varepsilon\mid X\sim N(0,\sigma^2)\), and let \(f_X(x)\) denote the distribution of \(X\). Bayes' rule gives the conditional density
When \(\beta_2>0\) and \(y>\beta_0\), values near both \(x=+\sqrt{(y-\beta_0)/\beta_2}\) and \(x=-\sqrt{(y-\beta_0)/\beta_2}\) make the exponential term large. If the distribution of \(X\) gives support to both signs, the conditional distribution may have two modes. A linear regression for \(X\mid Y\) cannot represent two separated modes merely by changing its slope. Even a flexible model for the conditional mean alone may be insufficient if the conditional distribution is multimodal.
This derivation assumes a normal error distribution for the outcome conditional on \(X\), and it does not assert that this is the only possible data-generating model. It establishes a more general lesson: the analysis model and the predictor distribution jointly determine the conditional distribution needed for imputation. A linear conditional model does not follow automatically from a quadratic analysis model.
Worked Example: A Symmetric Quadratic Relationship
Suppose a hypothetical population has \(X\) equally likely to be \(-2,-1,1,\) or \(2\), and \(Y=X^2\) exactly. Thus the pairs are \((-2,4),(-1,1),(1,1),(2,4)\). Find the true conditional distribution of \(X\) given \(Y\), and compare it with a linear conditional-mean model.
For \(Y=1\), the only possible values are \(X=-1\) and \(X=1\), each with conditional probability \(1/2\). Therefore \(E(X\mid Y=1)=0\), and \(\operatorname{Var}(X\mid Y=1)=1\). For \(Y=4\), the only possible values are \(-2\) and \(2\), also each with probability \(1/2\). Thus \(E(X\mid Y=4)=0\), while \(\operatorname{Var}(X\mid Y=4)=4\).
A linear regression of \(X\) on \(Y\) has slope zero in this population: for each value of \(Y\), the positive and negative \(X\) values cancel. Its conditional mean is therefore zero for both outcome values. The mean is correct, but a single normal error distribution with constant variance does not reproduce the two conditional distributions. In particular, the true distribution at \(Y=1\) is concentrated at \(-1\) and \(1\), while at \(Y=4\) it is concentrated at \(-2\) and \(2\); the conditional variances differ as well.
The example separates two failures that are sometimes conflated. The conditional mean can be correctly specified while the conditional distribution is still wrong; and a linear model with constant variance does not reproduce the conditional spread or the two modes. For imputation, plausible conditional draws matter, not only fitted conditional means.
How the Completed Analysis Can Change
If imputed values are systematically closer to the center of the predictor distribution than plausible values should be, the derived \(X^2\) values may be too small. This can weaken estimated curvature. Other misspecifications can distort curvature in the opposite direction, so the direction of bias is not universal. A particular missingness pattern can also make the completed-data relationship look stronger or change its shape.
The following example uses conditional-mean imputation to make the arithmetic transparent. Conditional-mean imputation is not recommended as a multiple-imputation procedure: it suppresses uncertainty and can reduce variation. The calculation isolates one way a linear conditional mean can misrepresent the missing values. A stochastic linear imputation model avoids replacing every missing value by the same fitted mean, but it can still be wrong if its conditional distribution is incompatible with the quadratic relationship.
Worked Example: A Linear Imputation Changes the Quadratic Fit
Consider the hypothetical complete data below, generated by \(Y=X^2\). The values of \(X\) for the rows with \(Y=4\) are missing. The other three rows are observed and are used to fit a linear regression of \(X\) on \(Y\).
| Row | \(X\), complete value | \(Y\) | \(X\) observed? |
|---|---|---|---|
| 1 | -1 | 1 | Yes |
| 2 | 0 | 0 | Yes |
| 3 | 1 | 1 | Yes |
| 4 | -2 | 4 | No |
| 5 | 2 | 4 | No |
For the observed rows, \(\bar{Y}=2/3\) and \(\bar{X}=0\). The fitted slope is
The fitted intercept is \(\hat{\gamma}_0=\bar{X}-\hat{\gamma}_1\bar{Y}=0\). Thus the conditional-mean model predicts \(X=0\) for each missing row, since each has \(Y=4\). After this imputation, the values used in the quadratic analysis are \((X,Y)=(-1,1),(0,0),(1,1),(0,4),(0,4)\).
Fit \(Y=a+bX+cX^2\) to these completed values. By symmetry, the coefficient \(b\) is zero. There are three observations at \(X^2=0\), with outcomes \(0,4,4\), whose mean is \(8/3\); there are two observations at \(X^2=1\), with outcomes \(1,1\), whose mean is \(1\). The fitted line of \(Y\) on \(X^2\) therefore has intercept \(a=8/3\) and slope \(c=1-8/3=-5/3\). In contrast, the complete data satisfy \(Y=X^2\), with \(a=0\) and \(c=1\).
This small example is deliberately extreme: only three rows train the imputation model, and their observed \(X\) values are symmetric around zero. It demonstrates that an imputation model can make the completed quadratic analysis contradict the complete-data relationship. It does not establish that a linear imputation model always reverses a quadratic coefficient. With stochastic draws, the numerical fit would vary across imputations; systematic model misspecification can nevertheless persist across those draws.
Assumptions, Diagnostics, and Alternatives
Interpreting an imputation as a remedy for missing \(X\) requires assumptions about both the missingness process and the models. A common assumption is that \(X\) is missing at random conditional on the variables included in the imputation model: after conditioning on those observed variables, missingness does not depend further on the unobserved value of \(X\). This assumption does not imply that a linear imputation model is correctly specified. The analysis model also needs to represent the intended outcome relationship adequately; if the quadratic form is the target, the imputation should not erase the information needed to estimate it.
Inspect the imputed data in ways tied to the analysis relationship. For cases with observed and imputed \(X\), compare distributions across outcome ranges, including spread and tails, rather than checking only overall means. Plot \(X\) against \(Y\), marking observed and imputed values separately. For a quadratic relationship, inspect whether imputed values preserve plausible values of \(X^2\) across the range of \(Y\), and whether the completed-data fit recovers a coherent curve. In this symmetric example, a scatterplot of \(X\) against \(Y\) would also reveal that a single linear trend cannot capture both signs.
A useful comparison is between the analysis fitted in each completed data set and the analysis fitted to cases with observed \(X\). The estimates need not be identical: complete cases may be less precise and can be selected in ways that alter their distribution. But a marked change in curvature, implausible fitted values, or systematic disagreement between observed and imputed conditional distributions warrants investigation. Diagnostic checks can reveal incompatibility; they cannot prove the missing-at-random assumption or guarantee that an alternative model is correct.
The main alternatives are not interchangeable shortcuts. A linear conditional model is simple but may omit needed curvature and distributional shape. A conditional imputation model that represents relevant nonlinear structure can be more appropriate, but simply adding a polynomial term in \(Y\) does not guarantee that the conditional distribution of \(X\mid Y\) is correct. A joint model can encode a coherent distribution for \(X\) and \(Y\), while a substantive-model-compatible approach targets the analysis relationship directly; each requires its own modeling assumptions and adequate data. Whatever approach is used, stochastic imputation should account for uncertainty in missing values and model parameters rather than treating fitted conditional means as known values.
Multiple imputation combines uncertainty across completed data sets, but it does not repair systematic misspecification shared by every imputation. Pooling can reflect between-imputation variability under the imputation procedure; it cannot make a conditional distribution appropriate when the model has ruled out important features such as a second mode. Likewise, increasing the number of imputations reduces Monte Carlo error in the pooled estimate, not bias caused by omitting nonlinearity.
Write the intended model explicitly, including \(X^2\) when curvature is part of the analysis.
Ask what distribution of missing \(X\) is plausible given the observed outcome and other variables; do not assume that reversing the regression gives a linear conditional model.
Compare observed and imputed values across outcome ranges, then inspect whether the completed-data analysis retains plausible curvature.
Use multiple imputations to represent uncertainty, but revise a systematically inadequate conditional model rather than relying on more draws.
Common Mistakes and Limitations
- Assuming the outcome automatically protects the analysis relationship. Including \(Y\) in a linear imputation of \(X\) may help preserve association, but it does not ensure that the conditional distribution of \(X\mid Y\) has the right curvature, spread, or modes.
- Checking only the imputed mean. A conditional mean can be correct while the conditional distribution is badly wrong, as when \(X\mid Y\) is symmetric and bimodal.
- Calling conditional-mean imputation multiple imputation. Replacing each missing value by one fitted mean suppresses uncertainty and often reduces variation. It is a diagnostic illustration here, not an adequate substitute for stochastic imputation.
- Claiming a fixed direction of bias. Omitting nonlinearity can attenuate, inflate, or otherwise distort a quadratic coefficient. The direction depends on the distributions and missingness pattern.
- Expecting pooling to fix misspecification. More imputations can improve numerical stability, but they cannot correct a conditional model that systematically excludes plausible values or shapes.
These limitations matter especially when missingness is substantial, the observed cases cover only a narrow range of \(Y\), or the quadratic analysis is sensitive to tail observations. In such settings, diagnostics may be weak because the data provide little information about the missing conditional distribution. Conclusions should then be accompanied by sensitivity analyses under defensible alternative imputation models.
Check Your Understanding
Answer each question, then compare with the solutions below.
- In the symmetric example with \(X\in\{-2,-1,1,2\}\) and \(Y=X^2\), why can a linear model for \(E(X\mid Y)\) give the correct mean and still be a poor imputation model?
- For the completed data in the second worked example, what are the fitted intercept and coefficient of \(X^2\) in the quadratic analysis?
- Does adding more imputations necessarily remove bias caused by an imputation model that omits nonlinearity? Explain.
- Name two features beyond the overall mean that should be checked when assessing imputed \(X\) under a quadratic analysis.
Solutions. (1) The conditional means are zero at both \(Y=1\) and \(Y=4\), but the true conditional distributions have two modes and different variances: 1 at \(Y=1\), and 4 at \(Y=4\). (2) Symmetry gives a coefficient of zero for \(X\); the means at \(X^2=0\) and \(X^2=1\) are \(8/3\) and \(1\), so the fitted intercept is \(8/3\) and the \(X^2\) coefficient is \(1-8/3=-5/3\). (3) No. More imputations reduce Monte Carlo error, but do not correct systematic misspecification shared across imputations. (4) Check conditional spread and tails across outcome ranges, and whether the imputed distribution has plausible modes or preserves the relationship between \(X^2\) and \(Y\).