Why Nonlinearity Matters When Imputing Predictors
Suppose a generalized linear analysis relates an outcome \(Y\) to a predictor \(X\) through a nonlinear function of \(X\). For example, a logistic regression might use both \(X\) and \(X^2\), or a Poisson regression might use a quadratic effect. If some values of \(X\) are missing, the imputation model needs to preserve the information about \(X\) carried by the observed outcome. A simple imputation model that treats this relationship as linear can produce completed data in which the nonlinear pattern is weakened or distorted.
The key issue is that the relationship needed for imputation is the distribution of the missing predictor given the observed outcome and other variables. That distribution is not generally a linear regression, even if the analysis model has a simple formula for the outcome given the predictor. In a generalized linear model, it is shaped by both the outcome likelihood and the distribution of the predictor.
This target is a conditional distribution, not a recipe to insert \(X^2\) as an ordinary predictor when \(X\) itself is missing. Since \(X^2\) is unknown whenever \(X\) is unknown, treating the square as a separately observed variable cannot solve the problem. The imputation procedure must generate a value of \(X\) in a way that respects the nonlinear outcome model.
The Conditional Distribution Behind the Imputation
Assume, for illustration, that the outcome follows a logistic regression with a quadratic predictor effect:
Here \(Y=1\) denotes the event, and \(Z\) is a vector of additional predictors. The conditional distribution needed to impute \(X\) follows from Bayes' rule:
For a binary outcome, the likelihood contribution is \(p(X,Z)^Y\{1-p(X,Z)\}^{1-Y}\). Thus, the imputation distribution depends on the outcome through the full nonlinear predictor \(X\) and \(X^2\), not merely through a linear trend in \(X\). In a substantive-model-compatible fully conditional specification (SMC-FCS) approach, this likelihood contribution is combined with a model for \(X\mid Z\), and draws are made from the resulting conditional distribution.
The same logic applies to other generalized linear models. For a Poisson outcome with log link, the likelihood factor for a count \(Y\) is determined by its mean \(\mu(X,Z)\), including whatever nonlinear function of \(X\) appears in the analysis model. The exact form of the conditional distribution for \(X\) changes with the outcome family, but the principle does not.
Worked Examples
Worked Example: A Discrete Predictor in a Quadratic Logistic Model
Consider a hypothetical setting in which \(X\) takes values 0, 1, or 2, with \(P(X=x)=1/3\) for each value. Suppose a quadratic logistic outcome model gives the following probabilities of a non-event. These are evaluations of a logistic model whose linear predictor is quadratic in \(X\).
| \(x\) | \(P(Y=0\mid X=x)\) | \(P(X=x)P(Y=0\mid X=x)\) |
|---|---|---|
| 0 | 0.519171 | 0.173057 |
| 1 | 0.601725 | 0.200575 |
| 2 | 0.421725 | 0.140575 |
For a person with an observed non-event, the unnormalised conditional weights are \(P(X=x)P(Y=0\mid X=x)\). Their sum is
Dividing each weight by this sum gives
For example, \(0.140575/0.514207\approx0.27338\), which rounds to 0.2734. These probabilities are the appropriate conditional probabilities for drawing a missing \(X\) under this model. They reflect the nonlinear outcome relationship; simply drawing \(X\) from its marginal distribution would ignore the observed non-event.
Worked Example: A Continuous Predictor and a Logistic Outcome
Consider hypothetical data with \(X\mid Z\sim N(0,1)\), and a binary outcome with \[ \operatorname{logit}\{P(Y=1\mid X)\}=-1+1.2X-0.4X^2. \] Assume the model parameters are known for this illustration. For a person with \(Y=1\), the conditional density of \(X\) is proportional to the normal density times the event probability:
where \(\phi(x)\) is the standard normal density and \(\operatorname{logit}^{-1}(u)=1/(1+e^{-u})\). Compare the unnormalised density at \(x=0,1,2\).
At \(x=0\), the linear predictor is \(-1\), so the event probability is \(1/(1+e^1)\approx0.2689\). The normal density is \(\phi(0)\approx0.3989\), giving a kernel value \(0.3989(0.2689)\approx0.1073\).
At \(x=1\), the linear predictor is \(-1+1.2-0.4=-0.2\), so the event probability is approximately 0.4502. Since \(\phi(1)\approx0.2420\), the kernel value is \(0.2420(0.4502)\approx0.1089\).
At \(x=2\), the linear predictor is also \(-0.2\), and the event probability is again approximately 0.4502. But \(\phi(2)\approx0.0540\), so the kernel value is only \(0.0540(0.4502)\approx0.0243\).
These are density-kernel values, not probabilities at three points and therefore should not be normalized as though \(X\) were discrete. A continuous draw requires sampling from the full conditional density. The example shows how both the quadratic outcome likelihood and the predictor distribution affect the draw.
Worked Example: A Nonlinear Effect in a Poisson Model
Suppose a hypothetical count outcome follows \(Y\mid X\sim\operatorname{Poisson}(\mu(X))\), with \(\log\{\mu(X)\}=-0.3+0.5X-0.2X^2\). Let \(X\) take values 0, 1, and 2 with equal prior probabilities, and suppose the observed count is \(Y=2\). The Poisson likelihood is proportional to \(\exp\{2\eta-\exp(\eta)\}\), where \(\eta=-0.3+0.5X-0.2X^2\); the factor \(1/2!\) is common to all candidate values and cancels in normalization.
For \(X=0\), \(\eta=-0.3\), so the likelihood kernel is \(\exp\{-0.6-\exp(-0.3)\}=\exp(-1.3408)\approx0.2616\). For \(X=1\), \(\eta=0\), giving \(\exp(0-1)=0.3679\). For \(X=2\), \(\eta=-0.1\), giving \(\exp\{-0.2-\exp(-0.1)\}=\exp(-1.1048)\approx0.3313\).
Because the prior probabilities are equal, these values are proportional to the conditional probabilities. Their sum is approximately \(0.9608\), so
The imputation distribution uses the Poisson likelihood implied by the quadratic analysis model. A linear model for \(X\) given \(Y\) would not automatically reproduce these conditional weights.
Practical Imputation and Assumptions
The conditional-distribution argument clarifies what a valid imputation strategy must represent, but it does not guarantee that a particular implementation will do so. SMC-FCS is one approach: specify the substantive outcome model, specify a predictor model such as \(f(X\mid Z)\), and sample missing \(X\) values using their product with the outcome likelihood. Other carefully specified conditional imputation models can also be suitable when they represent the same relevant relationship.
The reasoning relies on several assumptions. The analysis model for \(Y\mid X,Z\) must be a reasonable representation of the outcome process, including its nonlinear terms. The predictor model \(f(X\mid Z)\) must adequately represent the distribution of \(X\) and allow plausible values in relevant covariate strata. The missingness mechanism must be ignorable for the imputation given the variables in the model, as under missing at random with the needed observed variables included. There must also be enough information to estimate the models and to generate plausible draws. In multiple imputation, uncertainty in estimated model parameters should be reflected in the imputations, rather than treating estimates as known constants.
A practical sequence is to define the analysis model first, identify every nonlinear term in it, choose an imputation method that preserves the resulting relationship, and then inspect the completed data. Checks should include the distributions of observed and imputed \(X\), plots of the outcome against \(X\) with the analysis-model curve overlaid, and comparisons across outcome groups. Chain traces or other convergence checks are also relevant when the sampler is iterative. These checks can reveal poor mixing, implausible tail values, or an imputed relationship that has become nearly linear despite a curved analysis model.
Alternatives, Pitfalls, and Limitations
A routine chained-equations model that includes \(Y\) as a predictor of \(X\) can be useful, but including the outcome is not by itself a guarantee of compatibility. If the conditional model for \(X\) cannot represent the way the nonlinear outcome likelihood changes across values of \(X\), the imputed data may understate or alter the nonlinear association. SMC-FCS directly incorporates that likelihood; a suitably specified flexible conditional model is another possible route. The choice depends on the variable types, sample size, and ability to fit and diagnose the required models.
A common mistake is to say, “The outcome is in the imputation model, so the quadratic relationship is preserved.” That conclusion does not follow. The relevant question is whether the conditional distribution used to draw missing \(X\) values can reproduce the outcome dependence implied by the full analysis model. A second mistake is to calculate \(X^2\) only for observed \(X\) and then treat that square as a complete predictor. For a missing \(X\), its square is missing too; it must be consistent with the value of \(X\) drawn during imputation.
Compatibility also has limits. It does not correct a misspecified substantive model, a poor predictor distribution, non-ignorable missingness, sparse data, or weak overlap. A more complex imputation procedure can be unstable when there are few observed cases or when the nonlinear model is overly flexible. The earlier material on Overlaps Missing Data addresses missingness mechanisms; here the emphasis is the conditional relationship required to retain a nonlinear generalized linear analysis.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- In a logistic analysis with \(\operatorname{logit}\{P(Y=1\mid X,Z)\}=\beta_0+\beta_1X+\beta_2X^2+\gamma^\mathsf{T}Z\), what two components determine \(f(X\mid Y,Z)\)?
- Why should the kernel values in the continuous-predictor example not be normalized as three probabilities?
- In the discrete logistic example, what is \(P(X=2\mid Y=0)\) to four decimal places?
- Name one diagnostic that could reveal that imputations have failed to preserve the nonlinear outcome relationship.
Solutions. (1) The outcome likelihood \(f(Y\mid X,Z)\) and the predictor distribution \(f(X\mid Z)\). (2) A continuous variable assigns probability to intervals, not to individual points; the three kernel values are density values at selected locations and do not cover the full distribution. (3) \(0.140575/0.514207\approx0.27338\), which rounds to 0.2734. (4) For example, plot the outcome against \(X\) in observed and imputed cases and check whether the imputed relationship retains the curved pattern specified by the analysis model.