Compatibility Is Not the Same as Truth
An analysis model can be deliberately simpler than the process that generated the data. For example, investigators may use a linear regression for a concise summary even when the outcome changes nonlinearly with a predictor. The resulting coefficient can still be a well-defined statistical target. But it is not automatically a summary of the true relationship that answers every scientific question.
With missing data, a further question arises: does the imputation model preserve the target of the stated analysis? This is distinct from asking whether the analysis model correctly describes the data-generating process. An imputation procedure can be aligned with a working analysis model while both are imperfect descriptions of reality. Compatibility is a property of model distributions; it is not a certificate of scientific adequacy.
Suppose \(Y\) is an outcome and \(X\) is a predictor with some missing values. The analysis model might specify \(Y\mid X\), while an imputation model specifies \(X\mid Y\) and other observed variables. Compatibility asks whether those conditional models can be generated by a shared joint model. The earlier material on Limitations of Pairwise Conditional Reasoning is relevant here: agreement between conditional models should not be assumed merely because each equation seems plausible on its own.
For inference from multiple imputation, additional assumptions matter. The missingness mechanism must be adequately handled, commonly by assuming missing at random conditional on variables in the imputation procedure; the relevant conditional distributions must be adequately specified; and the imputation procedure must represent uncertainty through proper draws. These conditions do not make a misspecified analysis model true. They support inference for the target defined by the chosen analysis, subject to its interpretation and inferential assumptions.
A Nonlinear Data-Generating Example
Consider a hypothetical population in which \(X\) takes the five values \(-2,-1,0,1,2\), each with probability \(1/5\), and the outcome is determined by \(Y=10+2X+3X^2\). The quadratic term makes the relationship nonlinear. Suppose, however, that the planned analysis deliberately fits only a straight line with an intercept.
Worked Example: The Linear Working Target Under a Quadratic Relationship
First calculate the population linear-projection slope, the coefficient targeted by ordinary least squares with an intercept when the working line is fit to the full population. For any \(X\) with nonzero variance, this slope is \(\gamma^*=\operatorname{Cov}(X,Y)/\operatorname{Var}(X)\).
The five outcome values are \(18,11,10,15,26\), respectively. Thus \(E(X)=0\), \(E(Y)=(18+11+10+15+26)/5=16\), and \(E(X^2)=(4+1+0+1+4)/5=2\). Also, \(E(XY)=(-36-11+0+15+52)/5=4\). Therefore,
The intercept is \(E(Y)-\gamma^*E(X)=16\). The best-fitting population line in the least-squares sense is therefore \(16+2X\). It does not reproduce the true curve: its fitted values at \(X=-2,-1,0,1,2\) are \(12,14,16,18,20\), while the corresponding outcomes are \(18,11,10,15,26\). The residuals are \(6,-3,-6,-3,6\), showing a systematic pattern.
The slope \(2\) is a valid target for the stated linear analysis, but it is not the coefficient of a true linear conditional mean. In this symmetric example it happens to equal the coefficient of \(X\) in the generating formula. That coincidence does not generally hold, and it does not make the straight line a good description of the conditional relationship.
The calculation used above is a special case of a general result. The population least-squares line with an intercept is the line \(a+\gamma X\) that minimizes \(E[(Y-a-\gamma X)^2]\). For a fixed \(\gamma\), differentiating with respect to \(a\) gives the minimizing intercept \(a=E(Y)-\gamma E(X)\). Substituting this intercept into the objective and differentiating with respect to \(\gamma\) gives the normal equation \(\operatorname{Cov}(X,Y)-\gamma\operatorname{Var}(X)=0\). When \(\operatorname{Var}(X)>0\), the solution is \(\gamma^*=\operatorname{Cov}(X,Y)/\operatorname{Var}(X)\). This derivation defines the projection target even when \(E(Y\mid X)\) is nonlinear.
For a general nonlinear conditional mean \(m(X)=E(Y\mid X)\), the same target can be written \(\gamma^*=\operatorname{Cov}(X,m(X))/\operatorname{Var}(X)\). The line is a summary defined by the distribution of \(X\). Change that distribution, and the projection coefficient can change even if the conditional mean function \(m\) stays the same. A working coefficient therefore needs interpretation in the population and analysis context that define it.
What Compatibility Can Guarantee
Compatibility is useful because it makes the relationship between imputation and analysis explicit. In a simple normal working model, for example, let \(X\sim N(0,1)\) and \(Y\mid X\sim N(\alpha+\gamma X,\sigma^2)\). These assumptions define a coherent joint distribution. The conditional distribution needed to impute \(X\) can be derived from that joint distribution.
Worked Example: Deriving a Compatible Conditional Imputation Model
Use the hypothetical working model \(X\sim N(0,1)\) and \(Y\mid X\sim N(16+2X,25)\). The joint density is proportional to \(\exp\{-x^2/2-(y-16-2x)^2/50\}\). To obtain \(X\mid Y=y\), collect the terms involving \(x\) in the exponent and complete the square.
It follows that \(X\mid Y=y\) is normal with mean \(2(y-16)/29\) and variance \(25/29\). At the hypothetical value \(y=20\), the conditional mean is \(8/29\approx0.2759\), and the conditional variance is \(25/29\approx0.8621\). A proper draw from this conditional distribution includes variation around its mean; replacing every missing \(X\) by \(0.2759\) would not represent that uncertainty.
These conditionals are compatible by construction: they were both derived from the same joint normal model. But they do not describe the quadratic population in the previous example. In that population, the relationship between \(Y\) and \(X\) is nonlinear and \(X\) is discrete, whereas the working model here assumes a normal \(X\) and a linear conditional mean for \(Y\). Compatibility establishes internal coherence, not correctness for that population.
This distinction matters when \(X\) is incomplete. An imputation model that is compatible with the stated analysis can help preserve the analysis target under the working model. If the actual data depart from that model, compatibility alone cannot guarantee unbiased estimation of a different scientific quantity, accurate uncertainty, or a realistic representation of the missing values. A deliberately misspecified analysis may still be useful when its projection target is intentional, but that target should be named and its limitations reported.
Reviewing a Deliberately Misspecified Analysis
A useful review separates two questions. First, what parameter does the chosen analysis estimate under the distribution of interest? Second, is the imputation procedure adequate for estimating that parameter from incomplete data? Neither question can be answered just by noting that the imputation and analysis formulas look similar.
Decide whether the coefficient is intended as a linear projection, a coefficient under a substantive linear model, or something else. Do not describe a projection coefficient as the true effect of \(X\) unless the scientific and statistical assumptions justify that interpretation.
Identify the feature omitted from the analysis formula, such as curvature. Explain why the simpler analysis is being used and which interpretation remains defensible.
Review whether the imputation procedure preserves the variables and relationships needed by the stated analysis and whether its assumptions are credible given the observed data and missingness information.
Inspect whether the working analysis leaves systematic patterns, and compare the target estimate under plausible imputation and analysis specifications. A stable coefficient is informative but does not prove that the working model is true.
In the quadratic example, the residuals are positive at both extremes and negative near the center. That pattern is evidence that the straight-line conditional mean is inadequate as a description of the outcome. If the coefficient is retained as a projection summary, the pattern should be disclosed; if the scientific question concerns curvature, a linear working coefficient may not answer it.
Inference also requires care. The usual model-based standard error from a linear regression may rely on assumptions that are not met under deliberate misspecification. Depending on the target and sampling design, a robust variance estimator may be appropriate for the complete-data analysis. Multiple imputation adds uncertainty about missing values, but pooling estimates does not repair an unsuitable variance estimator or redefine the target. The analyst must consider both sources of inferential assumptions.
Comparing the Main Choices
| Choice | What it aims to do | Main caution |
|---|---|---|
| Use the simple analysis and an aligned imputation model | Estimate the parameter defined by the stated working formula, such as a linear projection coefficient. | Compatibility does not establish that the working relationship is scientifically adequate. |
| Use a richer analysis model | Represent features such as curvature when they are important to the scientific question. | The richer model still needs defensible assumptions and enough information to estimate its terms. |
| Use an imputation model richer than the analysis model | Allow imputation to represent plausible structure while retaining the planned analysis target. | Richer terms must be specified and drawn appropriately; adding terms mechanically does not ensure valid inference. |
| Change only the imputation model | Assess whether estimates depend on imputation assumptions while holding the analysis target fixed. | It does not answer a scientific question that the analysis formula itself omits. |
These choices are alternatives, not automatic upgrades from simple to complex. The right choice depends on the estimand and purpose. If a linear coefficient is deliberately used as a concise population summary, it can be reported as such. If the aim is to describe the nonlinear conditional relationship, retaining the linear formula and improving only the imputation model cannot supply the missing scientific interpretation.
Pooling Does Not Remove Analysis Misspecification
For \(M\) completed data sets, Rubin's rules combine the \(M\) coefficient estimates and their estimated variances. Let \(Q_m\) be the estimate and \(U_m\) its estimated variance in completed data set \(m\). The pooled estimate is \(\bar{Q}=M^{-1}\sum_{m=1}^M Q_m\); the within-imputation variance is \(\bar{U}=M^{-1}\sum_{m=1}^M U_m\); and the between-imputation variance is \(B=(M-1)^{-1}\sum_{m=1}^M(Q_m-\bar{Q})^2\). The total variance estimate is \(\bar{U}+(1+1/M)B\), under the usual conditions for multiple-imputation inference.
Worked Example: Interpreting Pooled Estimates for a Working Coefficient
Suppose five hypothetical imputations yield linear-projection estimates \(1.8, 2.1, 1.9, 2.2,\) and \(2.0\), each with estimated variance \(0.25\). The pooled estimate is \(\bar{Q}=(1.8+2.1+1.9+2.2+2.0)/5=2.0\), and \(\bar{U}=0.25\).
The squared deviations from 2.0 sum to \(0.04+0.01+0.01+0.04+0=0.10\), so \(B=0.10/(5-1)=0.025\). The estimated total variance and standard error are
Using a normal approximation for illustration, an interval is \(2.0\pm1.96(0.5292)\), or approximately \([0.963,3.037]\). This summarizes uncertainty for the coefficient defined by the stated linear analysis, subject to the imputation and variance assumptions. It is not an interval for the entire nonlinear relationship, nor does it establish that the true conditional mean is linear. In a real analysis, the degrees of freedom and reference distribution should be chosen using the appropriate multiple-imputation procedure; five imputations here are for transparent arithmetic, not a recommendation for routine practice.
Common Mistakes and Failure Modes
- Equating compatibility with correctness. “The conditional models can come from one joint distribution” does not imply “that distribution generated the data.” State compatibility and adequacy as separate judgments.
- Calling a working coefficient the true effect. Under a nonlinear relationship, the linear coefficient is generally a distribution-dependent projection. Describe it using the intended target and units.
- Assuming pooling repairs a misspecified formula. Rubin's rules account for uncertainty across imputations under their conditions. They do not add omitted curvature to the analysis or correct an incorrect scientific interpretation.
- Using a single conditional mean as an imputation. Substituting a mean for each missing value suppresses variation and does not represent proper imputation draws. A conditional model should reflect uncertainty as well as its expected value.
- Treating a clean diagnostic as proof. A lack of an obvious residual pattern or sensitivity to a few specifications cannot establish that the working model is true, especially where data are missing.
Check Your Understanding
Answer each question, then compare with the solutions below.
- In the quadratic example, why is the slope \(2\) a valid target even though the conditional mean is not linear?
- What does compatibility establish, and what does it not establish?
- For the working model in the conditional-imputation example, state the mean and variance of \(X\mid Y=16\).
- If a pooled linear coefficient is stable across several imputation specifications, what conclusion is justified and what conclusion is not?
Solutions. (1) It is the population least-squares projection slope, \(\operatorname{Cov}(X,Y)/\operatorname{Var}(X)=4/2=2\), which is defined even when \(E(Y\mid X)\) is nonlinear. (2) Compatibility establishes that the specified conditionals can arise from a shared joint distribution; it does not establish that the joint distribution is the actual data-generating distribution or that the analysis target is scientifically adequate. (3) Substituting \(y=16\) gives mean \(2(16-16)/29=0\) and variance \(25/29\approx0.8621\). (4) Stability supports the limited statement that the coefficient is not highly sensitive to those particular imputation choices. It does not prove that the linear model is true, that omitted curvature is irrelevant, or that the coefficient answers a different scientific question.