When Fully Conditional Models Have No Joint Distribution

Why Plausible Regressions Can Conflict

In fully conditional specification, each incomplete variable is imputed from a model conditional on the other variables. Each regression can be familiar and reasonable on its own. But that does not guarantee that all the regressions can be the conditional distributions of one common joint distribution. Compatibility is a global requirement: the conditional models must agree with one another as parts of the same probability model.

This distinction matters in MICE because fitting or specifying the conditional models one at a time does not, by itself, impose the restrictions that a shared joint distribution would require. The earlier material on compatibility of conditional imputation models defines the general issue; here we make the conflict concrete for a pair of Gaussian regressions.

Definition: Two conditional models are compatible if there exists a joint probability distribution whose full conditional distributions are exactly those models. They are incompatible if no such joint distribution exists. A model may be well-defined as a conditional distribution on its own and still be incompatible with another conditional model.

Let \(X\) and \(Y\) be continuous variables. Consider the proposed full conditionals

X\mid Y=y\sim N(\alpha_0+a y,u), \qquad Y\mid X=x\sim N(\beta_0+b x,v),

where \(u>0\) and \(v>0\) are constant conditional variances. The parameters \(a\) and \(b\) are the two regression slopes; \(u\) is the residual variance for predicting \(X\) from \(Y\), and \(v\) is the residual variance for predicting \(Y\) from \(X\). Each conditional is a valid normal model for any finite intercept and slope and any positive residual variance. The question is whether both can hold simultaneously under one joint distribution.

Restrictions Imposed by a Shared Joint Distribution

For the result below, suppose the proposed conditionals describe the same population, both variables have finite and positive marginal variances, and the conditional means are linear with the stated constant positive conditional variances. These conditions hold for the nondegenerate Gaussian setting considered here.

Theorem: The two Gaussian conditional models above are compatible if and only if \(a v=b u\) and \(ab<1\). When these conditions hold, the compatible joint distribution is bivariate normal. The intercepts may be arbitrary finite values; they determine the joint means but do not remove a conflict in slopes or variances.

The restrictions have distinct roles. The equality \(av=bu\) requires the slopes and residual variances to agree in the way implied by a common covariance structure. The inequality \(ab<1\) is a stability condition: even when the equality holds, slopes that are too large cannot be supported by finite positive marginal variances.

Derivation of the Compatibility Conditions

Suppose first that a joint distribution does exist. Write \(V_X=\operatorname{Var}(X)\), \(V_Y=\operatorname{Var}(Y)\), and \(C=\operatorname{Cov}(X,Y)\). Taking expectations of the first conditional mean gives \(\operatorname{E}(X)=\alpha_0+a\operatorname{E}(Y)\); subtracting this mean relation from the conditional mean shows that \(C=aV_Y\). Applying the same argument to the second conditional gives \(C=bV_X\). Thus the slopes must equal the covariance divided by the variance of the conditioning variable.

Because each conditional variance is constant, the law of total variance gives \(u=V_X-C^2/V_Y\) and \(v=V_Y-C^2/V_X\). Define the squared correlation \(\rho^2=C^2/(V_XV_Y)\). Then

u=V_X(1-\rho^2),\qquad v=V_Y(1-\rho^2),\qquad a=\frac{C}{V_Y},\qquad b=\frac{C}{V_X}.

Since the joint distribution is nondegenerate, \(0\leq \rho^2<1\). These expressions imply

av=\frac{C}{V_Y}V_Y(1-\rho^2) =C(1-\rho^2) =\frac{C}{V_X}V_X(1-\rho^2) =bu, \qquad ab=\frac{C^2}{V_XV_Y}=\rho^2<1.

This proves necessity. For sufficiency, suppose \(av=bu\) and \(ab<1\). Define means \(m_X,m_Y\) by solving the two linear equations \(m_X=\alpha_0+a m_Y\) and \(m_Y=\beta_0+b m_X\). Since \(1-ab>0\), a unique solution exists. Now set

V_X=\frac{u}{1-ab},\qquad V_Y=\frac{v}{1-ab},\qquad C=\frac{av}{1-ab}=\frac{bu}{1-ab}.

The equality in the expression for \(C\) follows from \(av=bu\). These variances are positive, and the covariance matrix is positive definite: its determinant is

V_XV_Y-C^2 =\frac{uv-a^2v^2}{(1-ab)^2} =\frac{uv}{1-ab}>0.

Here \(a^2v=abu\), which follows from \(av=bu\). A bivariate normal distribution with means \(m_X,m_Y\) and this covariance matrix has conditional slope \(C/V_Y=a\), conditional variance \(V_X-C^2/V_Y=u\), and the corresponding second conditional slope \(C/V_X=b\) and variance \(V_Y-C^2/V_X=v\). Its conditional intercepts are therefore \(\alpha_0\) and \(\beta_0\), by the equations defining the means. This constructs the required joint distribution and completes the justification.

Worked Compatibility Checks

Worked Example: A Residual-Variance Conflict

Suppose, hypothetically, that two standardized laboratory measures are assigned the conditional models \(X\mid Y=y\sim N(0.2+0.5y,1)\) and \(Y\mid X=x\sim N(-0.1+0.5x,2)\). Both are individually valid normal regressions: their residual variances are positive. Check whether they can be full conditionals of one joint distribution.

Here \(a=0.5\), \(b=0.5\), \(u=1\), and \(v=2\). The slope-variance condition requires

\(av=bu\), but \(av=(0.5)(2)=1.000\), whereas \(bu=(0.5)(1)=0.500\).

The values disagree, so no joint distribution can have both of these exact conditional models. The intercepts do not affect this conclusion. The problem is not that either regression is invalid by itself; the two residual-variance and slope specifications cannot both describe the same joint distribution.

Worked Example: Opposing Regression Directions

In another hypothetical specification, let \(X\mid Y=y\sim N(0.4y,1)\) and \(Y\mid X=x\sim N(-0.4x,1)\). Each conditional is mathematically valid, but the slopes point in opposite directions. Test compatibility.

The parameters are \(a=0.4\), \(b=-0.4\), and \(u=v=1\). The required equality gives \(av=(0.4)(1)=0.4\), but \(bu=(-0.4)(1)=-0.4\). Since \(0.4\ne-0.4\), the condition fails. In a common joint distribution, both slopes would be determined by the same covariance \(C\): \(a=C/V_Y\) and \(b=C/V_X\). Positive variances cannot make those slopes have opposite signs. Thus these two regressions cannot be the full conditionals of one joint distribution.

Worked Example: Slopes Too Large for Finite Variances

Consider hypothetical models \(X\mid Y=y\sim N(2y,2)\) and \(Y\mid X=x\sim N(x,1)\). This time the slope-variance equality holds: \(av=(2)(1)=2\) and \(bu=(1)(2)=2\). The second condition still needs checking: \(ab=(2)(1)=2\), which is not less than 1.

The impossibility can also be seen directly from total variance. A joint distribution with these conditionals would have to satisfy \(V_X=2+4V_Y\) and \(V_Y=1+V_X\). Substituting the second equation into the first gives

\(V_X=2+4(1+V_X)=6+4V_X\), so \(-3V_X=6\) and \(V_X=-2\).

A variance cannot be negative. Therefore the conditionals are incompatible, even though their slope-variance equality is satisfied and each regression is valid individually. This example illustrates why checking only \(av=bu\) is not enough.

What the Check Does—and Does Not—Tell You

For two Gaussian conditionals of this form, the algebra provides a direct compatibility check. A violation of either condition rules out a shared joint distribution. Passing both checks establishes compatibility under the stated model assumptions. For a fitted MICE specification, the calculation can also be applied to the stated parameter values or estimates. If an estimated set fails the check, it identifies a conflict in those fitted conditionals; it does not by itself establish that the underlying population models are incompatible, since estimates vary from sample to sample.

This result is specific to the stated pair of linear Gaussian conditionals with constant positive variances. Conditional models involving categorical variables, nonlinear means, interactions, or other distributions may require different compatibility arguments. There is no general rule that a quick pairwise slope check settles compatibility for an arbitrary collection of MICE models.

A practical diagnostic is to write down, for each pair of continuous variables, the two conditional slopes and residual variances and compare \(av\) with \(bu\), then check \(ab<1\). Also verify that the two regressions refer to the same variables, scales, population, and conditioning set. If they do not, the formulas above are not the appropriate comparison. For more complex models, a warning sign is that separate regressions impose conditional relationships that cannot be reconciled by a common covariance or probability structure; numerical convergence of the imputation algorithm does not prove compatibility.

Compatibility is also not the same as goodness of fit. Two regressions can satisfy the algebraic restrictions but describe the data poorly. Conversely, each regression can fit its own observed conditional pattern reasonably well while the pair fails the compatibility check. Fit diagnostics and compatibility address different questions.

Approach	What it specifies	Key limitation or strength
Separate fully conditional models	A model for each variable conditional on the others	Can be flexible, but compatibility is not guaranteed by fitting each model separately
Joint modeling	One joint distribution for the variables	Its conditionals are compatible by construction, but choosing a suitable joint distribution can be restrictive
Compatibility check	Whether stated conditionals can arise from one joint distribution	Can expose contradictions; does not establish good fit or assess every feature of an imputation procedure

If a specification fails the check, one option is to revise the conditional models so their parameters obey compatible restrictions. Another is to formulate a joint model first and derive its conditionals. These approaches involve a trade-off: separately specified conditionals can accommodate different variable types and relationships, while a single joint model makes shared structure explicit but may be harder to specify well. In either case, the substantive model and the imputation goal still need careful attention.

Common Mistake

Wrong reasoning: “Both regressions have normal errors and positive residual variances, so together they define a valid bivariate normal model.” Normality and positive residual variances establish that each conditional is valid separately; they do not establish that the pair is compatible.

Correction: Check the restrictions that a shared distribution imposes. For the Gaussian pair here, calculate \(av\), \(bu\), and \(ab\). Compatibility requires \(av=bu\) and \(ab<1\), as well as the model assumptions stated above. A chain that alternates between valid conditional regressions may still fail this test; running the updates is not a proof that a joint distribution has the specified conditionals.

Key takeaway: Individually plausible conditional regressions need not form a coherent set of full conditionals. In the two-variable Gaussian case, a common joint distribution requires matched slope-variance products and a slope product below one.

Check Your Understanding

For each question, use the Gaussian compatibility conditions derived above. Solutions are included so you can check each step.

Suppose \(a=0.3\), \(b=0.6\), \(u=2\), and \(v=1\). Are the two conditionals compatible?
Suppose \(a=0.8\), \(b=0.5\), \(u=4\), and \(v=2\). Check both conditions.
Why can the intercepts not repair a failure of \(av=bu\)?
Does numerical convergence of a chained-update algorithm establish that its specified conditional models are compatible?

Solutions. (1) \(av=(0.3)(1)=0.3\), while \(bu=(0.6)(2)=1.2\); the equality fails, so the models are incompatible. (2) \(av=(0.8)(2)=1.6\) and \(bu=(0.5)(4)=2.0\); the equality fails. Also \(ab=(0.8)(0.5)=0.4<1\), but both conditions are required, so these models are incompatible. (3) Intercepts set the means through \(m_X=\alpha_0+a m_Y\) and \(m_Y=\beta_0+b m_X\); the slope-variance equality follows from the covariance and conditional variances and is unaffected by adding constants to the means. (4) No. Convergence concerns the behavior of the updating algorithm; it does not show that the specified conditionals are full conditionals of one common joint distribution.