Compatibility Constraints for Gaussian Conditional Means

Why Conditional Models Need to Agree

In fully conditional specification, each variable can be assigned a model conditional on the other variables. For two continuous variables, a common choice is a linear regression with a Gaussian residual distribution in each direction. But two individually plausible models need not be the conditional distributions of the same joint distribution. Their parameters must agree in a precise way.

Here we focus on a pair of variables \(X\) and \(Y\). The earlier material on deriving Gaussian conditionals from a bivariate normal shows how a joint normal model produces linear conditional means. The question here is the reverse one: given both conditional models, which parameter values allow a proper joint bivariate normal distribution?

Assume the conditional models have constant, positive variances and take the form

The first slope \(b_{XY}\) describes the change in the conditional mean of \(X\) for a one-unit increase in \(Y\); the second slope \(b_{YX}\) describes the change in the conditional mean of \(Y\) for a one-unit increase in \(X\). The intercepts are \(a_X\) and \(a_Y\), and the conditional variances are \(\sigma^2_{X\mid Y}\) and \(\sigma^2_{Y\mid X}\). These variances are assumed strictly greater than zero.

The Compatibility Restrictions

For compatibility, the parameters must satisfy two conditions. First, the slopes and variances must obey a cross-equation equality. Second, the resulting joint distribution must be proper, which imposes a strict bound on the product of the slopes.

The first equality can also be written as \(b_{XY}\sigma^2_{Y\mid X}=b_{YX}\sigma^2_{X\mid Y}\). It is the slopes divided by their respective conditional variances—not the slopes alone—that must match. The second condition rules out an implied joint distribution with a zero or negative variance in some direction. Given these conditions, there is no additional restriction on the two intercepts.

Deriving the Conditions

Suppose first that a positive joint density exists. The logarithm of that density can be written as the logarithm of \(f(x\mid y)\) plus a term depending only on \(y\). Expanding the part of the conditional log-density that involves both \(x\) and \(y\), the coefficient of \(xy\) is \(b_{XY}/\sigma^2_{X\mid Y}\). Writing the same joint log-density using \(f(y\mid x)\) gives the coefficient \(b_{YX}/\sigma^2_{Y\mid X}\). These coefficients must be equal, giving the first restriction. This argument is valid because the Gaussian densities are positive and their log-densities are smooth.

The joint precision matrix—the inverse of the joint covariance matrix—implied by the conditional quadratic terms is

The off-diagonal entries agree with the second conditional model precisely when the slope-to-variance equality holds. A proper bivariate normal requires \(Q\) to be positive definite: its first diagonal entry must be positive, and its determinant must be positive. The first requirement follows from \(\sigma^2_{X\mid Y}>0\). Using the slope equality, the determinant is

The denominator is positive, so \(\det(Q)>0\) exactly when \(b_{XY}b_{YX}<1\). This proves why equality of the slope-to-variance ratios is not sufficient on its own: the resulting precision matrix must also be positive definite.

The intercepts determine the joint mean. Writing the conditional means as equations for that mean gives

When \(b_{XY}b_{YX}<1\), the matrix is invertible. Thus every pair of finite intercepts determines a finite joint mean; no further intercept constraint is needed. The joint covariance is \(Q^{-1}\), which is finite and positive definite under the same strict product condition.

Examples of Failed Compatibility

Interpreting the Restrictions and Checking Models

When the conditionals are compatible, their slope product has an additional interpretation. For a bivariate normal distribution with nonzero marginal variances, the two conditional slopes are \(b_{XY}=\operatorname{Cov}(X,Y)/\operatorname{Var}(Y)\) and \(b_{YX}=\operatorname{Cov}(X,Y)/\operatorname{Var}(X)\). Their product is therefore the squared correlation, so it lies between 0 and 1. This explains why compatible nonzero slopes have the same sign and why their product must be strictly less than 1 for a nondegenerate joint distribution.

This identity is a consequence of compatibility; it should not be used as a replacement for checking both restrictions. Slopes with product less than 1 can still fail the slope-to-variance equality, as the second worked example showed. Conversely, matching ratios do not prevent the product from exceeding 1, as the third example showed.

For a pair of fitted conditional models, a practical check is to record both conditional variances and both slopes, calculate the two ratios, and then calculate the slope product. Interpret these checks in light of how the models were specified and estimated. For estimated parameters, small numerical discrepancies may reflect rounding or estimation uncertainty; a substantial mismatch indicates that the fitted conditionals do not exactly represent a common bivariate normal model. The algebraic restrictions describe exact compatibility, not a statistical test for compatibility.

Common Pitfalls and Limitations

• Checking only whether the slopes are similar. The condition compares each slope with its own conditional variance. Equal slopes need not satisfy the required equality when the variances differ.
• Checking only the product. A product below 1 does not establish compatibility unless the slope-to-variance ratios also match.
• Treating equality at the boundary as acceptable. If \(b_{XY}b_{YX}=1\), the implied precision determinant is zero. The resulting covariance is singular, so there is no proper, nondegenerate bivariate normal distribution.
• Assuming compatibility means the model is correct. Compatibility concerns whether the two conditional distributions can belong to one joint distribution. It does not establish that either conditional form, its constant variance assumption, or its Gaussian errors are appropriate for a particular data set.
• Applying the test outside its assumptions. Nonlinear conditional means, non-Gaussian distributions, or variances that depend on the conditioning variable require different compatibility arguments. The restrictions derived here are specific to the stated linear Gaussian pair.

In MICE, these restrictions clarify what it means for two Gaussian update models to be consistent with a shared joint model. They also explain why specifying each regression separately does not automatically guarantee a common joint distribution. The comparison with an approach that starts from a joint distribution is direct: a joint model determines its conditionals, while separately specified conditionals must pass the compatibility checks to correspond to that same kind of joint model. Compatibility is a structural property, not a convergence diagnostic for an imputation chain.