A Gaussian Example of Compatible Conditionals

Why a Gaussian Example Makes Compatibility Concrete

When two variables are imputed from one another, each conditional model may look reasonable on its own. Compatibility asks a sharper question: could both conditionals be the conditionals of the same joint distribution? For jointly normal variables, the answer can be checked directly from their regression slopes and conditional variances.

This example is useful because it makes the constraints explicit. A Gaussian conditional model is not compatible merely because its mean is linear and its variance is positive. The two directions must also agree with one another. The calculations below concern exact population distributions; in applications, estimated coefficients and variances will have sampling uncertainty.

The conditional mean of \(Y\) given \(X\) has slope \(\rho\sigma_Y/\sigma_X\); the reverse conditional mean has slope \(\rho\sigma_X/\sigma_Y\). Both slopes have the sign of the correlation, and their product is \(\rho^2\). The conditional variances are constant: they do not depend on the value being conditioned on. These facts follow from the joint normal density, rather than being separate choices for the two regressions.

Deriving the Conditional Distributions

Write \(u=x-\mu_X\) and \(v=y-\mu_Y\). The exponent in the bivariate normal density is proportional to \((u^2/\sigma_X^2-2\rho uv/(\sigma_X\sigma_Y)+v^2/\sigma_Y^2)/(2(1-\rho^2))\). To obtain the conditional distribution of \(Y\) at fixed \(x\), complete the square in \(v\):

At fixed \(x\), the second term does not vary with \(y\), so it is part of the normalizing constant of the conditional density. The first term is the squared deviation from \(v/\sigma_Y=\rho u/\sigma_X\), scaled by \(1-\rho^2\). Consequently, \(Y\mid X=x\) has mean \(\mu_Y+\rho(\sigma_Y/\sigma_X)(x-\mu_X)\) and variance \(\sigma_Y^2(1-\rho^2)\). Completing the square in \(u\), instead, gives the reverse conditional stated in the definition.

The nondegeneracy assumption matters: \(|\rho|<1\) makes both conditional variances strictly positive. If \(|\rho|=1\), one variable is an exact linear function of the other and the bivariate distribution is degenerate; the positive-variance conditional formulas do not apply.

Compatibility Constraints for Two Gaussian Conditionals

Now start with two proposed conditional models, rather than a specified joint distribution: \(Y\mid X=x\sim N(a_Y+b_{YX}x,v_Y)\) and \(X\mid Y=y\sim N(a_X+b_{XY}y,v_X)\). The \(a\)'s are intercepts, the \(b\)'s are slopes, and \(v_Y,v_X\) are conditional variances. Assume \(v_Y>0\) and \(v_X>0\). The slopes have reciprocal measurement units: \(b_{YX}\) measures units of \(Y\) per unit of \(X\), whereas \(b_{XY}\) measures units of \(X\) per unit of \(Y\).

The slope-product constraint comes from the formulas for a joint bivariate normal: \(b_{YX}b_{XY}=\rho^2\), which must be less than 1. If the product is positive and both slopes are nonzero, they must have the same sign. For the variance constraint, the conditional formulas give \(v_Y=\sigma_Y^2(1-\rho^2)\) and \(v_X=\sigma_X^2(1-\rho^2)\). Therefore, when the slopes are nonzero, \(v_Y/v_X=\sigma_Y^2/\sigma_X^2=b_{YX}/b_{XY}\).

The zero-slope case must be handled separately. In a bivariate normal distribution, a zero correlation makes both slopes zero, but the marginal variances need not be equal. The conditionals are then independent normals with variances \(v_Y\) and \(v_X\); their variance ratio can be any positive number. Dividing by either slope in this case would be undefined and would incorrectly rule out valid independent distributions.

Why the Constraints Are Sufficient

The constraints do more than provide necessary checks. They also let us construct a joint bivariate normal distribution when they hold. For nonzero slopes, put \(q=b_{YX}b_{XY}\), so \(0<q<1\), and set \(\sigma_Y^2=v_Y/(1-q)\) and \(\sigma_X^2=v_X/(1-q)\). The variance-ratio condition ensures \(b_{YX}\sigma_X^2=b_{XY}\sigma_Y^2\), so either expression defines the same covariance. The resulting correlation has square \(q\), with the shared sign of the slopes. This gives the proposed slopes and conditional variances.

The intercepts determine means by solving \(\mu_Y=a_Y+b_{YX}\mu_X\) and \(\mu_X=a_X+b_{XY}\mu_Y\). Since \(1-q>0\), the solution is unique:

The constructed covariance matrix is positive definite: its determinant is \(\sigma_X^2\sigma_Y^2(1-q)>0\), and its diagonal entries are positive. Thus it defines a nondegenerate bivariate normal with precisely the proposed conditionals. If both slopes are zero, take independent normals with means \(a_X,a_Y\) and variances \(v_X,v_Y\); this proves sufficiency in that case as well.

Using the Check and Recognizing Its Limits

In a MICE setting, these calculations are a compatibility check for an idealized pair of Gaussian conditional models. They do not establish that a fitted imputation procedure has converged, that the Gaussian assumptions describe the data well, or that the model is appropriate for the eventual analysis. The conditional distributions here assume continuous variables, joint normality, linear conditional means, and constant conditional variances.

A practical check should keep the following distinctions clear:

• Check the model form first. These identities concern exact Gaussian conditionals. Curvature, changing residual spread, or strong departures from normality can make the model a poor description even if its estimated coefficients approximately satisfy the identities.
• Use estimates cautiously. Estimated slopes and variances will not generally satisfy exact equalities in finite samples. A small discrepancy is not, by itself, proof of incompatibility in the underlying population; consider estimation uncertainty and model fit.
• Check units and direction. The two slopes have reciprocal units, while their product is dimensionless. The variance ratio and slope ratio have matching units. Swapping a direction or comparing a variance to a slope without regard to units can conceal an error.
• Do not infer algorithmic behavior from compatibility alone. A shared joint distribution exists when the stated constraints hold, but this fact alone is not a convergence diagnostic or a guarantee of good imputations.

A direct joint Gaussian model is an alternative when the multivariate normal assumptions are suitable: it specifies one covariance structure and derives both conditionals from it. Fully conditional specification instead writes separate models for each variable, which can offer more flexibility but does not automatically enforce these Gaussian constraints. Neither approach removes the need to assess whether the distributional assumptions and substantive relationships are credible.