Deriving Gaussian Conditionals From a Bivariate Normal

Why Derive the Gaussian Conditionals?

In fully conditional specification, an update for one variable can be expressed as a distribution given the other variables. A bivariate normal model provides a clear setting in which to see exactly how a conditional mean and variance follow from a joint model. The calculation is useful in its own right and gives a reference point for interpreting Gaussian conditional models.

The result depends on a joint-normality assumption, not merely on knowing means, variances, and a correlation. Under joint normality, each conditional distribution is normal, its mean is linear in the conditioning value, and its variance does not depend on that value.

Here \(N(\mu,\sigma^2)\) uses its second argument for the variance. The two formulas are parallel, but their slopes are generally different: the slope for \(X\) conditional on \(Y\) is \(\rho\sigma_X/\sigma_Y\), whereas the slope for \(Y\) conditional on \(X\) is \(\rho\sigma_Y/\sigma_X\). Their units differ as well. The conditional variances are constant with respect to the observed conditioning value, but they need not equal the corresponding marginal variances.

Derivation by Completing the Square

For a nondegenerate bivariate normal, the joint density is

To find the conditional density of \(X\) given \(Y=y\), treat \(y\) as fixed and retain the terms that vary with \(x\). Write the quadratic expression in the exponent as

Expanding the square in the second term recovers the original expression: its cross-product is \(-2\rho(x-\mu_X)(y-\mu_Y)/(\sigma_X\sigma_Y(1-\rho^2))\), and combining its \(y\)-only term with \((y-\mu_Y)^2/\sigma_Y^2\) gives \((y-\mu_Y)^2/(\sigma_Y^2(1-\rho^2))\). Thus the parts that depend on \(x\) in the conditional density are proportional to

This is the kernel of a normal density in \(x\), with the stated conditional mean and variance. The factor needed to normalize the density is determined by those parameters, so this identifies the full conditional distribution, not just its shape.

The result for \(Y\mid X=x\) follows by the same argument with \(x\) and \(y\) interchanged. In particular, the squared term becomes \(\left[(y-\mu_Y)-\rho\frac{\sigma_Y}{\sigma_X}(x-\mu_X)\right]^2/(\sigma_Y^2(1-\rho^2))\), giving the stated mean and variance for \(Y\mid X=x\).

Reading the Formulas

The conditional mean for \(X\) is centered at \(\mu_X\): when \(y=\mu_Y\), its value is exactly \(\mu_X\). If \(\rho>0\), conditioning on a value of \(Y\) above its mean shifts the conditional mean of \(X\) above \(\mu_X\); if \(\rho<0\), the shift is in the opposite direction. The direction reverses in the corresponding way for \(Y\mid X=x\).

The conditional variance can also be written using the covariance \(c=\operatorname{Cov}(X,Y)\):

For example, the first identity follows from \(c=\rho\sigma_X\sigma_Y\): \(\sigma_X^2-c^2/\sigma_Y^2=\sigma_X^2-\rho^2\sigma_X^2=\sigma_X^2(1-\rho^2)\). These variances are positive under the nondegeneracy condition \(|\rho|<1\). Conditioning reduces variance in this bivariate setting, and the reduction is greater when the absolute correlation is larger.

Checks, Failure Modes, and Alternatives

A useful way to verify a calculation is to inspect its scale and limiting cases. The conditional mean of \(X\) must have units of \(X\): the ratio \(\sigma_X/\sigma_Y\) converts a deviation in \(Y\) into units of \(X\). The variance must have squared units of \(X\). Setting \(y=\mu_Y\) should return \(\mu_X\), and setting \(\rho=0\) should give the marginal distribution of \(X\), independent of \(y\). Apply the corresponding checks to \(Y\mid X=x\).

The density derivation above is not the only route. For a jointly normal vector, the conditional-mean slope can be written as covariance divided by the variance of the conditioning variable, and the conditional variance as the marginal variance minus covariance squared divided by that same variance. These are exactly the covariance-matrix formulas used in the second example. This route is often quicker when the covariance matrix is given; completing the square makes clear why the conditional distribution is normal. Without joint normality, the covariance formulas alone do not generally establish a normal conditional distribution or a linear conditional mean.

In a MICE update, these formulas describe a Gaussian conditional model when the variables are jointly bivariate normal. They do not by themselves show that two separately specified conditional models arise from a common joint distribution. That is a distinct question. Here, the assumptions specify one joint normal model, from which both conditionals follow.

• Swapping the standard-deviation ratio. For \(X\mid Y=y\), use \(\sigma_X/\sigma_Y\), not \(\sigma_Y/\sigma_X\). The conditional mean must be measured in units of \(X\).
• Using the wrong conditional variance. For \(X\mid Y\), use \(\sigma_X^2(1-\rho^2)\), or \(\sigma_X^2-c^2/\sigma_Y^2\). The result must have squared units of \(X\).
• Assuming a normal conditional from a correlation alone. The derivation requires joint normality. A correlation and marginal moments, without that distributional assumption, are not enough.
• Ignoring degeneracy. If \(|\rho|=1\), the covariance matrix is singular and the displayed nondegenerate density does not apply; the conditional distribution collapses to a deterministic linear relation rather than having positive variance.

A practical limitation is that the formulas are exact only for the stated joint-normal model. Real variables may be skewed, bounded, or related nonlinearly, so the model’s conditional shape and constant variance may be inadequate. A calculation should therefore be interpreted as a consequence of the assumed model, not as proof that the model describes a particular data set.