Deriving Conditional Normal Laws by Completing the Square

Why Completing the Square Reveals a Conditional Law

A multivariate Gaussian density couples its coordinates through a quadratic form. To condition on one block, the useful question is: which part of that quadratic form depends on the block being conditioned, and how can it be centered? Completing the square separates the joint exponent into a residual term involving the block of interest and a term involving only the observed block. The first term gives the conditional covariance and the center of the conditional density.

This derivation is for a nonsingular joint Gaussian vector. In particular, the covariance of the block being conditioned on must be invertible, and the resulting Schur complement must be positive definite. Singular conditioning requires additional care and is not covered here.

Definition: Let $X\in\mathbb{R}^p$ and $Y\in\mathbb{R}^q$ be jointly Gaussian, with mean vectors $\mu_X,\mu_Y$ and covariance matrix $$ \Sigma= \begin{pmatrix} A&C\\ C^T&B \end{pmatrix}. $$ Here $A=\operatorname{Cov}(X,X)$, $B=\operatorname{Cov}(Y,Y)$, and $C=\operatorname{Cov}(X,Y)$. Assume $\Sigma$ is positive definite. Then $B$ is positive definite, and the Schur complement of $B$ in $\Sigma$ is $$ S=A-CB^{-1}C^T. $$ It is also positive definite.

Write $u=x-\mu_X$ and $v=y-\mu_Y$. The block $X$ is the quantity whose conditional density we seek; $y$ is the fixed value on which we condition. The key square will involve $u-CB^{-1}v$. Its center, expressed as a value of $x$, is $\mu_X+CB^{-1}(y-\mu_Y)$.

The Square-Completion Identity

The block factorization of the covariance makes the algebra transparent. Define $$ R= \begin{pmatrix} I_p&CB^{-1}\\ 0&I_q \end{pmatrix}, \qquad D= \begin{pmatrix} S&0\\ 0&B \end{pmatrix}. $$ Since $S=A-CB^{-1}C^T$, direct block multiplication gives $$ RDR^T= \begin{pmatrix} S+CB^{-1}C^T&C\\ C^T&B \end{pmatrix} = \begin{pmatrix} A&C\\ C^T&B \end{pmatrix} =\Sigma. $$ The inverse transformation sends $(u,v)^T$ to $(u-CB^{-1}v,v)^T$. This yields the quadratic-form identity below.

Result: For $u\in\mathbb{R}^p$ and $v\in\mathbb{R}^q$, $$ \begin{pmatrix}u\\v\end{pmatrix}^{T} \Sigma^{-1} \begin{pmatrix}u\\v\end{pmatrix} = (u-CB^{-1}v)^TS^{-1}(u-CB^{-1}v) +v^TB^{-1}v. $$

Proof. From $\Sigma=RDR^T$, we have $\Sigma^{-1}=R^{-T}D^{-1}R^{-1}$. Also, $$ R^{-1}\begin{pmatrix}u\\v\end{pmatrix} = \begin{pmatrix}u-CB^{-1}v\\v\end{pmatrix}, \qquad D^{-1}= \begin{pmatrix}S^{-1}&0\\0&B^{-1}\end{pmatrix}. $$ Substituting these expressions into the quadratic form gives the stated sum. The first summand is a square centered at $u=CB^{-1}v$; the second depends only on $v$. $\square$

This identity is the multivariate completion of the square. It does not merely suggest a conditional mean: together with the density normalizer, it gives the complete conditional density.

Worked Example: A Scalar Block With a Small Marginal Variance

Suppose $(X,Y)^T$ is jointly normal with mean $(2,-1)^T$ and covariance $$ \Sigma= \begin{pmatrix} 1&1/6\\ 1/6&1/9 \end{pmatrix}. $$ The determinant is $1/9-1/36=1/12>0$, and the diagonal entries are positive, so this $2\times2$ covariance matrix is positive definite.

Here $A=1$, $C=1/6$, and $B=1/9$. Thus $$ S=1-\frac{(1/6)^2}{1/9} =1-\frac{1/36}{1/9} =1-\frac14 =\frac34. $$ The coefficient in the conditional mean is $CB^{-1}=(1/6)/(1/9)=3/2$. Therefore the square-completion identity gives $$ \frac{4}{3}(x-2)^2-4(x-2)(y+1)+12(y+1)^2 = \frac{\left(x-2-\frac32(y+1)\right)^2}{3/4} +9(y+1)^2. $$ Equivalently, the full exponent in the density is $-1/2$ times the right-hand side. The marginal term is $9(y+1)^2/2=4.5(y+1)^2$. This is consistent with $Y\sim N(-1,1/9)$: its exponent is $-(y+1)^2/[2(1/9)]=-4.5(y+1)^2$.

Consequently, for each $y$, $$ X\mid Y=y\sim N\left(2+\frac32(y+1),\frac34\right). $$ The conditional variance is the Schur complement $3/4$, not the marginal variance $1$ of $X$.

Conditional Density From the Joint Density

For a positive-definite covariance matrix, the joint density of $(X,Y)$, evaluated at $(x,y)$, is $$ f_{X,Y}(x,y) =(2\pi)^{-(p+q)/2}|\Sigma|^{-1/2} \exp\left\{-\frac12 \begin{pmatrix}u\\v\end{pmatrix}^{T} \Sigma^{-1} \begin{pmatrix}u\\v\end{pmatrix}\right\}. $$ The factorization $\Sigma=RDR^T$ also gives $|\Sigma|=|S||B|$, since $|R|=1$. Applying the square-completion identity separates this density into two factors.

Theorem: If $(X,Y)$ is jointly Gaussian with positive-definite covariance partition $\Sigma$ as above, then for every $y\in\mathbb{R}^q$ the conditional density of $X$ given $Y=y$ is the $N_p$ density with mean $\mu_X+CB^{-1}(y-\mu_Y)$ and covariance $S=A-CB^{-1}C^T$. In particular, $$ f_{X\mid Y}(x\mid y) =(2\pi)^{-p/2}|S|^{-1/2} \exp\left\{-\frac12 \left[x-\mu_X-CB^{-1}(y-\mu_Y)\right]^T S^{-1} \left[x-\mu_X-CB^{-1}(y-\mu_Y)\right]\right\}. $$

Proof. Substitution of the quadratic-form identity and $|\Sigma|=|S||B|$ into the joint density gives $$ f_{X,Y}(x,y) = \left[(2\pi)^{-p/2}|S|^{-1/2} \exp\left\{-\frac12(u-CB^{-1}v)^TS^{-1}(u-CB^{-1}v)\right\}\right] \left[(2\pi)^{-q/2}|B|^{-1/2} \exp\left\{-\frac12v^TB^{-1}v\right\}\right]. $$ The second bracket is the $N_q(\mu_Y,B)$ density $f_Y(y)$. To verify that the first bracket is the conditional density, integrate the joint density with respect to $x$, holding $y$ fixed. Under the translation $t=x-\mu_X-CB^{-1}v$, the integral of the first bracket is $1$, because $S$ is positive definite and it is a normalized $N_p(0,S)$ density. Thus integrating gives $f_Y(y)$, and dividing the joint density by the positive marginal density $f_Y(y)$ leaves exactly the first bracket. Since $f_Y(y)>0$ for every $y$, this density defines the conditional law at every $y$. $\square$

Worked Example: A Two-Dimensional Conditional Block

Let $X\in\mathbb{R}^2$ and $Y\in\mathbb{R}$ be jointly Gaussian with means $\mu_X=(0,1)^T$, $\mu_Y=5$, and covariance blocks $$ A=\begin{pmatrix}3&-1\\-1&4\end{pmatrix}, \qquad C=\begin{pmatrix}2\\-2\end{pmatrix}, \qquad B=4. $$ First compute the Schur complement: $$ S=A-CB^{-1}C^T = \begin{pmatrix}3&-1\\-1&4\end{pmatrix} -\frac14 \begin{pmatrix}4&-4\\-4&4\end{pmatrix} = \begin{pmatrix}2&0\\0&3\end{pmatrix}. $$ It is positive definite; therefore the joint covariance is positive definite as well, as seen from the factorization with positive-definite $S$ and $B$.

For an observed value $y$, $CB^{-1}(y-\mu_Y)=\big((y-5)/2,-(y-5)/2\big)^T$. Hence $$ X\mid Y=y\sim N_2\left( \begin{pmatrix}(y-5)/2\\1-(y-5)/2\end{pmatrix}, \begin{pmatrix}2&0\\0&3\end{pmatrix} \right). $$ For example, at $y=7$, the conditional mean is $(1,0)^T$, while the conditional covariance remains $\operatorname{diag}(2,3)$. The value $y=7$ shifts the center but does not alter the conditional covariance.

Worked Example: Conditioning on a Two-Dimensional Block

Let $X$ be scalar and $Y\in\mathbb{R}^2$, with zero means and covariance blocks $$ A=5,\qquad C=\begin{pmatrix}1&1\end{pmatrix}, \qquad B=\begin{pmatrix}2&0\\0&1\end{pmatrix}. $$ Then $B^{-1}=\operatorname{diag}(1/2,1)$, and $$ S=5-CB^{-1}C^T =5-\begin{pmatrix}1&1\end{pmatrix} \begin{pmatrix}1/2\\1\end{pmatrix} =5-\frac32 =\frac72. $$ For $y=(y_1,y_2)^T$, the conditional mean is $CB^{-1}y=y_1/2+y_2$. Thus $$ X\mid Y=y\sim N\left(\frac{y_1}{2}+y_2,\frac72\right). $$ The exponent separates into $(x-y_1/2-y_2)^2/(7/2)$ and $y_1^2/2+y_2^2$, all multiplied by $-1/2$. The latter is exactly $y^TB^{-1}y$, confirming the marginal quadratic form for this $Y$ block.

Common Mistakes and Why the Calculation Matters

The completion separates two different roles. The observed block $y$ determines the conditional center through $CB^{-1}(y-\mu_Y)$; the residual covariance is $S$. Reversing the blocks changes the dimensions and the covariance block used in the inverse, so the order of $X$ and $Y$ must be kept consistent throughout.

Confusing variance and precision. A univariate normal density with variance $\tau^2$ has exponent $-(z-\mu)^2/(2\tau^2)$. If the coefficient of $(z-\mu)^2$ in the quadratic form is $1/\tau^2$, the exponent includes the additional factor $1/2$. For variance $1/9$, the exponent coefficient is $1/[2(1/9)]=4.5$, not $9$.
Using the marginal covariance as the conditional covariance. The conditional covariance of $X$ given $Y$ is $A-CB^{-1}C^T$, not $A$, unless the cross-covariance block $C$ is zero.
Forgetting the mean shifts. The square is in the centered variables $u=x-\mu_X$ and $v=y-\mu_Y$. The conditional center in the original coordinates is $\mu_X+CB^{-1}(y-\mu_Y)$.
Assuming the calculation covers singular cases. In this derivation $B^{-1}$ and $S^{-1}$ are used, so positive definiteness is essential. A singular Gaussian block cannot be handled by simply using the same density formula.

The square-completion argument is useful well beyond a single conditioning calculation: it explains directly why Gaussian conditional densities remain Gaussian, and why the Schur complement controls their spread. It also supplies a reliable algebraic check: expanding the completed square must recover the original joint quadratic form, and integrating the resulting conditional density must leave the marginal density of the conditioned block.

Check Your Understanding

Use the block order shown in each question, and distinguish covariance from precision when checking exponents.

For blocks with covariance $\begin{pmatrix}A&C\\C^T&B\end{pmatrix}$, write the residual inside the completed square for the conditional density of $X$ given $Y=y$.
If $A=6$, $C=2$, and $B=4$ in a scalar-block problem, calculate the Schur complement and the coefficient multiplying $y-\mu_Y$ in the conditional mean.
Why does the determinant factor as $|\Sigma|=|S||B|$ in the square-completion derivation?
A marginal normal variable has variance $1/4$. What is the coefficient of its centered square in the exponent of its density?
State the positive-definiteness conditions needed for the conditional density formula proved here, and explain why a singular case is not covered by the same calculation.

Deriving Conditional Normal Laws by Completing the Square

What You'll Learn