Block Formulas for Gaussian Conditional Means and Covariances

From a Partition to Conditional Parameters

When a Gaussian vector is divided into two blocks, conditioning on one block changes the mean of the other and leaves it with a conditional covariance matrix. The block formulas make both changes explicit: the cross-covariance determines how the observed block shifts the conditional mean, while the conditioning covariance determines how that cross-covariance is scaled.

Let $X$ be a $p$-dimensional Gaussian vector and $Y$ a $q$-dimensional Gaussian vector. Write their joint mean and covariance in the partitioned form

\begin{pmatrix}X\\Y\end{pmatrix} \sim N\!\left( \begin{pmatrix}\mu_X\\\mu_Y\end{pmatrix}, \begin{pmatrix} \Sigma_{XX} & \Sigma_{XY}\\ \Sigma_{YX} & \Sigma_{YY} \end{pmatrix} \right).

Here $\Sigma_{XX}$ is $p\times p$, $\Sigma_{YY}$ is $q\times q$, $\Sigma_{XY}$ is $p\times q$, and $\Sigma_{YX}=\Sigma_{XY}^{T}$ is $q\times p$. The joint covariance is assumed positive semidefinite, as every covariance matrix must be. Throughout this tutorial, assume that $\Sigma_{YY}$ is invertible. Because it is a covariance matrix, invertibility also makes it positive definite.

Theorem: For jointly Gaussian $X$ and $Y$, if $\Sigma_{YY}$ is invertible, the conditional distribution of $X$ given $Y=y$ is Gaussian, with $$ E[X\mid Y=y] = \mu_X+\Sigma_{XY}\Sigma_{YY}^{-1}(y-\mu_Y) $$ and $$ \operatorname{Cov}(X\mid Y=y) = \Sigma_{XX}-\Sigma_{XY}\Sigma_{YY}^{-1}\Sigma_{YX}. $$ These formulas specify a conditional Gaussian law; the conditional covariance does not depend on the particular value $y$.

The completing-the-square result for conditional normal laws gives this conditional Gaussian form. The block notation identifies its parameters: the matrix multiplying the centered observation $y-\mu_Y$ is $p\times q$, and the covariance adjustment is $p\times p$. In particular, the rightmost factor in the covariance adjustment is $\Sigma_{YX}$, not $\Sigma_{XY}$.

Reading the Formulas as Matrix Operations

It is useful to calculate the formulas in an order that keeps dimensions visible. Set

K=\Sigma_{XY}\Sigma_{YY}^{-1}.

Then $K$ is $p\times q$, and the conditional mean is $\mu_X+K(y-\mu_Y)$. The matrix $K$ takes a centered $q$-vector of observations and produces a $p$-vector shift. For the covariance, first multiply $K$ by $\Sigma_{YX}$, which is $q\times p$. The result is $p\times p$, matching $\Sigma_{XX}$, so the subtraction is defined.

\underbrace{\Sigma_{XY}}_{p\times q} \underbrace{\Sigma_{YY}^{-1}}_{q\times q} \underbrace{\Sigma_{YX}}_{q\times p} \quad\text{is }p\times p.

The formulas also pass simple checks. If $y=\mu_Y$, the observed block is at its mean, so the conditional mean is $\mu_X$. If $\Sigma_{XY}=0$, the displayed mean shift and covariance adjustment both vanish. The general formulas do not require either of these special cases.

Worked Example: Conditioning a Two-Dimensional Block on One Observation

Suppose $X=(X_1,X_2)^T$ and $Y$ have means $\mu_X=(1,-1)^T$, $\mu_Y=2$, and covariance blocks $$ \Sigma_{XX}=\begin{pmatrix}3&1\\1&5\end{pmatrix}, \qquad \Sigma_{XY}=\begin{pmatrix}1\\2\end{pmatrix}, \qquad \Sigma_{YY}=(4). $$ Find the conditional mean and covariance of $X$ when $Y=3$.

Conditional mean. The inverse of the conditioning covariance is $\Sigma_{YY}^{-1}=(1/4)$, and the centered observation is $3-2=1$. Thus $$ \mu_X+\Sigma_{XY}\Sigma_{YY}^{-1}(3-\mu_Y) = \begin{pmatrix}1\\-1\end{pmatrix} + \begin{pmatrix}1\\2\end{pmatrix}\frac14(1) = \begin{pmatrix}5/4\\-1/2\end{pmatrix}. $$

Conditional covariance. The adjustment is $$ \Sigma_{XY}\Sigma_{YY}^{-1}\Sigma_{YX} = \begin{pmatrix}1\\2\end{pmatrix} \frac14 \begin{pmatrix}1&2\end{pmatrix} = \begin{pmatrix}1/4&1/2\\1/2&1\end{pmatrix}. $$ Therefore $$ \operatorname{Cov}(X\mid Y=3) = \begin{pmatrix}3&1\\1&5\end{pmatrix} - \begin{pmatrix}1/4&1/2\\1/2&1\end{pmatrix} = \begin{pmatrix}11/4&1/2\\1/2&4\end{pmatrix}. $$ The off-diagonal entries agree, as a covariance matrix must be symmetric.

Why the Conditional Covariance Is Valid

The covariance formula subtracts a matrix from $\Sigma_{XX}$, so it is important to check that the result is still positive semidefinite. This follows directly from the positive semidefiniteness of the joint covariance.

Result: If the joint covariance $$ \begin{pmatrix} \Sigma_{XX}&\Sigma_{XY}\\ \Sigma_{YX}&\Sigma_{YY} \end{pmatrix} $$ is positive semidefinite and $\Sigma_{YY}$ is invertible, then $$ S=\Sigma_{XX}-\Sigma_{XY}\Sigma_{YY}^{-1}\Sigma_{YX} $$ is positive semidefinite.

Proof. The matrix $\Sigma_{YY}$ is symmetric positive definite, so its inverse is symmetric. For any $u\in\mathbb{R}^p$, form the $(p+q)$-vector $$ v=\begin{pmatrix}u\\-\Sigma_{YY}^{-1}\Sigma_{YX}u\end{pmatrix}. $$ Positive semidefiniteness of the joint covariance implies that its quadratic form at $v$ is nonnegative. Expanding the block multiplication gives $$ \begin{aligned} v^T \begin{pmatrix}\Sigma_{XX}&\Sigma_{XY}\\ \Sigma_{YX}&\Sigma_{YY}\end{pmatrix}v &= u^T\Sigma_{XX}u -2u^T\Sigma_{XY}\Sigma_{YY}^{-1}\Sigma_{YX}u\\ &\quad+ u^T\Sigma_{XY}\Sigma_{YY}^{-1} \Sigma_{YY}\Sigma_{YY}^{-1}\Sigma_{YX}u\\ &= u^T\bigl(\Sigma_{XX}-\Sigma_{XY}\Sigma_{YY}^{-1}\Sigma_{YX}\bigr)u\\ &=u^TSu\geq 0. \end{aligned} $$ Since this holds for every $u$, $S$ is positive semidefinite. $\square$

This check guarantees that the formula gives a valid covariance, including in cases where $S$ is singular. It does not require the entire joint covariance matrix to be invertible; only the conditioning block $\Sigma_{YY}$ must be invertible.

Worked Example: Inverting a Two-by-Two Conditioning Block

Now let $X$ be scalar and $Y=(Y_1,Y_2)^T$. Take $\mu_X=10$, $\mu_Y=(0,1)^T$, and $$ \Sigma_{XX}=(5),\qquad \Sigma_{XY}=\begin{pmatrix}2&1\end{pmatrix},\qquad \Sigma_{YY}=\begin{pmatrix}4&1\\1&2\end{pmatrix}. $$ Find the conditional law of $X$ given $Y=(1,3)^T$.

The determinant of $\Sigma_{YY}$ is $4(2)-1(1)=7$, so $$ \Sigma_{YY}^{-1} = \frac17\begin{pmatrix}2&-1\\-1&4\end{pmatrix}. $$ Multiplying in order, $$ \Sigma_{XY}\Sigma_{YY}^{-1} = \frac17\begin{pmatrix}2&1\end{pmatrix} \begin{pmatrix}2&-1\\-1&4\end{pmatrix} = \begin{pmatrix}3/7&2/7\end{pmatrix}. $$ The centered observed vector is $Y-\mu_Y=(1,2)^T$, hence the mean is $$ 10+\begin{pmatrix}3/7&2/7\end{pmatrix} \begin{pmatrix}1\\2\end{pmatrix} = 10+\frac{3+4}{7} =11. $$ For the variance, use $\Sigma_{YX}=(2,1)^T$: $$ \Sigma_{XY}\Sigma_{YY}^{-1}\Sigma_{YX} = \begin{pmatrix}3/7&2/7\end{pmatrix} \begin{pmatrix}2\\1\end{pmatrix} = \frac87. $$ Thus $$ \operatorname{Var}(X\mid Y=(1,3)^T)=5-\frac87=\frac{27}{7}. $$ This is positive, as required for a scalar variance.

More Than One Response Component

When both blocks have multiple components, the same order of multiplication applies. Keeping the intermediate products visible helps prevent a transpose from being placed incorrectly.

Worked Example: Two Components in Each Block

Suppose $\mu_X=(0,2)^T$, $\mu_Y=(1,-1)^T$, and $$ \Sigma_{XX}=\begin{pmatrix}4&1\\1&5\end{pmatrix},\qquad \Sigma_{XY}=\begin{pmatrix}1&0\\1&1\end{pmatrix},\qquad \Sigma_{YY}=\begin{pmatrix}2&0\\0&3\end{pmatrix}. $$ Calculate the conditional parameters at $y=(3,2)^T$.

First, $\Sigma_{YY}^{-1}=\begin{pmatrix}1/2&0\\0&1/3\end{pmatrix}$, so $$ \Sigma_{XY}\Sigma_{YY}^{-1} = \begin{pmatrix}1/2&0\\1/2&1/3\end{pmatrix}. $$ The centered observation is $(2,3)^T$. Therefore $$ E[X\mid Y=y] = \begin{pmatrix}0\\2\end{pmatrix} + \begin{pmatrix}1/2&0\\1/2&1/3\end{pmatrix} \begin{pmatrix}2\\3\end{pmatrix} = \begin{pmatrix}0\\2\end{pmatrix} + \begin{pmatrix}1\\2\end{pmatrix} = \begin{pmatrix}1\\4\end{pmatrix}. $$ For the covariance adjustment, $\Sigma_{YX}=\begin{pmatrix}1&1\\0&1\end{pmatrix}$, giving $$ \Sigma_{XY}\Sigma_{YY}^{-1}\Sigma_{YX} = \begin{pmatrix}1/2&0\\1/2&1/3\end{pmatrix} \begin{pmatrix}1&1\\0&1\end{pmatrix} = \begin{pmatrix}1/2&1/2\\1/2&5/6\end{pmatrix}. $$ Consequently, $$ \operatorname{Cov}(X\mid Y=y) = \begin{pmatrix}4&1\\1&5\end{pmatrix} - \begin{pmatrix}1/2&1/2\\1/2&5/6\end{pmatrix} = \begin{pmatrix}7/2&1/2\\1/2&25/6\end{pmatrix}. $$ The result is symmetric. Its diagonal entries are positive, and its determinant is $$ \frac72\frac{25}{6}-\left(\frac12\right)^2 = \frac{175}{12}-\frac14 = \frac{43}{3}>0, $$ so in this example the conditional covariance is positive definite.

Common Mistakes and a Reliable Check

The formulas are compact, but several errors are easy to make when the blocks have different dimensions. A reliable habit is to write the size of each matrix before multiplying.

Reversing the cross-covariance blocks. The mean formula starts with $\Sigma_{XY}$, which maps a $q$-vector into a $p$-vector. Using $\Sigma_{YX}$ there usually makes the product dimensionally impossible.
Using the wrong inverse. The inverse is $\Sigma_{YY}^{-1}$, the covariance of the block being conditioned on. Inverting $\Sigma_{XX}$ answers a different algebraic question.
Forgetting to center the observation. The mean shift uses $y-\mu_Y$, not $y$. If the observed value equals $\mu_Y$, the conditional mean must equal $\mu_X$.
Assuming the conditional covariance depends on $y$. In the Gaussian formulas here, it is determined by the covariance blocks alone.
Checking only the mean calculation. The covariance product must end as a symmetric $p\times p$ matrix. A nonsymmetric result often signals a missing transpose or incorrect multiplication order.

The block formulas matter because they translate a joint Gaussian specification into conditional parameters using ordinary matrix operations. The cross-covariance block determines the direction and scale of the mean shift, while the Schur-complement expression supplies the covariance of the remaining uncertainty. When applying them, first confirm that the conditioning covariance is invertible, then check dimensions at every multiplication, and finally verify that the resulting covariance is symmetric and positive semidefinite.

Check Your Understanding

Use the dimensions and formulas from the tutorial to answer these questions.

If $X$ has dimension $p$ and $Y$ has dimension $q$, what are the dimensions of $\Sigma_{XY}$, $\Sigma_{YY}^{-1}$, and the conditional covariance?
Why does observing $y=\mu_Y$ leave the conditional mean equal to $\mu_X$?
In the covariance formula, explain why the final cross-covariance factor is $\Sigma_{YX}$, not $\Sigma_{XY}$.
What assumption is needed about $\Sigma_{YY}$ to use the displayed formulas, and why is its invertibility important?
How does evaluating the joint covariance quadratic form at $(u,-\Sigma_{YY}^{-1}\Sigma_{YX}u)^T$ establish that the conditional covariance is positive semidefinite?