Schur Complements as Conditional Covariance Matrices

Why a Schur Complement Appears in Gaussian Conditioning

Suppose a Gaussian vector is divided into two blocks, \(X\) and \(Y\). Before observing \(Y\), the uncertainty in \(X\) is described by its covariance matrix. After observing \(Y\), some of that uncertainty may be removed. The matrix that describes what remains has a standard block-matrix form: the Schur complement.

This connection is useful in two ways. It gives a compact expression for conditional covariance, and it lets us check whether a proposed block matrix can be a valid covariance matrix. The key algebraic condition is that the covariance block for the conditioned vector be invertible; here we assume it is positive definite.

Definition: Let \(\Sigma=\begin{pmatrix}A&C\\C^T&D\end{pmatrix}\), where \(A\) and \(D\) are symmetric and \(D\) is invertible. The Schur complement of \(D\) in \(\Sigma\) is \(S=A-CD^{-1}C^T\). In covariance applications, \(A=\operatorname{Cov}(X)\), \(D=\operatorname{Cov}(Y)\), and \(C=\operatorname{Cov}(X,Y)\).

If \(X\) has \(p\) coordinates and \(Y\) has \(q\), then \(A\) is \(p\times p\), \(D\) is \(q\times q\), and \(C\) is \(p\times q\). The product \(CD^{-1}C^T\) is therefore \(p\times p\), just like \(A\). The order of the blocks matters: this expression is the Schur complement associated with conditioning on \(Y\), whose covariance is \(D\).

Conditional Covariance Is the Schur Complement

Let \(\begin{pmatrix}X\\Y\end{pmatrix}\) be jointly Gaussian, with mean \(\begin{pmatrix}\mu_X\\\mu_Y\end{pmatrix}\) and covariance \(\Sigma=\begin{pmatrix}A&C\\C^T&D\end{pmatrix}\). Assume \(D\) is positive definite. The block conditional formula for a Gaussian vector gives the conditional covariance of \(X\) given \(Y=y\) as \(A-CD^{-1}C^T\). Thus this conditional covariance is exactly the Schur complement of \(D\) in \(\Sigma\).

The formula does not depend on the particular value \(y\). For a Gaussian distribution, changing the observed value changes the conditional mean, but not the conditional covariance. In the next example, we calculate the Schur complement directly and check that it is a valid variance.

Worked Example: Conditioning One Gaussian Variable

Let \(\begin{pmatrix}X\\Y\end{pmatrix}\sim N\left(\begin{pmatrix}0\\0\end{pmatrix},\begin{pmatrix}5&2\\2&3\end{pmatrix}\right)\). Find the conditional variance of \(X\) given \(Y=y\).

Here \(A=5\), \(C=2\), and \(D=3\). Since \(D^{-1}=1/3\), the Schur complement is

S=A-CD^{-1}C^T =5-2\left(\frac{1}{3}\right)2 =5-\frac{4}{3} =\frac{11}{3}.

Therefore, \(\operatorname{Var}(X\mid Y=y)=11/3\), for every \(y\). This is positive, as a variance must be. The joint covariance is also positive definite: its leading diagonal entry is \(5>0\), and its determinant is \(5(3)-2^2=11>0\). The positive conditional variance is consistent with the joint covariance being positive definite.

Block Decomposition and Positivity

The Schur complement does more than name the conditional covariance. A block factorization shows precisely how its positivity is tied to the positivity of the full covariance matrix.

Theorem: Let \(\Sigma=\begin{pmatrix}A&C\\C^T&D\end{pmatrix}\) be symmetric, with \(D\) positive definite, and let \(S=A-CD^{-1}C^T\). Then \(\Sigma\) is positive semidefinite if and only if \(S\) is positive semidefinite. Likewise, \(\Sigma\) is positive definite if and only if \(S\) is positive definite.

Proof. Define the invertible block matrix \(L=\begin{pmatrix}I&CD^{-1}\\0&I\end{pmatrix}\). Direct block multiplication gives

L \begin{pmatrix}S&0\\0&D\end{pmatrix} L^T = \begin{pmatrix} S+CD^{-1}C^T&C\\ C^T&D \end{pmatrix} = \begin{pmatrix}A&C\\C^T&D\end{pmatrix} =\Sigma.

Because \(L\) is invertible, this is a congruence transformation: for any vector \(w\), the quadratic form of \(\Sigma\) at \(w\) equals the quadratic form of \(\operatorname{diag}(S,D)\) at \(L^Tw\). As \(w\) ranges over all vectors, so does \(L^Tw\). Consequently, \(\Sigma\) and \(\operatorname{diag}(S,D)\) are positive semidefinite at the same time, and they are positive definite at the same time. Since \(D\) is already positive definite, \(\operatorname{diag}(S,D)\) is positive semidefinite exactly when \(S\) is, and positive definite exactly when \(S\) is. This proves both claims. \(\square\)

For a jointly Gaussian vector with \(D\) positive definite, \(\Sigma\) is a covariance matrix and hence is positive semidefinite. The theorem therefore shows that the conditional covariance \(S\) must be positive semidefinite. If the full joint covariance is positive definite, then \(S\) must be positive definite as well. Positive semidefinite does not always mean positive definite: a conditional distribution can have zero variance in some direction.

Worked Example: A Two-Dimensional Conditional Covariance

Suppose the covariance blocks for \(X\in\mathbb{R}^2\) and \(Y\in\mathbb{R}^2\) are

A=\begin{pmatrix}3.5&0.5\\0.5&3.5\end{pmatrix}, \qquad C=\begin{pmatrix}1&0\\1&1\end{pmatrix}, \qquad D=\begin{pmatrix}2&0\\0&1\end{pmatrix}.

The matrix \(D\) is positive definite and \(D^{-1}=\begin{pmatrix}1/2&0\\0&1\end{pmatrix}\). First calculate

CD^{-1} = \begin{pmatrix}1/2&0\\1/2&1\end{pmatrix}, \qquad CD^{-1}C^T = \begin{pmatrix}1/2&1/2\\1/2&3/2\end{pmatrix}.

Subtracting this matrix from \(A\) gives

S=A-CD^{-1}C^T = \begin{pmatrix}3&0\\0&2\end{pmatrix}.

Thus the conditional covariance of \(X\) given \(Y=y\) is diagonal, with conditional variances \(3\) and \(2\). Both are positive, so \(S\) is positive definite. The block factorization also verifies that the displayed blocks form a valid positive-definite joint covariance: its middle block matrix \(\operatorname{diag}(S,D)\) is positive definite, and the factor \(L\) is invertible.

A Quadratic-Form Characterization

There is another way to read the Schur complement. For a fixed vector \(u\in\mathbb{R}^p\), consider the quadratic form of the full block matrix over vectors whose first block is \(u\), while the second block \(v\in\mathbb{R}^q\) is allowed to vary. Completing the square identifies the smallest value of that quadratic form.

Theorem: If \(D\) is positive definite and \(S=A-CD^{-1}C^T\), then for every \(u\in\mathbb{R}^p\), \[ u^TSu=\min_{v\in\mathbb{R}^q} \begin{pmatrix}u\\v\end{pmatrix}^{T} \begin{pmatrix}A&C\\C^T&D\end{pmatrix} \begin{pmatrix}u\\v\end{pmatrix}. \] The minimum is attained at \(v=-D^{-1}C^Tu\).

Proof. Expand the quadratic form on the right:

$$ u^TAu+2u^TCv+v^TDv. $$

Since \(D\) is symmetric and positive definite, \(D^{-1}\) is also symmetric. Expanding the following square gives

\begin{aligned} u^TSu+(v+D^{-1}C^Tu)^TD(v+D^{-1}C^Tu) &=u^T(A-CD^{-1}C^T)u\\ &\quad+v^TDv+2u^TCv+u^TCD^{-1}C^Tu\\ &=u^TAu+2u^TCv+v^TDv. \end{aligned}

The equality holds because the negative term \(-u^TCD^{-1}C^Tu\) in the first part cancels the same positive term from the expanded square. The square term is nonnegative and equals zero exactly when \(v=-D^{-1}C^Tu\), because \(D\) is positive definite. The minimum is therefore \(u^TSu\), as claimed. \(\square\)

In particular, if the full block matrix is positive semidefinite, its quadratic form is nonnegative for every pair \((u,v)\). Its minimum over \(v\) must then also be nonnegative, so \(u^TSu\geq 0\) for every \(u\). This gives a direct explanation for why the Schur complement of a covariance block is positive semidefinite.

Worked Example: A Singular Conditional Covariance

Let \(Y\) and \(Z\) be independent \(N(0,1)\) variables, and define \(X_1=Y\) and \(X_2=Y+Z\). Take \(X=(X_1,X_2)^T\). The covariances are \(\operatorname{Var}(X_1)=1\), \(\operatorname{Var}(X_2)=2\), and \(\operatorname{Cov}(X_1,X_2)=1\). Also, \(\operatorname{Cov}(X,Y)=(1,1)^T\) and \(\operatorname{Var}(Y)=1\). Thus the joint covariance, ordered as \((X_1,X_2,Y)\), is

\Sigma= \begin{pmatrix} 1&1&1\\ 1&2&1\\ 1&1&1 \end{pmatrix}.

Here \(A=\begin{pmatrix}1&1\\1&2\end{pmatrix}\), \(C=\begin{pmatrix}1\\1\end{pmatrix}\), and \(D=1\). Therefore

S=A-CD^{-1}C^T = \begin{pmatrix}1&1\\1&2\end{pmatrix} - \begin{pmatrix}1&1\\1&1\end{pmatrix} = \begin{pmatrix}0&0\\0&1\end{pmatrix}.

This matrix is positive semidefinite but not positive definite. Directly from the construction, after conditioning on \(Y=y\), \(X_1=y\) is fixed and \(X_2=y+Z\) still has variance \(1\). The conditional covariance is therefore exactly the displayed matrix. Its zero first diagonal entry reflects the fact that \(X_1\) is determined by the observed \(Y\); it is not a contradiction or an invalid covariance.

Conditions and a Common Mistake

The inverse \(D^{-1}\) is essential to the ordinary Schur complement formula. Positive definiteness of \(D\) guarantees that this inverse exists and that the block factorizations and minimization above apply. If \(D\) is singular, the expression \(A-CD^{-1}C^T\) is not defined as written; such cases require separate treatment rather than silently substituting an inverse.

A common mistake is to subtract \(CC^T\) from \(A\), omitting the factor \(D^{-1}\). That calculation is correct only in the special case \(D=I\). For example, in the scalar example above, the incorrect expression would give \(5-2^2=1\), while the correct conditional variance is \(5-2(1/3)2=11/3\). Another error is to use the wrong covariance block in the inverse: conditioning on \(Y\) uses \(D=\operatorname{Cov}(Y)\), not \(A=\operatorname{Cov}(X)\).

The Schur complement is therefore both a conditional covariance formula and a structural check on a block covariance matrix. Given a positive-definite conditioning block \(D\), a proposed joint covariance is positive semidefinite precisely when its Schur complement is positive semidefinite. This can simplify theoretical arguments and practical checks of multivariate covariance models.

Check Your Understanding

Use the block covariance definitions and conditions in this tutorial to answer the following.

For \(\Sigma=\begin{pmatrix}A&C\\C^T&D\end{pmatrix}\), which block is inverted when calculating the conditional covariance of \(X\) given \(Y\), and why?
If \(D\) is positive definite and \(\Sigma\) is positive semidefinite, what can be concluded about \(A-CD^{-1}C^T\)?
Why does positive definiteness of the full joint covariance imply positive definiteness, rather than merely positive semidefiniteness, of the Schur complement?
What is the minimizing value of \(v\) in the quadratic-form characterization, and what is the minimum?
Can a Gaussian conditional covariance be singular? Give the feature of the worked example that accounts for its zero variance direction.