Schur Complements in Block Covariance Matrices

Why a Block Covariance Matrix Has a Schur Complement

A covariance matrix for two groups of random variables naturally separates into blocks. The diagonal blocks describe variation within each group, while the off-diagonal block describes their cross-covariances. The Schur complement combines these blocks into a smaller matrix that captures what remains after an algebraic elimination of one block. This makes it a practical tool for checking whether a proposed block matrix can be a covariance matrix.

Let \(U\) be a random vector in \(\mathbb{R}^p\) and \(V\) a random vector in \(\mathbb{R}^q\), both with finite second moments. Their joint covariance matrix has the form

M= \begin{pmatrix} A&B\\ B^\mathsf{T}&C \end{pmatrix}, \qquad A=\operatorname{Cov}(U),\quad B=\operatorname{Cov}(U,V),\quad C=\operatorname{Cov}(V).

Here \(A\) and \(C\) are symmetric, and \(B\) is a \(p\)-by-\(q\) matrix. Every covariance matrix is positive semidefinite: for any vectors \(x\in\mathbb{R}^p\) and \(y\in\mathbb{R}^q\), the quadratic form \((x^\mathsf{T},y^\mathsf{T})M(x^\mathsf{T},y^\mathsf{T})^\mathsf{T}\) is the variance of a linear combination of the entries of \(U\) and \(V\), so it cannot be negative.

Definition: For a block matrix \(M=\begin{pmatrix}A&B\\B^\mathsf{T}&C\end{pmatrix}\) with \(A\) invertible, the Schur complement of \(A\) in \(M\) is

S=C-B^\mathsf{T}A^{-1}B.

For a covariance matrix, \(A\) is symmetric, so whenever it is invertible it is also symmetric positive definite if it is a covariance block.

Block Elimination When the Leading Block Is Positive Definite

Suppose \(A\) is symmetric positive definite. The matrix \(A^{-1}\) is then symmetric, and the following block matrix is invertible:

P= \begin{pmatrix} I_p&-A^{-1}B\\ 0&I_q \end{pmatrix}.

Multiplying out gives the block-elimination identity

P^\mathsf{T}MP = \begin{pmatrix} A&0\\ 0&C-B^\mathsf{T}A^{-1}B \end{pmatrix} = \begin{pmatrix} A&0\\ 0&S \end{pmatrix}.

For clarity, the first multiplication gives \(MP=\begin{pmatrix}A&0\\B^\mathsf{T}&C-B^\mathsf{T}A^{-1}B\end{pmatrix}\). Multiplying on the left by \(P^\mathsf{T}\) cancels the lower-left block because \(A^{-1}\) is symmetric. Since \(P\) is invertible, \(P^\mathsf{T}MP\) is a congruence transformation of \(M\). An invertible congruence preserves positive semidefiniteness: its quadratic form at a vector \(z\) is the quadratic form of \(M\) at \(Pz\), and \(Pz\) ranges over all vectors.

Theorem: If \(A\) is symmetric positive definite, then

\begin{pmatrix}A&B\\B^\mathsf{T}&C\end{pmatrix}\succeq 0 \quad\Longleftrightarrow\quad C-B^\mathsf{T}A^{-1}B\succeq 0.

The full block matrix is positive definite if and only if its Schur complement is positive definite.

Proof. The block-elimination identity shows that \(M\) is positive semidefinite exactly when the block-diagonal matrix with diagonal blocks \(A\) and \(S\) is positive semidefinite. Because \(A\) is positive definite, this is equivalent to \(S\) being positive semidefinite. Replacing “positive semidefinite” throughout by “positive definite” gives the positive-definite claim. \(\square\)

The same multiplication also gives a useful factorization:

M= \begin{pmatrix} I_p&0\\ B^\mathsf{T}A^{-1}&I_q \end{pmatrix} \begin{pmatrix} A&0\\ 0&S \end{pmatrix} \begin{pmatrix} I_p&A^{-1}B\\ 0&I_q \end{pmatrix}.

In particular, when \(A\) is positive definite, \(M\) is a valid covariance matrix precisely when \(S\) is positive semidefinite. If \(S\) has a negative quadratic direction, the block matrix does too, even if both diagonal blocks \(A\) and \(C\) look individually acceptable.

Worked Example: Elimination With a Positive-Definite Block

Take \(A=\begin{pmatrix}2&1\\1&2\end{pmatrix}\), \(B=\begin{pmatrix}1\\0\end{pmatrix}\), and \(C=(2)\). First, \(A\) is positive definite: its leading diagonal entry is \(2>0\), and its determinant is \(4-1=3>0\). Its inverse is

A^{-1}=\frac{1}{3} \begin{pmatrix} 2&-1\\ -1&2 \end{pmatrix}, \qquad B^\mathsf{T}A^{-1}B=\frac{2}{3}.

Thus the Schur complement is the scalar \(S=2-\frac{2}{3}=\frac{4}{3}>0\). The theorem therefore says that \(M=\begin{pmatrix}2&1&1\\1&2&0\\1&0&2\end{pmatrix}\) is positive definite. The elimination can also be checked directly: with \(A^{-1}B=(\frac{2}{3},-\frac{1}{3})^\mathsf{T}\),

P^\mathsf{T}MP= \begin{pmatrix} 2&1&0\\ 1&2&0\\ 0&0&\frac{4}{3} \end{pmatrix}.

The upper block remains \(A\), and the lower scalar is exactly the Schur complement.

Singular Leading Blocks

A covariance block need not be invertible. For example, two coordinates may be exact linear combinations of one another. In this case, the ordinary inverse in the Schur complement does not exist, and positive semidefiniteness requires an additional compatibility condition between \(A\) and \(B\).

For a symmetric positive semidefinite matrix \(A\), write \(A^+\) for its Moore–Penrose pseudoinverse. It is obtained by diagonalizing \(A\), replacing each positive eigenvalue by its reciprocal, and leaving each zero eigenvalue at zero. The range \(\operatorname{Range}(A)\) is the set of vectors \(Aw\) as \(w\) varies.

Theorem: Let \(A\) and \(C\) be symmetric, and let \(A\succeq0\). Then

\begin{pmatrix}A&B\\B^\mathsf{T}&C\end{pmatrix}\succeq0

if and only if both conditions hold:

every column of \(B\) belongs to \(\operatorname{Range}(A)\);
\(C-B^\mathsf{T}A^+B\succeq0\).

Proof. Consider the quadratic form

q(x,y)=x^\mathsf{T}Ax+2x^\mathsf{T}By+y^\mathsf{T}Cy.

First suppose \(M\succeq0\). If \(u\in\operatorname{Null}(A)\), then for any \(y\) and real \(t\), \(q(tu,y)=2t\,u^\mathsf{T}By+y^\mathsf{T}Cy\) must be nonnegative. This is possible for every real \(t\) only if \(u^\mathsf{T}By=0\) for every \(y\). Thus every column of \(B\) is orthogonal to \(\operatorname{Null}(A)\). Since \(A\) is symmetric, \(\operatorname{Range}(A)=\operatorname{Null}(A)^\perp\), so the range condition follows.

Under that condition, \(AA^+B=B\). Using the defining identities of the pseudoinverse for symmetric positive semidefinite \(A\), in particular \(A^+AA^+=A^+\), we can complete the quadratic form:

q(x,y) = (x+A^+By)^\mathsf{T}A(x+A^+By) + y^\mathsf{T}(C-B^\mathsf{T}A^+B)y.

For necessity of the second condition, choose \(x=-A^+By\). The first term is then zero, so \(q(x,y)=y^\mathsf{T}(C-B^\mathsf{T}A^+B)y\geq0\) for every \(y\).

Conversely, if the range condition and the stated positive-semidefiniteness condition hold, the displayed decomposition writes \(q(x,y)\) as a sum of two nonnegative terms for every \(x,y\). Hence \(M\succeq0\). \(\square\)

When \(A\) is invertible and positive semidefinite, it is positive definite and \(A^+=A^{-1}\); the range condition is automatic. The singular theorem therefore extends the earlier criterion rather than contradicting it.

Worked Example: A Singular Block That Is Compatible

Let \(A=\begin{pmatrix}1&1\\1&1\end{pmatrix}\), \(B=\begin{pmatrix}1\\1\end{pmatrix}\), and \(C=(3)\). The matrix \(A\) is singular: it has eigenvalues \(2\) and \(0\), with range spanned by \((1,1)^\mathsf{T}\). The column of \(B\) lies in that range. Since \(A^+=A/4\),

A^+B= \frac{1}{4} \begin{pmatrix}1&1\\1&1\end{pmatrix} \begin{pmatrix}1\\1\end{pmatrix} = \begin{pmatrix}1/2\\1/2\end{pmatrix}, \qquad B^\mathsf{T}A^+B=1.

The generalized Schur complement is \(S=3-1=2\geq0\). Both conditions of the theorem hold, so the block matrix is positive semidefinite. Indeed, its quadratic form is \(q(x,y)=(x_1+x_2+y)^2+2y^2\), which is nonnegative for every \(x_1,x_2,y\).

Worked Example: A Cross-Block That Makes a Covariance Impossible

Consider \(A=\begin{pmatrix}1&0\\0&0\end{pmatrix}\), \(B=\begin{pmatrix}0\\1\end{pmatrix}\), and \(C=(2)\). The range of \(A\) is spanned by \((1,0)^\mathsf{T}\), but the column of \(B\) is \((0,1)^\mathsf{T}\), which is not in that range. The required range condition fails.

The failure is visible directly in the quadratic form. With \(x=(0,-2)^\mathsf{T}\) and \(y=1\), \(x^\mathsf{T}Ax=0\), \(2x^\mathsf{T}By=-4\), and \(y^\mathsf{T}Cy=2\). Therefore \(q(x,y)=0-4+2=-2<0\). This block matrix cannot be a covariance matrix, despite \(A\succeq0\) and \(C\geq0\). A pseudoinverse calculation alone would miss the problem: \(A^+=A\) gives \(B^\mathsf{T}A^+B=0\), but the range condition must be checked as well.

Interpretation, Calculation, and a Common Pitfall

When \(A\) is positive definite, completing the square also gives a useful optimization interpretation. For each fixed \(y\), the quadratic form \(q(x,y)\) is minimized over \(x\) at \(x=-A^{-1}By\). Substitution yields

\min_{x\in\mathbb{R}^p}q(x,y) = y^\mathsf{T}(C-B^\mathsf{T}A^{-1}B)y = y^\mathsf{T}Sy.

Thus the Schur complement records the remaining quadratic form after the first block has been eliminated. This algebraic role explains why it is useful in covariance calculations: it reduces a question about a large block matrix to a smaller matrix, while retaining the relevant definiteness information. The singular case follows the same idea, but only after the cross-block satisfies the range condition.

Common mistake: applying a symmetric congruence identity without symmetry. The expression \(C-B^\mathsf{T}A^{-1}B\) can be written whenever \(A\) is invertible, but the covariance positive-semidefiniteness argument above relies on \(M\) being symmetric and \(A\) being symmetric. If \(A\) is nonsymmetric, the claimed multiplication \(P^\mathsf{T}MP=\operatorname{diag}(A,S)\) need not eliminate both off-diagonal blocks. In a covariance matrix, symmetry is guaranteed; in a general block-matrix calculation, it must be stated rather than assumed. A Schur-complement formula alone is not a license to use a symmetric congruence argument for a nonsymmetric matrix.

For a proposed covariance matrix, a reliable check is to verify the assumptions before calculating: confirm symmetry; establish whether the leading block is positive definite or singular; use the ordinary inverse criterion only in the invertible case; and, in the singular case, check both the range condition and the generalized Schur complement. Checking only the diagonal blocks is insufficient.

Key takeaway: For a symmetric block covariance matrix with positive-definite leading block, positive semidefiniteness is equivalent to positive semidefiniteness of its Schur complement. With a singular leading block, the cross-block must also lie in the leading block’s range.

Check Your Understanding

Use the definitions and conditions developed above to answer these questions.

For a symmetric block matrix with \(A\) positive definite, state the Schur complement of \(A\) and the condition it must satisfy for the full matrix to be positive semidefinite.
Why does the block-elimination argument use a congruence transformation, and which symmetry assumption makes the stated identity valid?
If \(A\) is singular and positive semidefinite, what additional condition on \(B\) is required before checking \(C-B^\mathsf{T}A^+B\)?
For \(A=\begin{pmatrix}2&0\\0&1\end{pmatrix}\), \(B=\begin{pmatrix}1\\1\end{pmatrix}\), and \(C=(2)\), calculate the Schur complement and decide whether the block matrix is positive semidefinite.
Give a reason that checking only whether \(A\succeq0\) and \(C\succeq0\) does not suffice to establish that the full block matrix is a covariance matrix.

Schur Complements in Block Covariance Matrices

What You'll Learn