Schur Complements Of Partitioned Wishart Matrices

Why the Schur Complement Appears

A partitioned scatter matrix separates variation in one group of coordinates from variation in another. Its Schur complement measures the scatter in the second group that remains after its linear association with the first group is removed. In a Gaussian model, this residual scatter has a Wishart distribution, but with fewer degrees of freedom than the original matrix. The loss is exactly the number of coordinates used in the projection.

We derive that law and its independence properties from the Gaussian sample construction. Throughout, the covariance matrix is positive definite, and the Wishart matrix is central.

Setup and Definition

Definition: Let $X_1,\ldots,X_\nu$ be independent $N_p(0,\Sigma)$ random vectors, where $\Sigma$ is positive definite, and define $W=\sum_{j=1}^{\nu}X_jX_j^T$. Then $W\sim W_p(\Sigma,\nu)$. Partition $p=p_1+p_2$, with $p_1,p_2\geq 1$, and write $$ W=\begin{pmatrix}W_{11}&W_{12}\\W_{21}&W_{22}\end{pmatrix}, \qquad \Sigma=\begin{pmatrix}\Sigma_{11}&\Sigma_{12}\\\Sigma_{21}&\Sigma_{22}\end{pmatrix}, $$ where the first block has dimension $p_1$. If $W_{11}$ is invertible, the Schur complement of $W_{11}$ in $W$ is $$ W_{22\cdot 1}=W_{22}-W_{21}W_{11}^{-1}W_{12}. $$ The corresponding covariance Schur complement is $\Sigma_{22\cdot 1}=\Sigma_{22}-\Sigma_{21}\Sigma_{11}^{-1}\Sigma_{12}$.

Assume $\nu\geq p$. Then $W$ and $W_{11}$ are positive definite with probability one, so all inverses above exist. Positive definiteness of $\Sigma$ also implies that $\Sigma_{11}$ and $\Sigma_{22\cdot 1}$ are positive definite.

Projection Identity and Distribution

Arrange the sample vectors as columns of a $p\times \nu$ matrix $X$, and partition its rows as $X=(X_1^T,X_2^T)^T$, with $X_1$ of size $p_1\times \nu$. Then $W=XX^T$. Since $X_1$ has full row rank with probability one, define the projection onto its row space by $$ P=X_1^T(X_1X_1^T)^{-1}X_1. $$ This is a symmetric idempotent matrix of rank $p_1$. Direct multiplication gives the key identity.

Result: The Schur complement is the scatter of the part of $X_2$ orthogonal to the row space of $X_1$: $$ W_{22\cdot 1}=X_2(I-P)X_2^T. $$

Indeed, $X_2X_2^T=W_{22}$, $X_2X_1^T=W_{21}$, and $X_1X_1^T=W_{11}$. Substituting $P$ into the right-hand side gives $$ X_2X_2^T-X_2X_1^T(X_1X_1^T)^{-1}X_1X_2^T =W_{22}-W_{21}W_{11}^{-1}W_{12}. $$ Thus the Schur complement is a residual scatter matrix, not merely an algebraic block operation.

Theorem: Under the stated Gaussian assumptions, if $\nu\geq p$, then $$ W_{22\cdot 1}\sim W_{p_2}(\Sigma_{22\cdot 1},\nu-p_1). $$ Moreover, $W_{22\cdot 1}$ is independent of the pair $(W_{11},W_{12})$, and hence is independent of each of those blocks separately.

Proof. Set $B=\Sigma_{21}\Sigma_{11}^{-1}$ and $\Omega=\Sigma_{22\cdot 1}$. Gaussian regression gives the decomposition $X_2=BX_1+E$, where $E$ is a $p_2\times\nu$ matrix independent of $X_1$, with independent $N_{p_2}(0,\Omega)$ columns. To verify the covariance parameter, for a single observation the residual $X_2-BX_1$ has covariance $\Sigma_{22}-\Sigma_{21}\Sigma_{11}^{-1}\Sigma_{12}=\Omega$; its covariance with $X_1$ is zero. Joint normality therefore gives independence.

Because $X_1(I-P)=0$, the projection identity becomes $$ W_{22\cdot 1}=E(I-P)E^T. $$ Condition on $X_1$, so $P$ is fixed. Choose an orthogonal $\nu\times\nu$ matrix $Q$ that diagonalizes $P$, with $Q^TPQ$ having $p_1$ ones and $\nu-p_1$ zeros on its diagonal. The columns of $EQ$, conditional on $X_1$, are still independent $N_{p_2}(0,\Omega)$: multiplication by $Q$ mixes the column index orthogonally, preserving their joint Gaussian covariance. Consequently $E(I-P)E^T$ is the sum of outer products of the $\nu-p_1$ columns corresponding to the zero entries of $Q^TPQ$. Its conditional law is $W_{p_2}(\Omega,\nu-p_1)$, a law that does not depend on $X_1$. This proves the asserted Wishart distribution.

For the independence claim, conditional on $X_1$, the Gaussian matrices $EP$ and $E(I-P)$ are independent: their cross-covariance is zero because $P(I-P)=0$. The conditional law of $E(I-P)E^T$ is constant in $X_1$, so this Schur complement is independent jointly of $X_1$ and $EP$. More explicitly, for bounded measurable functions $g,h$, conditional independence gives $$ E[g(E(I-P)E^T)h(X_1,EP)\mid X_1] =E[g(E(I-P)E^T)\mid X_1]E[h(X_1,EP)\mid X_1]. $$ The first conditional expectation is a constant, since the conditional law is fixed. Taking expectations proves factorization of the joint expectation, and hence independence.

Finally, $W_{11}=X_1X_1^T$, while $$ W_{12}=X_1X_2^T=X_1X_1^TB^T+X_1E^T =X_1X_1^TB^T+X_1(EP)^T. $$ The last equality follows from $X_1(I-P)=0$. Thus both $W_{11}$ and $W_{12}$, jointly, are functions of $(X_1,EP)$. The Schur complement is independent of that pair, and therefore independent of $(W_{11},W_{12})$. $\square$

A Determinant Consequence

The same block operation gives a useful exact determinant identity. It requires only an invertible $W_{11}$, not Gaussianity.

Result: For a partitioned matrix $W$ with invertible $W_{11}$, $$ \det(W)=\det(W_{11})\det(W_{22\cdot 1}). $$

Derivation. Left-multiply $W$ by the block lower-triangular matrix $$ \begin{pmatrix}I&0\\-W_{21}W_{11}^{-1}&I\end{pmatrix}. $$ Its determinant is one, and the product is the block upper-triangular matrix $$ \begin{pmatrix}W_{11}&W_{12}\\0&W_{22}-W_{21}W_{11}^{-1}W_{12}\end{pmatrix}. $$ The determinant of a block triangular matrix is the product of the determinants of its diagonal blocks. Since the left multiplier has determinant one, this gives the stated identity. $\square$

For $\nu\geq p$, both factors are positive with probability one. In particular, the degrees of freedom of the Schur-complement Wishart matrix are $\nu-p_1$, the dimension of the orthogonal residual space.

Worked Example: Computing the Residual Wishart Parameters

Let $p_1=1$, $p_2=2$, and take $$ \Sigma= \begin{pmatrix} 2&1&-1\\ 1&3.5&0.5\\ -1&0.5&2.5 \end{pmatrix}, \qquad \nu=8. $$ The leading block is $\Sigma_{11}=2$, and $\Sigma_{21}=(1,-1)^T$. Compute the covariance Schur complement: $$ \Sigma_{22\cdot 1} = \begin{pmatrix}3.5&0.5\\0.5&2.5\end{pmatrix} -\frac{1}{2} \begin{pmatrix}1\\-1\end{pmatrix} \begin{pmatrix}1&-1\end{pmatrix} = \begin{pmatrix}3&1\\1&2\end{pmatrix}. $$ Its determinant is $3(2)-1(1)=5>0$, and its first diagonal entry is positive, so it is positive definite. The full $\Sigma$ is positive definite by the block criterion, since $\Sigma_{11}>0$ and $\Sigma_{22\cdot 1}$ is positive definite.

Here $\nu-p_1=8-1=7$. Therefore $$ W_{22\cdot 1}\sim W_2\left( \begin{pmatrix}3&1\\1&2\end{pmatrix},7\right). $$ Its expectation is $7\Sigma_{22\cdot 1}=\begin{pmatrix}21&7\\7&14\end{pmatrix}$. The theorem also says that this residual scatter is independent of the pair $(W_{11},W_{12})$, not just of $W_{11}$ alone.

Worked Example: Verifying the Projection Calculation

Consider the following particular two-row data matrix with three columns: $$ X_1=\begin{pmatrix}1&0&1\end{pmatrix}, \qquad X_2=\begin{pmatrix}2&1&3\end{pmatrix}. $$ This is a deterministic illustration of the block identity. The scatter blocks are $$ W_{11}=X_1X_1^T=2,\qquad W_{12}=X_1X_2^T=5,\qquad W_{22}=X_2X_2^T=14. $$ Thus the Schur complement is $$ W_{22\cdot 1}=14-5(2^{-1})5=14-\frac{25}{2}=\frac{3}{2}. $$

The projection onto the row space of $X_1$ is $$ P=X_1^T(2^{-1})X_1 =\frac12\begin{pmatrix}1&0&1\\0&0&0\\1&0&1\end{pmatrix}. $$ The projected row is $X_2P=(2.5,0,2.5)$, so the residual row is $X_2(I-P)=(-0.5,1,0.5)$. Its scatter is $$ (-0.5)^2+1^2+(0.5)^2=0.25+1+0.25=1.5=\frac32, $$ matching the block calculation exactly.

Worked Example: Independence and the Trailing Block

Let $\nu=4$, $p_1=p_2=1$, and $$ \Sigma=\begin{pmatrix}1&0.6\\0.6&1\end{pmatrix}. $$ The covariance Schur complement is $\Omega=1-(0.6)^2=0.64$. The result gives $S=W_{22\cdot 1}\sim W_1(0.64,3)$, independent jointly of $W_{11}$ and $W_{12}$. In the scalar case, $S$ is a sum of three independent squared $N(0,0.64)$ residual coordinates, so $$ E[S]=3(0.64)=1.92,\qquad \operatorname{Var}(S)=3\{3(0.64)^2-(0.64)^2\}=2.4576. $$ Here the variance calculation uses $E[Z^2]=v$ and $E[Z^4]=3v^2$ for $Z\sim N(0,v)$.

Although $S$ is independent of both leading blocks jointly, it is generally dependent on $W_{22}$. The block identity gives $W_{22}=S+R$, where $R=W_{21}^2/W_{11}$ is a function of $(W_{11},W_{12})$. Hence $R$ is independent of $S$. All these variables have finite second moments in this example, so $$ \operatorname{Cov}(S,W_{22}) =\operatorname{Cov}(S,S+R) =\operatorname{Var}(S)+\operatorname{Cov}(S,R) =2.4576>0. $$ Thus $S$ and $W_{22}$ are not independent. The trailing block includes both residual scatter and the scatter explained by the first block.

Common Mistake: Misstating the Independence

A tempting but incorrect statement is: “The Schur complement is independent of $W_{11}$, but generally dependent on $W_{12}$.” Under the Gaussian assumptions used here, the second clause is false. The projection proof establishes independence from $(W_{11},W_{12})$ jointly, and therefore from $W_{12}$ as well. The block for which dependence is generally present is $W_{22}$, because $W_{22}=W_{22\cdot 1}+W_{21}W_{11}^{-1}W_{12}$. Independence from the leading and cross blocks does not mean independence from every block of the original matrix.

The degrees-of-freedom adjustment is another point worth checking: the Schur complement uses $\nu-p_1$, not $\nu$. The projection removes a $p_1$-dimensional row space from a $\nu$-dimensional sample-coordinate space, leaving exactly $\nu-p_1$ residual directions.

Key takeaway: For a central Wishart matrix constructed from $\nu$ independent Gaussian vectors, the Schur complement of a $p_1$-dimensional leading block has Wishart scale $\Sigma_{22\cdot 1}$ and $\nu-p_1$ degrees of freedom. It is independent jointly of $W_{11}$ and $W_{12}$, while the trailing block $W_{22}$ generally contains additional dependent variation.

Check Your Understanding

Use the Gaussian projection representation to answer each question.

For a Wishart matrix with $\nu=10$ and a leading block of dimension $p_1=3$, how many degrees of freedom does the Schur complement have?
State the covariance scale matrix of the Schur complement in terms of the blocks of $\Sigma$.
Why does the Schur complement have the same conditional distribution for every full-rank value of $X_1$?
Which blocks are jointly independent of $W_{22\cdot 1}$ under the stated Gaussian assumptions?
Explain why independence from $W_{11}$ and $W_{12}$ does not imply independence from $W_{22}$.