Marginal Laws Of Wishart Submatrices

Why a Wishart Submatrix Has a Wishart Law

A Wishart matrix summarizes scatter across several coordinates. If an analysis keeps only some of those coordinates, it is natural to ask whether the retained scatter still has a familiar distribution. The key is that the retained block is formed from the same observations, with the same number of outer products; only the dimension of each Gaussian vector changes.

The answer is exact: a principal submatrix of a Wishart matrix is itself Wishart. Its scale matrix is the matching principal submatrix of the original scale matrix, and its degrees of freedom are unchanged. Both details matter. The degrees of freedom count the Gaussian vectors in the construction, not the number of coordinates retained.

Definition: Let $\Sigma$ be a $p\times p$ symmetric positive-definite matrix and let $\nu$ be a positive integer. If $X_1,\ldots,X_\nu$ are independent $N_p(0,\Sigma)$ random vectors, then $$ W=\sum_{r=1}^{\nu}X_rX_r^\top $$ has the Wishart distribution with $\nu$ degrees of freedom and scale matrix $\Sigma$, written $W\sim W_p(\nu,\Sigma)$. This construction-based definition is sufficient for the marginal result below.

For a set of coordinate indices $I=\{i_1,\ldots,i_q\}$, listed in increasing order, let $W_{II}$ denote the $q\times q$ submatrix using those indices for both its rows and columns. Such a block is a principal submatrix. Write $\Sigma_{II}$ for the corresponding principal submatrix of $\Sigma$. A principal submatrix of a positive-definite matrix is positive definite, so it can serve as a Wishart scale matrix.

The Principal-Submatrix Result

Theorem: If $W\sim W_p(\nu,\Sigma)$ under the Gaussian construction above, and $I$ contains $q$ coordinate indices, then $$ W_{II}\sim W_q(\nu,\Sigma_{II}). $$ The degrees of freedom remain $\nu$; the dimension changes from $p$ to $q$; and the scale matrix is restricted to the same coordinates.

The reason is visible directly from the construction. Let $A$ be the $q\times p$ coordinate-selection matrix that retains coordinates in $I$. Then $AX_r$ is the retained part of $X_r$, and $$ A W A^\top =A\left(\sum_{r=1}^{\nu}X_rX_r^\top\right)A^\top =\sum_{r=1}^{\nu}(AX_r)(AX_r)^\top. $$ For each $r$, the characteristic function of $AX_r$, evaluated at $t\in\mathbb{R}^q$, is $$ E\!\left[\exp\{i t^\top AX_r\}\right] =\exp\!\left\{-\frac12 t^\top A\Sigma A^\top t\right\}. $$ This is the characteristic function of $N_q(0,A\Sigma A^\top)$. The vectors $AX_1,\ldots,AX_\nu$ remain independent because each is a function of a different independent vector. Thus the displayed sum is a sum of $\nu$ independent outer products of $N_q(0,A\Sigma A^\top)$ vectors. By the definition of the Wishart law, it has distribution $W_q(\nu,A\Sigma A^\top)$. For this coordinate selector, $AWA^\top=W_{II}$ and $A\Sigma A^\top=\Sigma_{II}$, proving the theorem. $\square$

Worked Example: Keeping Coordinates One and Three

Suppose $W\sim W_3(8,\Sigma)$, where $$ \Sigma= \begin{pmatrix} 4&1&2\\ 1&3&-1\\ 2&-1&5 \end{pmatrix}. $$ The scale matrix is positive definite: its leading principal minors are $4$, $4(3)-1=11$, and $\det(\Sigma)=35$, all positive.

Retain coordinates $I=\{1,3\}$. The corresponding principal scale submatrix is $$ \Sigma_{II}= \begin{pmatrix} 4&2\\ 2&5 \end{pmatrix}. $$ Therefore the retained block has law $$ W_{II}\sim W_2\!\left(8, \begin{pmatrix} 4&2\\ 2&5 \end{pmatrix}\right). $$ The dimension has fallen from three to two, but the degrees of freedom are still eight: the original construction used eight Gaussian vectors, and retaining two coordinates does not remove any of those vectors.

The scale submatrix is selected using the same indices on both sides. It retains the variances of the chosen coordinates on the diagonal and their covariances off the diagonal. It is not obtained by taking a submatrix of $\Sigma^{-1}$, nor by inverting a submatrix of $\Sigma$.

Nested Coordinate Sets

The theorem also makes successive marginalization consistent. If $J\subseteq I\subseteq\{1,\ldots,p\}$, first retaining $I$ and then retaining $J$ gives the same law as retaining $J$ directly. In either route, the final scale is $\Sigma_{JJ}$ and the degrees of freedom are $\nu$.

Corollary: For nested coordinate sets $J\subseteq I$, the $J$-indexed principal submatrix of $W_{II}$ has distribution $W_{|J|}(\nu,\Sigma_{JJ})$, the same distribution as $W_{JJ}$.

To verify this, apply the theorem first to $I$, giving $W_{II}\sim W_{|I|}(\nu,\Sigma_{II})$. Apply it again to the coordinates $J$ within that matrix. The resulting scale matrix is the principal submatrix of $\Sigma_{II}$ indexed by $J$, which is exactly $\Sigma_{JJ}$; the degrees of freedom stay $\nu$. Applying the theorem directly to $J$ gives that same law for $W_{JJ}$. Hence the two routes agree. $\square$

Worked Example: A Centered Gaussian Sample

Suppose $m=6$ independent observations are sampled from $N_3(\mu,\Sigma)$, with $$ \Sigma= \begin{pmatrix} 2&0.5&0.2\\ 0.5&1&-0.1\\ 0.2&-0.1&3 \end{pmatrix}. $$ This scale matrix is positive definite: its leading principal minors are $2$, $2(1)-0.5^2=1.75$, and $\det(\Sigma)=5.17$, all positive. Let $\bar X$ be the sample mean and let $C=\sum_{r=1}^{6}(X_r-\bar X)(X_r-\bar X)^\top$ be the centered scatter matrix.

For independent multivariate normal observations, the centered scatter has distribution $C\sim W_3(m-1,\Sigma)=W_3(5,\Sigma)$. If the analysis retains variables one and two, the corresponding scatter block is $$ C_{\{1,2\},\{1,2\}}\sim W_2\!\left(5, \begin{pmatrix} 2&0.5\\ 0.5&1 \end{pmatrix}\right). $$ In particular, centering gives five degrees of freedom, and reducing from three variables to two does not change that count. If the unbiased sample covariance matrix is reported instead, its retained block is $C_{\{1,2\},\{1,2\}}/5$; that is a scaled Wishart matrix, not the unscaled Wishart matrix in the displayed law.

Checking the Result on an Explicit Scatter Matrix

The theorem concerns a distribution, but the same submatrix operation is also apparent in individual realizations. The next calculation checks that retaining coordinates commutes with forming the sum of outer products.

Worked Example: Computing a Retained Block Directly

Take three vectors in $\mathbb{R}^3$: $x_1=(1,0,2)^\top$, $x_2=(0,1,-1)^\top$, and $x_3=(2,1,0)^\top$. Their outer products sum to $$ W=\sum_{r=1}^{3}x_rx_r^\top = \begin{pmatrix} 5&2&2\\ 2&2&-1\\ 2&-1&5 \end{pmatrix}. $$ For example, the diagonal entries are $1^2+0^2+2^2=5$, $0^2+1^2+1^2=2$, and $2^2+(-1)^2+0^2=5$; the $(1,2)$, $(1,3)$, and $(2,3)$ entries are respectively $1(0)+0(1)+2(1)=2$, $1(2)+0(-1)+2(0)=2$, and $0(2)+1(-1)+1(0)=-1$.

Retaining coordinates one and three gives $$ W_{\{1,3\},\{1,3\}}= \begin{pmatrix} 5&2\\ 2&5 \end{pmatrix}. $$ The same result comes from first reducing each vector: the retained vectors are $(1,2)^\top$, $(0,-1)^\top$, and $(2,0)^\top$. Their outer products sum to $$ \begin{pmatrix}1&2\\2&4\end{pmatrix} + \begin{pmatrix}0&0\\0&1\end{pmatrix} + \begin{pmatrix}4&0\\0&0\end{pmatrix} = \begin{pmatrix}5&2\\2&5\end{pmatrix}. $$ If the original vectors had been independent $N_3(0,\Sigma)$ vectors, the retained vectors would be independent $N_2(0,\Sigma_{\{1,3\},\{1,3\}})$ vectors, and this construction would give the theorem's Wishart law with three degrees of freedom.

Marginal Is Not Conditional

The principal-submatrix theorem is a statement about a marginal block: it describes the scatter in a subset of coordinates without conditioning on the others. The block is made by deleting the unwanted rows and columns of the realized matrix. Its law uses the matching principal submatrix of the scale matrix.

A common mistake is to replace that scale submatrix with a conditional covariance formula, or to suppose that the retained block must be adjusted because other coordinates were omitted. That confuses a marginal distribution with a conditional one. For a Gaussian vector, the marginal covariance of selected coordinates is the corresponding principal submatrix of $\Sigma$. The conditional covariance given other coordinates is a different quantity. Since this result concerns a marginal block, $\Sigma_{II}$, not a conditional covariance, is the correct scale.

Another frequent error is to change $\nu$ when reducing the dimension. The Wishart construction sums $\nu$ outer products. Coordinate selection changes the size of each vector and each outer product, but there are still exactly $\nu$ terms. For centered sample scatter, the relevant count is already $m-1$; it remains $m-1$ after selecting variables.

Key takeaway: To find the marginal law of a principal block, keep the same coordinate indices in rows and columns of the scale matrix. The resulting block is Wishart in the reduced dimension, with the original degrees of freedom unchanged.

Check Your Understanding

Use the construction and theorem to answer these questions.

If $W\sim W_5(12,\Sigma)$ and coordinates $2,4,5$ are retained, what are the dimension, degrees of freedom, and scale matrix of the resulting Wishart law?
Why do the degrees of freedom stay the same when a principal submatrix is taken?
For centered scatter from $m$ independent multivariate normal observations, what are the degrees of freedom of the retained principal block?
Explain why the scale matrix for a marginal principal block is not obtained by inverting $\Sigma$ and extracting the corresponding block.
If $J\subseteq I$, what scale matrix results from retaining $I$ and then $J$, and why does it match direct retention of $J$?