Additivity Of Independent Wishart Matrices

Why Additivity Matters

In multivariate sampling, separate groups or batches can each produce a scatter matrix. If those matrices are independent and share the same scale matrix, their sum has a particularly simple distribution: it is Wishart again, with degrees of freedom added. This rule explains why within-group scatter can be pooled without changing the form of its distribution.

The condition on the scale is essential. Independent matrices with different scales do not generally combine into one Wishart distribution. We will establish the additivity rule from the Gaussian construction, derive a transform that provides a second check, and apply the rule to pooled-scatter calculations.

Wishart Matrices and the Additivity Theorem

Definition: Let $\Sigma$ be a $p\times p$ positive-definite matrix, and let $X_1,\ldots,X_\nu$ be independent random vectors with distribution $N_p(0,\Sigma)$, where $\nu$ is a positive integer. The matrix $$ W=\sum_{i=1}^{\nu}X_iX_i^T $$ has a Wishart distribution with $\nu$ degrees of freedom and scale matrix $\Sigma$, written $W\sim W_p(\nu,\Sigma)$. The sum is called a scatter matrix because each term is an outer product.

Here, $X_i$ is a column vector, so $X_iX_i^T$ is a $p\times p$ matrix. The integer-$\nu$ construction is the sampling version of the Wishart distribution used here. It makes the role of degrees of freedom explicit: they count independent Gaussian vectors contributing outer products.

Theorem (Additivity of independent Wishart matrices): Suppose $W_1\sim W_p(\nu_1,\Sigma)$ and $W_2\sim W_p(\nu_2,\Sigma)$ are independent, with positive integer degrees of freedom and the same positive-definite scale matrix $\Sigma$. Then $$ W_1+W_2\sim W_p(\nu_1+\nu_2,\Sigma). $$

Proof. By the Gaussian construction, write $$ W_1=\sum_{i=1}^{\nu_1}X_iX_i^T,\qquad W_2=\sum_{j=1}^{\nu_2}Y_jY_j^T, $$ where the $X_i$ are independent $N_p(0,\Sigma)$ vectors, the $Y_j$ are independent $N_p(0,\Sigma)$ vectors, and the two collections are independent of each other. Consequently, all $\nu_1+\nu_2$ vectors in the combined collection are independent and have the same $N_p(0,\Sigma)$ distribution. Therefore $$ W_1+W_2 =\sum_{i=1}^{\nu_1}X_iX_i^T+\sum_{j=1}^{\nu_2}Y_jY_j^T $$ is the sum of outer products of $\nu_1+\nu_2$ independent $N_p(0,\Sigma)$ vectors. By the definition, it has distribution $W_p(\nu_1+\nu_2,\Sigma)$. $\square$

Worked Example: Adding Two Wishart Matrices

Let $W_1\sim W_2(4,\Sigma)$ and $W_2\sim W_2(7,\Sigma)$ be independent, with $$ \Sigma=\begin{pmatrix}2&1\\1&3\end{pmatrix}. $$ Find the distribution of their sum.

First check that the common scale is positive definite: its first leading principal minor is $2>0$, and its determinant is $2(3)-1(1)=5>0$. Thus it is a valid scale matrix. The matrices are independent, have the same dimension and scale, and have degrees of freedom 4 and 7. Additivity gives $$ W_1+W_2\sim W_2(4+7,\Sigma)=W_2(11,\Sigma). $$ The scale matrix remains $\Sigma$; it is the degrees of freedom, not the scale, that are added.

A Laplace-Transform Derivation

The Gaussian construction proves additivity directly. A transform calculation makes the shared-scale requirement visible in another way. For a symmetric positive-semidefinite $p\times p$ matrix $T$, the Laplace transform of $W$ is $$ E\!\left[\exp\!\left\{-\tfrac12\operatorname{tr}(TW)\right\}\right] =|I+\Sigma T|^{-\nu/2}, $$ where $|A|$ denotes the determinant of $A$, $I$ is the identity matrix, and $\operatorname{tr}(A)$ is the trace.

Derivation. For a single vector $X\sim N_p(0,\Sigma)$, the exponent satisfies $\operatorname{tr}(TXX^T)=X^TTX$. Multiplying the Gaussian density by $\exp(-X^TTX/2)$ changes the quadratic form in the exponent from $X^T\Sigma^{-1}X$ to $X^T(\Sigma^{-1}+T)X$. Since $T$ is positive semidefinite and $\Sigma^{-1}$ is positive definite, $\Sigma^{-1}+T$ is positive definite. The Gaussian integral therefore gives $$ E\!\left[\exp\!\left\{-\tfrac12X^TTX\right\}\right] =|\Sigma|^{-1/2}|\Sigma^{-1}+T|^{-1/2} =|I+\Sigma T|^{-1/2}. $$ The last equality follows from $|\Sigma^{-1}+T|=|\Sigma^{-1}|\,|I+\Sigma T|$ and $|\Sigma^{-1}|=|\Sigma|^{-1}$. For $W=\sum_{i=1}^{\nu}X_iX_i^T$, independence makes the transform a product of $\nu$ identical factors, yielding $|I+\Sigma T|^{-\nu/2}$. $\square$

For independent $W_1$ and $W_2$ with common scale, the transform of their sum is the product $$ |I+\Sigma T|^{-\nu_1/2}|I+\Sigma T|^{-\nu_2/2} =|I+\Sigma T|^{-(\nu_1+\nu_2)/2}. $$ This is the transform corresponding to the summed degrees of freedom. The direct construction above establishes the distributional result; the transform calculation confirms how the exponents combine.

Pooling Within-Group Scatter

A common application arises when independent groups are sampled from the same multivariate normal population. Let group $g$ contain $n_g\geq 2$ independent observations from $N_p(\mu,\Sigma)$, and define its centered scatter by $$ C_g=\sum_{i=1}^{n_g}(X_{gi}-\bar X_g)(X_{gi}-\bar X_g)^T,\qquad \bar X_g=\frac{1}{n_g}\sum_{i=1}^{n_g}X_{gi}. $$ Under this normal model, the Gaussian projection result for sample scatter gives $C_g\sim W_p(n_g-1,\Sigma)$. Different groups are independent, so their centered scatter matrices are independent. Additivity then yields $$ \sum_{g=1}^{G}C_g\sim W_p\!\left(\sum_{g=1}^{G}(n_g-1),\Sigma\right). $$ This is the distribution of pooled within-group scatter. It does not assert that every kind of scatter formed from the same observations has this distribution.

Worked Example: Pooling Scatter From Two Clinical Groups

Suppose two independent treatment groups each consist of observations from the same $N_2(\mu,\Sigma)$ population. The first group has $n_1=6$ patients and the second has $n_2=9$. Let $C_1$ and $C_2$ be their centered scatter matrices. Find the distribution of the pooled within-group scatter and its degrees-of-freedom denominator.

The first centered scatter has $6-1=5$ degrees of freedom, and the second has $9-1=8$. Both have scale $\Sigma$, and they are independent because they are computed from independent groups. Thus $$ C_1+C_2\sim W_2(5+8,\Sigma)=W_2(13,\Sigma). $$ The total sample size is $6+9=15$, and the pooled within-group degrees of freedom are $15-2=13$, the same count obtained by adding the group-specific degrees of freedom. If the pooled within-group covariance estimate is defined as $(C_1+C_2)/13$, its denominator is therefore 13. This calculation concerns within-group scatter; it does not include differences between the two group means.

Within-Group Scatter Is Not Grand-Mean Scatter

The distinction between within-group and grand-mean scatter is consequential. For two groups, let $\bar X$ be the mean of all $n_1+n_2$ observations. Expanding each observation around its group mean gives the exact identity $$ \sum_{g=1}^{2}\sum_{i=1}^{n_g}(X_{gi}-\bar X)(X_{gi}-\bar X)^T =C_1+C_2+\sum_{g=1}^{2}n_g(\bar X_g-\bar X)(\bar X_g-\bar X)^T. $$ To see this, write $X_{gi}-\bar X=(X_{gi}-\bar X_g)+(\bar X_g-\bar X)$ and expand the outer product. The cross terms sum to zero within each group because $\sum_i(X_{gi}-\bar X_g)=0$. The remaining mean-difference terms are exactly the displayed final sum.

The final term is between-group scatter. It is generally nonzero, so the total scatter about the grand mean is not simply $C_1+C_2$. In the two-group case the between-group term simplifies to $$ \frac{n_1n_2}{n_1+n_2}(\bar X_1-\bar X_2)(\bar X_1-\bar X_2)^T. $$ It vanishes only when the two observed group means are equal. Additivity applies to the independent Wishart within-group matrices; it does not license dropping or absorbing the between-group term.

Worked Example: Three Independent Batches

Three independent batches each produce a scatter matrix with distribution $W_3(4,\Sigma)$, for the same positive-definite $3\times3$ scale matrix $\Sigma$. Find the law of the combined scatter and compare its degrees of freedom with the total number of Gaussian vectors in the construction.

The three matrices are independent and share dimension and scale. Applying additivity twice gives $$ W_1+W_2\sim W_3(4+4,\Sigma)=W_3(8,\Sigma), $$ and then $$ W_1+W_2+W_3\sim W_3(8+4,\Sigma)=W_3(12,\Sigma). $$ Equivalently, each batch contributes four independent $N_3(0,\Sigma)$ vectors, so the combined construction has $4+4+4=12$ such vectors. Both calculations give 12 degrees of freedom. The order of pooling does not affect the result.

Why the Common Scale Condition Cannot Be Dropped

If independent Wishart matrices have different scale matrices, the common-scale theorem does not apply. A one-dimensional example makes the issue explicit. Under the Gaussian construction, $W_1(1,1)$ is the square of a standard normal variable, while an independent $W_1(1,4)$ is four times the square of another standard normal variable. Their sum is a weighted sum of two independent squared standard normals, not a standard Wishart sum with a common scale.

Indeed, for $t\geq 0$, its Laplace transform is $$ (1+2t)^{-1/2}(1+8t)^{-1/2}. $$ A one-dimensional Wishart distribution $W_1(\nu,\sigma^2)$ has transform $(1+2\sigma^2t)^{-\nu/2}$. The first expression has two distinct factors, with different zero locations after extending the expressions as functions of $t$; the second has only one factor location. They cannot agree for all $t$. Thus independence alone is insufficient: the shared scale is part of the additivity theorem's hypothesis.

Common Mistakes

Adding scale matrices as well as degrees of freedom. In the theorem, the scale is the same for both summands and remains unchanged; only the degrees of freedom add.
Assuming independence is enough. Independent Wishart matrices with unequal scales do not generally have a Wishart sum.
Using sample sizes instead of centered-scatter degrees of freedom. A group of $n_g$ observations contributes $n_g-1$ degrees of freedom to its centered scatter, not $n_g$.
Confusing pooled within-group scatter with total scatter. The total scatter about the grand mean includes a between-group term in addition to the within-group sum.

Key takeaway: Independent Wishart matrices add to a Wishart matrix when they have the same dimension and scale; their degrees of freedom add. In pooled sampling problems, first identify exactly which scatter matrices are being summed and verify their independence and common scale.

Check Your Understanding

Use the construction and conditions above to answer the following questions.

If independent $W_4(6,\Sigma)$ and $W_4(9,\Sigma)$ matrices share the same scale, what is the distribution of their sum?
Why does a group of $n$ observations contribute $n-1$, rather than $n$, degrees of freedom to its centered scatter?
Two independent Wishart matrices have different scale matrices. Which hypothesis of the additivity theorem fails, and what conclusion can you no longer draw?
For two groups, name the additional term in the decomposition of grand-mean scatter beyond the sum of within-group scatters.
Three independent groups have sizes 5, 8, and 10 and share a common multivariate normal distribution. What are the degrees of freedom of their pooled within-group scatter?