Expected Wishart Matrices And Trace Moments

Why Wishart Expectations Matter

A Wishart matrix summarizes multivariate scatter. Its entries describe variation and co-variation across coordinates, while its trace gives a single measure of total scatter across all coordinates. Expectations answer two practical questions: what matrix does the random scatter estimate on average, and how large is its total scatter on average?

The key calculations follow directly from the Gaussian sampling representation. They also clarify an important distinction: the expected trace, the expected squared trace, and the expected trace of the squared matrix are different quantities. We derive each from the same representation.

Definition and Sampling Representation

Definition: Let $Z_1,\ldots,Z_\nu$ be independent $p$-dimensional Gaussian random vectors with distribution $N_p(0,\Sigma)$, where $\nu$ is a positive integer and $\Sigma$ is a $p\times p$ positive semidefinite matrix. The random matrix $$ W=\sum_{r=1}^{\nu} Z_r Z_r^{\mathsf T} $$ has a Wishart distribution with scale matrix $\Sigma$ and $\nu$ degrees of freedom, written $W\sim W_p(\Sigma,\nu)$. The trace $\operatorname{tr}(A)$ of a square matrix $A$ is the sum of its diagonal entries; $A^2$ means the matrix product $AA$.

The representation also gives an immediate interpretation: $W$ is a sum of $\nu$ Gaussian outer products. If the $Z_r$ are observations centered at a known mean, this sum is their sample scatter matrix. More generally, whenever a Gaussian sample-scatter construction has this Wishart law, the formulas below apply with its degrees of freedom $\nu$.

Worked Example: Mean Matrix and Trace for a Diagonal Scale

Suppose $W\sim W_2(\Sigma,5)$, where $$ \Sigma=\begin{pmatrix}4&0\\0&1\end{pmatrix}. $$ Find the expected matrix and expected trace.

For each $r$, $Z_r$ has mean zero and covariance $\Sigma$. Thus $E[Z_rZ_r^{\mathsf T}]=\Sigma$. Summing five such expectations gives $$ E[W]=5\Sigma =\begin{pmatrix}20&0\\0&5\end{pmatrix}. $$ The expected trace is therefore $20+5=25$. Equivalently, each outer product contributes an expected trace of $\operatorname{tr}(\Sigma)=4+1=5$, so five contributions give $5(5)=25$.

In a known-center Gaussian sample-scatter interpretation, the average scatter in the first coordinate is larger because its variance is $4$, compared with variance $1$ in the second coordinate. The total expected scatter is the sum of the coordinate variances, accumulated over the five Gaussian vectors.

Expected Wishart Matrix and Expected Trace

Theorem: If $W\sim W_p(\Sigma,\nu)$ under the sampling representation above, then $$ E[W]=\nu\Sigma \qquad\text{and}\qquad E[\operatorname{tr}(W)]=\nu\operatorname{tr}(\Sigma). $$

The expectation of a random matrix is defined entry by entry, provided the entries have finite expectations. Here those expectations exist because Gaussian coordinates have finite second moments.

Proof. For any indices $a,b\in\{1,\ldots,p\}$, the $(a,b)$ entry of $Z_rZ_r^{\mathsf T}$ is $Z_{r,a}Z_{r,b}$. By the definition of covariance and the zero mean, $E[Z_{r,a}Z_{r,b}]=\Sigma_{ab}$. Finite additivity of expectation therefore gives $$ E[W_{ab}] =E\left[\sum_{r=1}^{\nu}Z_{r,a}Z_{r,b}\right] =\sum_{r=1}^{\nu}E[Z_{r,a}Z_{r,b}] =\nu\Sigma_{ab}. $$ This holds for every pair of indices, proving $E[W]=\nu\Sigma$. Since the trace is a finite sum of diagonal entries, $$ E[\operatorname{tr}(W)] =\sum_{a=1}^{p}E[W_{aa}] =\nu\sum_{a=1}^{p}\Sigma_{aa} =\nu\operatorname{tr}(\Sigma). $$ $\square$

This result is the matrix version of adding the expected contribution from each Gaussian vector. The trace identity is also consistent with linearity: $\operatorname{tr}(E[W])=\nu\operatorname{tr}(\Sigma)$. No independence among the entries of $W$ is required for either calculation.

Worked Example: A Correlated Bivariate Scale

Let $W\sim W_2(\Sigma,4)$ with $$ \Sigma=\begin{pmatrix}2&1\\1&3\end{pmatrix}. $$ Calculate its expected matrix and expected trace.

Multiplying the scale matrix by the degrees of freedom gives $$ E[W]=4\begin{pmatrix}2&1\\1&3\end{pmatrix} =\begin{pmatrix}8&4\\4&12\end{pmatrix}. $$ The diagonal entries have expectations $8$ and $12$, so $E[\operatorname{tr}(W)]=8+12=20$. Directly, $\operatorname{tr}(\Sigma)=2+3=5$, and $\nu\operatorname{tr}(\Sigma)=4(5)=20$, as required.

The off-diagonal expectation is $4$, reflecting the positive covariance in $\Sigma$. It contributes to the expected matrix, but not directly to the trace, which sums only diagonal entries.

Second-Order Trace Moments

The expected trace alone describes the average total scatter. A second moment describes how that scalar total varies, while $\operatorname{tr}(W^2)$ measures a different aspect of the matrix. The following formulas require only Gaussian moments and the sampling representation.

Theorem: For $W\sim W_p(\Sigma,\nu)$, $$ E\left[(\operatorname{tr}(W))^2\right] =\nu^2\bigl(\operatorname{tr}(\Sigma)\bigr)^2 +2\nu\operatorname{tr}(\Sigma^2), $$ and $$ E[\operatorname{tr}(W^2)] =\nu\bigl(\operatorname{tr}(\Sigma)\bigr)^2 +\nu(\nu+1)\operatorname{tr}(\Sigma^2). $$ Consequently, $\operatorname{Var}(\operatorname{tr}(W))=2\nu\operatorname{tr}(\Sigma^2)$.

Proof. Write $Q_r=Z_r^{\mathsf T}Z_r$, so that $\operatorname{tr}(W)=\sum_r Q_r$. To calculate the moments of $Q_r$, diagonalize $\Sigma$: its eigenvalues are $\lambda_1,\ldots,\lambda_p$, all nonnegative. Rotational invariance of a standard Gaussian vector gives $Q_r$ the same distribution as $\sum_{a=1}^{p}\lambda_a G_a^2$, where the $G_a$ are independent $N(0,1)$ random variables. Using $E[G_a^2]=1$, $E[G_a^4]=3$, and independence, $$ E[Q_r]=\sum_a\lambda_a=\operatorname{tr}(\Sigma), \qquad \operatorname{Var}(Q_r)=2\sum_a\lambda_a^2=2\operatorname{tr}(\Sigma^2). $$ The $Q_r$ are independent because the $Z_r$ are independent. Hence their sum has mean $\nu\operatorname{tr}(\Sigma)$ and variance $2\nu\operatorname{tr}(\Sigma^2)$. Adding the square of this mean to the variance proves the stated formula for $E[(\operatorname{tr}(W))^2]$.

For the other formula, expand the matrix square and take traces: $$ \operatorname{tr}(W^2)=\sum_{r=1}^{\nu}\sum_{s=1}^{\nu}(Z_r^{\mathsf T}Z_s)^2. $$ When $r=s$, the expectation of the term is $E[Q_r^2]=(\operatorname{tr}(\Sigma))^2+2\operatorname{tr}(\Sigma^2)$, by the variance and mean just calculated. When $r\ne s$, independence and zero means give $$ E[(Z_r^{\mathsf T}Z_s)^2] =\sum_{a=1}^{p}\sum_{b=1}^{p}E[Z_{r,a}Z_{r,b}]E[Z_{s,a}Z_{s,b}] =\sum_{a,b}\Sigma_{ab}^2 =\operatorname{tr}(\Sigma^2), $$ where $\Sigma$ is symmetric. There are $\nu$ diagonal terms and $\nu(\nu-1)$ off-diagonal terms in the double sum. Therefore $$ E[\operatorname{tr}(W^2)] =\nu\left((\operatorname{tr}(\Sigma))^2+2\operatorname{tr}(\Sigma^2)\right) +\nu(\nu-1)\operatorname{tr}(\Sigma^2) =\nu(\operatorname{tr}(\Sigma))^2+\nu(\nu+1)\operatorname{tr}(\Sigma^2). $$ This proves the second formula. The variance identity follows by subtracting $(E[\operatorname{tr}(W)])^2$ from the second raw moment. $\square$

Worked Example: Comparing Two Trace Moments

For the bivariate scale and four degrees of freedom used above, calculate the second moment of the trace, its variance, and the expected trace of the matrix square.

First, $\operatorname{tr}(\Sigma)=5$. Direct multiplication gives $$ \Sigma^2=\begin{pmatrix}5&5\\5&10\end{pmatrix}, \qquad \operatorname{tr}(\Sigma^2)=15. $$ Thus $$ E[(\operatorname{tr}(W))^2]=4^2(5^2)+2(4)(15)=400+120=520, $$ and $$ \operatorname{Var}(\operatorname{tr}(W))=2(4)(15)=120. $$ As a check, the squared mean trace is $20^2=400$, and $400+120=520$.

For the matrix-square trace, the formula instead gives $$ E[\operatorname{tr}(W^2)] =4(5^2)+4(4+1)(15) =100+300 =400. $$ The two second-order quantities are not equal: $E[(\operatorname{tr}(W))^2]=520$, whereas $E[\operatorname{tr}(W^2)]=400$. They summarize different functions of the random matrix.

Interpreting the Formulas for Sample Scatter

Suppose $X_1,\ldots,X_\nu$ are independent observations from $N_p(\mu,\Sigma)$, with the center $\mu$ treated as known, and define their scatter about that center by $$ W=\sum_{r=1}^{\nu}(X_r-\mu)(X_r-\mu)^{\mathsf T}. $$ Then $Z_r=X_r-\mu$ meets the sampling representation, so $W\sim W_p(\Sigma,\nu)$. The expected scatter is $\nu\Sigma$, and the expected total squared deviation across coordinates is $E[\operatorname{tr}(W)]=\nu\operatorname{tr}(\Sigma)$. Dividing the scatter by $\nu$ therefore gives a matrix whose expectation is $\Sigma$.

For example, with $p=3$, $\nu=8$, and $\Sigma=\operatorname{diag}(1,2,4)$, the expected scatter is $\operatorname{diag}(8,16,32)$ and the expected trace is $56$. The scatter divided by eight has expectation $\Sigma$, and its expected trace is $7$. The degrees of freedom in the Wishart representation determine the multiplier; they should be taken from the sampling construction rather than assumed to equal a raw observation count in every setting.

Worked Example: Total Scatter Under a Known Center

In the three-dimensional setting just described, calculate both second-order trace summaries for $W$. Here $\operatorname{tr}(\Sigma)=1+2+4=7$, while $\operatorname{tr}(\Sigma^2)=1+4+16=21$. Therefore $$ E[(\operatorname{tr}(W))^2]=8^2(7^2)+2(8)(21)=3136+336=3472, $$ and the variance of the total scatter is $336$.

The expected trace of the squared scatter matrix is $$ E[\operatorname{tr}(W^2)] =8(7^2)+8(8+1)(21) =392+1512 =1904. $$ These values describe different summaries: $3472$ is the second raw moment of the total scatter, whereas $1904$ is the expected trace after squaring the matrix. Neither should be substituted for the other.

Common Pitfall

A frequent mistake is to confuse $(\operatorname{tr}(W))^2$ with $\operatorname{tr}(W^2)$. The first squares a single scalar after summing the diagonal; the second multiplies the matrix by itself and then sums its diagonal. For the bivariate example, the expected values are $520$ and $400$, respectively, so treating the expressions as interchangeable produces an incorrect result.

Another tempting shortcut is to report $E[W]=\Sigma$. That omits the degrees-of-freedom multiplier: for the Wishart matrix defined here, $E[W]=\nu\Sigma$. The expectation equals $\Sigma$ only after scaling $W$ by $\nu$. For sample scatter, check whether the quantity of interest is the unscaled scatter matrix or a scaled covariance estimate.

Key takeaway: Under the Gaussian sampling representation, expected scatter is $\nu\Sigma$, and expected total scatter is $\nu\operatorname{tr}(\Sigma)$. Second-order trace calculations depend on $\operatorname{tr}(\Sigma^2)$ as well as $\operatorname{tr}(\Sigma)$; keep the two meanings of “squared trace” distinct.

Check Your Understanding

Use the sampling representation and the formulas derived above.

If $W\sim W_3(\Sigma,6)$, what are $E[W]$ and $E[\operatorname{tr}(W)]$ in terms of $\Sigma$?
For $\Sigma=\operatorname{diag}(1,3)$ and $\nu=2$, calculate $E[\operatorname{tr}(W)]$.
Which quantity is the variance of $\operatorname{tr}(W)$, and how does it depend on $\operatorname{tr}(\Sigma^2)$?
Explain why $E[(\operatorname{tr}(W))^2]$ and $E[\operatorname{tr}(W^2)]$ need not agree.
For known-center Gaussian sample scatter, what happens to its expected matrix when the scatter is divided by its degrees of freedom?