Trace And Eigenvalue Checks For Wishart Samples

Why Check a Simulated Wishart Matrix?

A simulation can return a symmetric matrix without returning a correct Wishart draw. The matrix may have the wrong overall scale, an impossible negative eigenvalue, or a rank that contradicts the degrees of freedom. Trace and eigenvalue checks are inexpensive diagnostics: they do not establish that a simulation algorithm is correct, but they can expose many errors before the matrix is used in an analysis.

Throughout this tutorial, use the scale convention $W\sim W_p(\nu,\Sigma)$: for integer $\nu$, $W$ is the sum of $\nu$ independent outer products of $p$-dimensional $N_p(0,\Sigma)$ vectors. Assume $\Sigma$ is positive definite unless stated otherwise. This convention fixes the expected scale at $E[W]=\nu\Sigma$. The trace is the sum of diagonal entries, and the eigenvalues of a symmetric matrix are real.

Definition: A matrix $W$ is positive semidefinite if $v^\mathsf{T}Wv\geq 0$ for every vector $v$, and positive definite if $v^\mathsf{T}Wv>0$ for every nonzero $v$. For a real symmetric matrix, positive semidefiniteness is equivalent to all eigenvalues being nonnegative; positive definiteness is equivalent to all eigenvalues being positive.

Trace as a Scale Check

Write $W=\sum_{i=1}^{\nu}X_iX_i^\mathsf{T}$, where the $X_i$ are independent $N_p(0,\Sigma)$ vectors. The trace of an outer product is the squared length of its vector, so $\operatorname{tr}(W)=\sum_i X_i^\mathsf{T}X_i$. This gives both an intuitive scale check and, by diagonalizing $\Sigma$, an exact distributional result.

Theorem: Let $\Sigma$ be positive definite with eigenvalues $s_1,\ldots,s_p$, and let $W\sim W_p(\nu,\Sigma)$ for integer $\nu$. Then $$ \operatorname{tr}(W)\ \overset{d}{=}\ \sum_{j=1}^{p}s_j Q_j,\qquad Q_1,\ldots,Q_p\ \text{independent with }Q_j\sim\chi^2_\nu. $$ Consequently, $$ E[\operatorname{tr}(W)]=\nu\operatorname{tr}(\Sigma),\qquad \operatorname{Var}(\operatorname{tr}(W))=2\nu\operatorname{tr}(\Sigma^2). $$ If $\Sigma=\sigma^2 I_p$, then $\operatorname{tr}(W)/\sigma^2\sim\chi^2_{p\nu}$.

Proof. Let $U$ be an orthogonal matrix such that $\Sigma=U\operatorname{diag}(s_1,\ldots,s_p)U^\mathsf{T}$. Each vector $X_i$ has the same distribution as $U\operatorname{diag}(\sqrt{s_1},\ldots,\sqrt{s_p})Z_i$, where the entries of $Z_i$ are independent standard normal variables. Orthogonal transformations preserve squared length, so $$ X_i^\mathsf{T}X_i\ \overset{d}{=}\ \sum_{j=1}^p s_j Z_{ij}^2. $$ The variables $Z_{ij}$ are independent across both indices. Summing over the $\nu$ independent observations gives the stated sum of independent chi-square variables. Since a $\chi^2_\nu$ variable has mean $\nu$ and variance $2\nu$, independence yields mean $\nu\sum_j s_j$ and variance $2\nu\sum_j s_j^2$. These sums equal $\nu\operatorname{tr}(\Sigma)$ and $2\nu\operatorname{tr}(\Sigma^2)$. When all $s_j=\sigma^2$, the sum is $\sigma^2$ times a sum of $p$ independent $\chi^2_\nu$ variables, which has a $\chi^2_{p\nu}$ distribution. $\square$

The expected trace is a useful first check, not an acceptance boundary. A trace above its expectation can occur by sampling variation. In particular, compare a discrepancy with the trace variance, and remember that even a plausible trace cannot confirm the individual entries or dependence structure. Different matrices can have the same trace.

Worked Example: A Plausible Trace and Eigenvalue Check

Suppose $W\sim W_2(5,\Sigma)$, where $\Sigma=\begin{pmatrix}2&0\\0&1\end{pmatrix}$, and a simulated draw is $W=\begin{pmatrix}12&2\\2&7\end{pmatrix}$. Check its trace and eigenvalues against the target.

The expected trace is $5\operatorname{tr}(\Sigma)=5(3)=15$. The trace of the draw is $12+7=19$. Its trace variance is $2(5)\operatorname{tr}(\Sigma^2)=10(2^2+1^2)=50$, so its standard deviation is $\sqrt{50}\approx7.07$. The observed trace is $4/\sqrt{50}\approx0.57$ standard deviations above its mean: that difference alone is not a warning of a scale failure.

The characteristic equation is $(12-\lambda)(7-\lambda)-4=0$, or $\lambda^2-19\lambda+80=0$. Thus the eigenvalues are $(19+\sqrt{41})/2\approx12.70$ and $(19-\sqrt{41})/2\approx6.30$, both positive. Their sum is $19$, as required by the trace, and their product is $80$, the determinant $12(7)-2^2$. The matrix passes these basic checks; that does not prove it was generated by the intended algorithm.

Eigenvalues, Rank, and Positive-Definiteness

For any simulated symmetric matrix, the spectral theorem gives $W=V\Lambda V^\mathsf{T}$, where $V$ is orthogonal and $\Lambda$ contains the eigenvalues $\lambda_1,\ldots,\lambda_p$. Two identities turn this decomposition into a consistency check: $$ \operatorname{tr}(W)=\sum_{j=1}^p\lambda_j,\qquad \det(W)=\prod_{j=1}^p\lambda_j. $$

If $W$ is an exact Wishart draw under the stated convention, it is positive semidefinite: each term $X_iX_i^\mathsf{T}$ is positive semidefinite, and a sum of such matrices is positive semidefinite. Its eigenvalues therefore cannot be negative. The rank gives a sharper condition than a generic nonnegativity check.

Theorem: Suppose $\Sigma$ is positive definite and $W=\sum_{i=1}^{\nu}X_iX_i^\mathsf{T}$, with independent $X_i\sim N_p(0,\Sigma)$ and integer $\nu\geq1$. Then $\operatorname{rank}(W)=\min(p,\nu)$ with probability one. In particular, $W$ is positive definite with probability one exactly when $\nu\geq p$.

Proof. Form the $p$-by-$\nu$ matrix $X$ whose columns are $X_1,\ldots,X_\nu$. Then $W=XX^\mathsf{T}$, and for every $v\in\mathbb{R}^p$, $$ v^\mathsf{T}Wv=v^\mathsf{T}XX^\mathsf{T}v=\lVert X^\mathsf{T}v\rVert^2. $$ Thus the null space of $W$ is the null space of $X^\mathsf{T}$, and $\operatorname{rank}(W)=\operatorname{rank}(X)$. Because $\Sigma$ is positive definite, the entries of $X$ have a joint density on $\mathbb{R}^{p\nu}$. The rank of $X$ cannot exceed $\min(p,\nu)$. Let $m=\min(p,\nu)$. At least one $m$-by-$m$ minor determinant is a polynomial in the entries of $X$ that is not identically zero: for example, it is nonzero at a matrix whose first $m$-by-$m$ block is the identity. The zero set of a nonzero polynomial has Lebesgue measure zero. Since $X$ has a density, the probability that this minor vanishes is zero; hence $X$ has rank at least $m$ almost surely, and therefore exactly $m$. Finally, $W$ is positive definite precisely when its rank is $p$, which holds precisely when $\nu\geq p$. $\square$

Worked Example: A Structural Zero Eigenvalue

Take $p=3$, $\nu=2$, and $\Sigma=I_3$. Consider the two possible observed columns $X_1=(1,0,1)^\mathsf{T}$ and $X_2=(0,1,1)^\mathsf{T}$. Their scatter matrix is $$ W=X_1X_1^\mathsf{T}+X_2X_2^\mathsf{T} =\begin{pmatrix}1&0&1\\0&1&1\\1&1&2\end{pmatrix}. $$

The vector $(-1,-1,1)^\mathsf{T}$ is in the null space, because multiplying $W$ by it gives zero. The nonzero eigenvalues can be found from $X^\mathsf{T}X=\begin{pmatrix}2&1\\1&2\end{pmatrix}$, whose eigenvalues are $3$ and $1$. The nonzero eigenvalues of $XX^\mathsf{T}$ are the same, so the eigenvalues of $W$ are $3,1,0$. Their sum is $4=\operatorname{tr}(W)$. Rank $2$, rather than positive definiteness, is exactly what the theorem predicts when $\nu=2<p=3$.

Numerical Checks in Practice

A reliable diagnostic separates checks that are exact in theory from decisions that must allow for floating-point error. For a computed matrix, first check its symmetry; then inspect its trace and eigenvalues. In exact arithmetic, a Wishart draw cannot have a negative eigenvalue. In floating-point arithmetic, a very small negative eigenvalue can result from rounding, especially when the matrix is ill-conditioned.

Confirm dimensions and symmetry.
Check that the output is $p$-by-$p$ and that the largest discrepancy between $W_{ij}$ and $W_{ji}$ is small relative to the matrix scale. A material asymmetry suggests an implementation or storage problem.

Compare the trace with its target.
For the convention here, use $\nu\operatorname{tr}(\Sigma)$ as the mean and $2\nu\operatorname{tr}(\Sigma^2)$ as the variance. Treat these as a sampling benchmark, not as a deterministic interval that every draw must satisfy.

Check eigenvalue signs and rank.
There should be no material negative eigenvalues. For positive-definite $\Sigma$, expect rank $\min(p,\nu)$ almost surely; in particular, a full-rank draw is expected when $\nu\geq p$.

Investigate borderline values.
Use a tolerance scaled to the matrix norm and the precision of the computation. A common form is $\tau=c\epsilon\max(1,\lVert W\rVert_2)$, where $\epsilon$ is machine precision and $c$ is chosen with the dimension and numerical method in mind. This is a diagnostic convention, not a universal cutoff.

Worked Example: A Tiny Negative Eigenvalue

Suppose a calculation intended to produce a full-rank $2$-by-$2$ Wishart draw reports the symmetric matrix $$ \widetilde W=\begin{pmatrix}10&6.000000000001\\6.000000000001&3.600000000001\end{pmatrix}. $$ Its determinant, using the displayed rounded entries, is approximately $-2.0\times10^{-12}$, and its trace is $13.600000000001$. For a symmetric $2$-by-$2$ matrix with trace $t$ and determinant $d$, the eigenvalues are $(t\pm\sqrt{t^2-4d})/2$. Here the larger eigenvalue is approximately $13.6$, and the smaller is approximately $d/13.6\approx-1.5\times10^{-13}$.

A negative eigenvalue is impossible for an exact Wishart draw, but this one is about $10^{-14}$ of the matrix’s largest eigenvalue. It is therefore a rounding-scale warning, not by itself evidence of a substantively indefinite matrix. Check the raw generated observations, recompute their scatter with adequate precision, and compare the negative value with a scale-aware tolerance. Do not simply replace every negative eigenvalue by zero: a larger negative value can indicate a real coding, input, or parameter problem.

Common Pitfalls and Interpretation

A trace check only checks overall scale in one direction. For example, matrices with different eigenvalues can have the same trace, so a plausible trace cannot establish that the target covariance shape is correct. Likewise, the eigenvalues of a Wishart draw are random; they should not be expected to equal $\nu$ times the eigenvalues of $\Sigma$. The expectation $E[W]=\nu\Sigma$ describes the average matrix over repeated draws, not the eigenvalues of each draw.

A common mistake is to reject any computed matrix that has a negative eigenvalue, however tiny, or to declare it valid after forcing such values to zero. The correction is to distinguish exact theory from numerical computation: exact Wishart matrices are positive semidefinite, while computed eigenvalues require a tolerance tied to scale and precision. Then investigate material violations rather than concealing them. When $\nu<p$, zero eigenvalues are not numerical defects; they are required by the rank constraint.

Together, trace, eigenvalue signs, and rank form a compact set of checks. They can identify many failures in simulated Wishart matrices, but they do not replace validation of the generation procedure or repeated-simulation checks of its distribution.

Check Your Understanding

Use the stated $W_p(\nu,\Sigma)$ convention in answering these questions.

For $W\sim W_3(4,\Sigma)$ with $\operatorname{tr}(\Sigma)=6$ and $\operatorname{tr}(\Sigma^2)=14$, calculate the mean and variance of $\operatorname{tr}(W)$.
If $\Sigma$ is positive definite, what rank should a $5$-by-$5$ Wishart draw have almost surely when $\nu=3$? Is it positive definite?
Why does a plausible trace not establish that a simulated matrix has the correct eigenvalue pattern?
What should you investigate if a computed eigenvalue is slightly negative relative to the largest eigenvalue? What if it is substantially negative?
For $\Sigma=\sigma^2 I_p$, state the distribution of $\operatorname{tr}(W)/\sigma^2$.