Worked Problems In Wishart Sampling Theory

Putting Wishart Calculations Together

A Wishart calculation often combines several decisions: which scatter matrix is being used, how many degrees of freedom it has, what scale results after a linear transformation, and whether the requested moment or sampling law exists under the stated assumptions. A correct formula applied to the wrong scatter matrix can still produce a wrong answer. The reliable approach is to identify the random matrix first, then track its degrees of freedom and scale through each operation.

This tutorial works through those decisions using centered Gaussian scatter, congruence transformations, determinant and inverse moments, and an exact Hotelling $T^2$ distribution. The Wishart convention is stated explicitly so that the scale parameter and degrees of freedom are unambiguous.

Definition: For a positive-definite $p\times p$ matrix $\Sigma$ and $\nu>p-1$, $W\sim W_p(\nu,\Sigma)$ denotes the Wishart distribution with degrees of freedom $\nu$ and scale matrix $\Sigma$. Its density on positive-definite $W$ is $$ f(W)=\frac{|W|^{(\nu-p-1)/2}\exp\{-\tfrac12\operatorname{tr}(\Sigma^{-1}W)\}}{2^{\nu p/2}|\Sigma|^{\nu/2}\Gamma_p(\nu/2)}, \qquad \Gamma_p(a)=\pi^{p(p-1)/4}\prod_{j=1}^p\Gamma\!\left(a-\frac{j-1}{2}\right). $$ In particular, if $X_1,\ldots,X_n$ are independent $N_p(\mu,\Sigma)$ observations, the centered scatter $W=\sum_{i=1}^n(X_i-\bar X)(X_i-\bar X)^T$ has distribution $W_p(n-1,\Sigma)$, whereas scatter about a known $\mu$, $\sum_{i=1}^n(X_i-\mu)(X_i-\mu)^T$, has distribution $W_p(n,\Sigma)$.

Degrees of Freedom and Scale Under Transformation

The degrees of freedom are determined by how the scatter is formed, not by the dimension $p$ of its matrices. Estimating the mean and centering the observations costs one degree of freedom. Dividing a scatter matrix by a constant to form a sample covariance changes its scale, but it does not change the degrees of freedom of the underlying Wishart matrix.

A second bookkeeping rule concerns a linear change of coordinates. If $A$ is nonsingular and $W\sim W_p(\nu,\Sigma)$, then $AWA^T$ has the same degrees of freedom and scale $A\Sigma A^T$. The following derivation uses the Gaussian sampling representation; it also makes clear why the transformation is a congruence, not $A\Sigma A$.

Theorem: If $W\sim W_p(\nu,\Sigma)$ is a Gaussian scatter matrix and $A$ is a nonsingular $p\times p$ matrix, then $AWA^T\sim W_p(\nu,A\Sigma A^T)$.

Represent the scatter.
Write $W=\sum_{i=1}^{\nu}Z_iZ_i^T$, where the $Z_i$ are independent $N_p(0,\Sigma)$ vectors. For centered scatter this representation follows by taking orthogonal contrasts of the original observations.

Transform each vector.
Set $Y_i=AZ_i$. A linear transformation of a multivariate normal vector gives $Y_i\sim N_p(0,A\Sigma A^T)$, and the $Y_i$ remain independent.

Reassemble the scatter.
$AWA^T=\sum_{i=1}^{\nu}(AZ_i)(AZ_i)^T=\sum_{i=1}^{\nu}Y_iY_i^T$. This is Wishart scatter with the same number of terms and transformed scale, proving the claim. $\square$

Worked Example: Centered Scatter After Rescaling Coordinates

Suppose $n=12$ independent observations from $N_2(\mu,\Sigma)$ have $\Sigma=\begin{pmatrix}4&1\\1&2\end{pmatrix}$. Let $W$ be their centered scatter, and rescale each observation using $A=\operatorname{diag}(2,1/3)$. Find the distribution of the transformed scatter $AWA^T$.

Because the mean is estimated from the observations, the degrees of freedom are $n-1=11$, so $W\sim W_2(11,\Sigma)$. The transformed scale is $$ A\Sigma A^T = \begin{pmatrix}2&0\\0&1/3\end{pmatrix} \begin{pmatrix}4&1\\1&2\end{pmatrix} \begin{pmatrix}2&0\\0&1/3\end{pmatrix} = \begin{pmatrix}16&2/3\\2/3&2/9\end{pmatrix}. $$ Thus $AWA^T\sim W_2\!\left(11,\begin{pmatrix}16&2/3\\2/3&2/9\end{pmatrix}\right)$.

As a check, $|\Sigma|=4(2)-1=7$ and $|A|=2/3$. Therefore $|A\Sigma A^T|=|A|^2|\Sigma|=(4/9)(7)=28/9$. Directly, the determinant of the displayed transformed scale is $16(2/9)-(2/3)^2=32/9-4/9=28/9$, as required.

Determinant and Inverse Moments

Determinant moments are conveniently calculated through the Bartlett factorization. For $W\sim W_p(\nu,\Sigma)$, that factorization gives $$ |W|\ \overset{d}{=}\ |\Sigma|\prod_{i=1}^{p}U_i, \qquad U_i\ \text{independent},\quad U_i\sim\chi^2_{\nu-i+1}. $$ The degrees of freedom in the product descend from $\nu$ to $\nu-p+1$. This last, smallest degree of freedom determines how far a negative determinant moment can be taken.

Theorem: If $W\sim W_p(\nu,\Sigma)$, $\nu>p-1$, and $a>-(\nu-p+1)/2$, then $$ E[|W|^a]=|\Sigma|^a\,2^{pa}\prod_{i=1}^{p} \frac{\Gamma((\nu-i+1)/2+a)}{\Gamma((\nu-i+1)/2)}. $$ The strict inequality on $a$ is necessary for finiteness.

To prove the formula, a chi-square variable $U\sim\chi^2_k$ has density $u^{k/2-1}e^{-u/2}/(2^{k/2}\Gamma(k/2))$ for $u>0$. For $a>-k/2$, integrate against this density and substitute $v=u/2$: $$ E[U^a] =\frac{1}{2^{k/2}\Gamma(k/2)}\int_0^\infty u^{k/2+a-1}e^{-u/2}\,du =2^a\frac{\Gamma(k/2+a)}{\Gamma(k/2)}. $$ The integral is finite precisely when $k/2+a>0$. Applying this result separately to each independent $U_i$ in the determinant factorization gives the displayed product. The most restrictive integrability condition is for $k=\nu-p+1$, yielding $a>-(\nu-p+1)/2$. $\square$

Worked Example: A Negative Determinant Moment and an Inverse Moment

Let $W\sim W_3(8,\Sigma)$, with $\Sigma=\operatorname{diag}(2,1,3)$. Calculate $E[|W|^{-1}]$, and then calculate $E[W^{-1}]$.

First, $|\Sigma|=2\cdot1\cdot3=6$. For $p=3,\nu=8$, the chi-square degrees of freedom in the determinant factorization are $8,7,6$. The requested power $a=-1$ is permitted because $-1>-(8-3+1)/2=-3$. For $U\sim\chi^2_k$, the moment formula above gives $E[U^{-1}]=1/(k-2)$ when $k>2$. Hence $$ E[|W|^{-1}] =|\Sigma|^{-1}\left(\frac{1}{8-2}\right) \left(\frac{1}{7-2}\right)\left(\frac{1}{6-2}\right) =\frac{1}{6\cdot6\cdot5\cdot4} =\frac{1}{720}. $$

For the inverse-matrix moment, the inverse-Wishart mean formula is $E[W^{-1}]=\Sigma^{-1}/(\nu-p-1)$, valid when $\nu>p+1$. Here $8>4$, so the expectation exists. Since $\Sigma^{-1}=\operatorname{diag}(1/2,1,1/3)$, it follows that $$ E[W^{-1}]=\frac{1}{8-3-1}\operatorname{diag}(1/2,1,1/3) =\operatorname{diag}(1/8,1/4,1/12). $$ The determinant moment and inverse-matrix moment are different quantities; the first is a scalar expectation and the second is an entrywise matrix expectation.

Selecting an Exact Sampling Distribution

Exact sampling laws require more than recognizing a familiar expression. Check the model, the centering, and the divisor in the covariance estimate. For example, the exact one-sample Hotelling law applies to independent multivariate normal observations when the mean is compared with a fixed hypothesized value and the covariance is estimated by the usual centered sample covariance. It does not follow from a Wishart scatter alone if the Gaussian sampling assumptions or the matching mean–scatter construction are absent.

Theorem: Let $X_1,\ldots,X_n$ be independent $N_p(\mu,\Sigma)$ observations, where $\Sigma$ is positive definite, and define $$ \bar X=\frac{1}{n}\sum_{i=1}^{n}X_i,\qquad S=\frac{1}{n-1}\sum_{i=1}^{n}(X_i-\bar X)(X_i-\bar X)^T. $$ For a fixed $\mu_0$, under $H_0:\mu=\mu_0$ and $n>p$, $$ \frac{n-p}{p(n-1)}\,n(\bar X-\mu_0)^TS^{-1}(\bar X-\mu_0) \sim F_{p,n-p}. $$

The scaling can be checked by writing $W=(n-1)S$. Under the null, $n(\bar X-\mu_0)^T\Sigma^{-1}(\bar X-\mu_0)\sim\chi^2_p$; independently, $W\sim W_p(n-1,\Sigma)$. The normal–Wishart ratio then gives Hotelling’s exact $T^2$ law. Since $W^{-1}=(n-1)^{-1}S^{-1}$, the statistic in that ratio is $n(\bar X-\mu_0)^TW^{-1}(\bar X-\mu_0)=T^2/(n-1)$; thus the ratio gives $\frac{n-p}{p(n-1)}T^2\sim F_{p,n-p}$. The condition $n>p$ ensures the centered scatter is invertible with probability one and the $F$ denominator degrees of freedom are positive.

Worked Example: An Exact One-Sample Hotelling Test

Ten independent bivariate normal observations have sample mean $\bar X$ and sample covariance $S$, computed with divisor $n-1$. To test $H_0:\mu=\mu_0$, suppose the observed statistic is $T^2=9$, where $T^2=n(\bar X-\mu_0)^TS^{-1}(\bar X-\mu_0)$. Find its exact null reference distribution and upper-tail probability.

Here $n=10$ and $p=2$, so $n>p$ and the exact result applies. The scaled statistic is $$ \frac{n-p}{p(n-1)}T^2 =\frac{10-2}{2(10-1)}T^2 =\frac{8}{18}T^2 =\frac{4}{9}T^2. $$ Under the null, $(4/9)T^2\sim F_{2,8}$. At the observed value $T^2=9$, the $F$ statistic is $4$.

For an $F_{2,8}$ variable, the upper-tail probability at $x\geq0$ is $(1+2x/8)^{-4}$. Therefore the exact p-value is $$ P(F_{2,8}\geq4)=\left(1+\frac{2(4)}{8}\right)^{-4}=2^{-4}=\frac{1}{16}=0.0625. $$ This probability is exact under the stated normal model and null hypothesis, not an approximation based only on the sample size.

Common Mistakes and Practical Checks

Most errors in integrated Wishart calculations are bookkeeping errors. A useful check is to write down the random matrix, its construction, degrees of freedom, and scale before substituting into a formula.

Using $n$ degrees of freedom for centered scatter. Centering at the estimated sample mean gives $n-1$ degrees of freedom. The $n$-degree result applies to scatter about a known population mean.
Transforming the scale incorrectly. For $AWA^T$, the transformed scale is $A\Sigma A^T$, and its determinant is $|A|^2|\Sigma|$. In general, it is not $A\Sigma A$.
Ignoring a moment’s existence condition. A negative determinant power requires $a>-(\nu-p+1)/2$. An inverse-Wishart mean additionally requires $\nu>p+1$.
Using the wrong sample-covariance divisor in Hotelling’s law. The displayed $T^2$-to-$F$ formula uses $S=W/(n-1)$. Replacing $S$ by a differently scaled matrix changes the statistic and its multiplier.
Calling a distribution exact without its model assumptions. The Hotelling reference law requires independent multivariate normal observations and the stated null setup; a matrix having a Wishart distribution by itself does not establish those conditions.

Key takeaway: Track the Wishart scatter’s origin before calculating. Centering determines its degrees of freedom, a congruence transforms its scale, and the smallest Bartlett chi-square degree of freedom controls negative determinant moments. Exact sampling laws also require their full model assumptions.

Check Your Understanding

For each problem, state the relevant assumptions before selecting a formula.

For $n=15$ independent $N_p(\mu,\Sigma)$ observations, what are the degrees of freedom of the centered scatter? How do they differ from scatter about known $\mu$?
If $W\sim W_p(\nu,\Sigma)$ and $A$ is nonsingular, what are the degrees of freedom and scale of $AWA^T$?
For $W\sim W_4(9,\Sigma)$, is $E[|W|^{-2}]$ finite? Check the strict determinant-moment condition.
For $W\sim W_3(6,\Sigma)$, does the inverse-Wishart mean formula for $E[W^{-1}]$ apply? Explain using its degrees-of-freedom condition.
In the one-sample Hotelling test, why must the sample covariance divisor and the centering convention match the stated $T^2$-to-$F$ formula?