Wishart And Inverse-Wishart Parameterization Pitfalls

Why Wishart Parameters Are Easy to Misread

A Wishart matrix summarizes Gaussian scatter, but its parameters do not all have the interpretation their names might suggest. Its scale matrix is not the mean of the random matrix, and its degrees of freedom need not equal the number of observations used to construct centered scatter. Inverse-Wishart notation introduces another hazard: the same family may be indexed by degrees of freedom or by a shifted parameter. A reliable comparison starts with the density, support, and moment conditions, not the symbols alone.

Throughout, $p$ is the matrix dimension. We use $W_p(\nu,\Sigma)$ for a Wishart distribution with degrees of freedom $\nu$ and scale matrix $\Sigma$, and $IW_p(\nu,\Psi)$ for an inverse-Wishart distribution with degrees of freedom $\nu$ and scale matrix $\Psi$. Both scale matrices are positive definite.

Definition: Under the convention used here, $A\sim W_p(\nu,\Sigma)$ has density, with respect to Lebesgue measure on the independent entries of a symmetric matrix, $$ f_A(A)= \frac{|A|^{(\nu-p-1)/2}\exp\{-\tfrac12\operatorname{tr}(\Sigma^{-1}A)\}} {2^{\nu p/2}|\Sigma|^{\nu/2}\Gamma_p(\nu/2)},\qquad A>0, $$ where $\nu>p-1$, and $\Gamma_p(a)=\pi^{p(p-1)/4}\prod_{j=1}^{p}\Gamma(a-(j-1)/2)$. The density is zero outside the positive-definite cone. In this convention, $E[A]=\nu\Sigma$.

The density immediately separates two quantities that are often conflated. The degrees of freedom $\nu$ control the determinant exponent and contribute to the mean. The scale $\Sigma$ enters the trace term and the normalizing constant. Thus $\Sigma$ is not generally the expected value of $A$; rather, $E[A]=\nu\Sigma$.

Scatter, Sample Covariance, and Degrees of Freedom

Suppose $X_1,\ldots,X_n$ are independent $N_p(\mu,\Sigma)$ observations, with $\mu$ unknown, and define the centered scatter $$ C=\sum_{i=1}^n (X_i-\bar X)(X_i-\bar X)^\mathsf{T}, \qquad \bar X=\frac1n\sum_{i=1}^n X_i. $$ For $n>p$, $C\sim W_p(n-1,\Sigma)$ under the density convention above. The condition matters: centered observations satisfy a linear constraint, so $\operatorname{rank}(C)\leq n-1$. If $n\leq p$, $C$ is singular and does not have the positive-definite density just defined.

The unbiased sample covariance is $S=C/(n-1)$, while the Gaussian maximum-likelihood covariance estimate is $\widehat\Sigma_{\mathrm{ML}}=C/n$. Rescaling changes the Wishart scale, not the degrees of freedom: $$ S\sim W_p\left(n-1,\frac{\Sigma}{n-1}\right), \qquad \widehat\Sigma_{\mathrm{ML}}\sim W_p\left(n-1,\frac{\Sigma}{n}\right), \qquad (n>p). $$ Consequently, $E[S]=\Sigma$, whereas $E[\widehat\Sigma_{\mathrm{ML}}]=\frac{n-1}{n}\Sigma$. Neither rescaled matrix has degrees of freedom $n$.

Worked Example: Converting Scatter to Covariance

Let $n=6$, $p=2$, and $\Sigma=\operatorname{diag}(4,1)$. The centered scatter has degrees of freedom $n-1=5$, and $5>p-1$, so it has a nonsingular Wishart density: $C\sim W_2(5,\Sigma)$. Its expectation is $$ E[C]=5\Sigma=\begin{pmatrix}20&0\\0&5\end{pmatrix}. $$

The unbiased covariance estimate is $S=C/5$, so its Wishart scale is $\Sigma/5=\operatorname{diag}(0.8,0.2)$, and $E[S]=\Sigma$. The maximum-likelihood estimate is $C/6$, with scale $\Sigma/6=\operatorname{diag}(2/3,1/6)$, and expectation $(5/6)\Sigma$. Dividing the observed matrix by a constant changes its scale; it does not turn five degrees of freedom into six.

The density also confirms the distinction numerically. At $C=\Sigma$, $|C|=4$, $|\Sigma|=4$, $\operatorname{tr}(\Sigma^{-1}C)=2$, and $\Gamma_2(5/2)=3\pi/4$. Therefore $$ f_C(\Sigma)= \frac{4e^{-1}}{2^5\,4^{5/2}(3\pi/4)} =\frac{e^{-1}}{192\pi}. $$ Since $S=C/5$ and the space of symmetric $2\times2$ matrices has dimension $3$, the change-of-variables factor is $5^3$. Hence $f_S(\Sigma/5)=125e^{-1}/(192\pi)$, consistent with the rescaled Wishart scale.

A Density Check for Rescaling

The rescaling rule can be derived directly rather than memorized. If $A\sim W_p(\nu,\Sigma)$ and $B=cA$ for $c>0$, then $A=B/c$. The vector space of symmetric $p\times p$ matrices has $q=p(p+1)/2$ independent coordinates, so the density transformation contributes the Jacobian $c^{-q}$ when expressing the density of $B$ in terms of $A=B/c$. Substituting in the Wishart density gives $$ f_B(B)=c^{-q} f_A(B/c) = \frac{|B|^{(\nu-p-1)/2} \exp\{-\tfrac12\operatorname{tr}((c\Sigma)^{-1}B)\}} {2^{\nu p/2}|c\Sigma|^{\nu/2}\Gamma_p(\nu/2)}. $$ Here $|B/c|=c^{-p}|B|$, and $c^{-q}c^{-p(\nu-p-1)/2}=c^{-\nu p/2}$; these are exactly the factors absorbed into $|c\Sigma|^{\nu/2}$. Thus the degrees of freedom remain $\nu$ and the scale becomes $c\Sigma$.

Theorem: If $A\sim W_p(\nu,\Sigma)$ and $c>0$, then $cA\sim W_p(\nu,c\Sigma)$. The density calculation above proves the claim, including its normalizing constant. In particular, multiplying or dividing a scatter matrix by a scalar does not change its degrees of freedom. $\square$

A related rank check prevents an invalid use of that density. Centering $n$ observations removes one independent direction across the observations, so $C$ has rank at most $n-1$. A positive-definite $p\times p$ matrix requires rank $p$; therefore $n>p$ is needed for the centered scatter to have the nonsingular law $W_p(n-1,\Sigma)$. By contrast, if the mean is known and one forms $\sum_i(X_i-\mu)(X_i-\mu)^\mathsf{T}$, the degrees of freedom are $n$; the nonsingular density requires $n>p-1$. These conditions follow from the support and should not be dropped when quoting the sampling law.

Worked Example: Detecting a Singular Centered Scatter

Take $n=2$ observations in dimension $p=3$. The centered residuals are $X_1-\bar X$ and $X_2-\bar X$, and $$ X_1-\bar X=\frac{X_1-X_2}{2}, \qquad X_2-\bar X=-\frac{X_1-X_2}{2}. $$ The two residual vectors are opposites, so their outer products sum to a matrix of rank at most one. In fact, $C$ cannot be positive definite in dimension three.

The formal degrees of freedom from centering would be $n-1=1$, but the density convention requires $\nu>p-1=2$. Thus it is incorrect to state that this $C$ has the nonsingular $W_3(1,\Sigma)$ density. One may describe it through its Gaussian sampling construction as a singular Wishart matrix, but that is a different support statement from the positive-definite density defined above.

Inverse-Wishart Conventions and Moment Checks

Under the convention used here, $X\sim IW_p(\nu,\Psi)$ has density $$ f_X(X)= \frac{|\Psi|^{\nu/2}} {2^{\nu p/2}\Gamma_p(\nu/2)} |X|^{-(\nu+p+1)/2} \exp\{-\tfrac12\operatorname{tr}(\Psi X^{-1})\}, \qquad X>0,\quad \nu>p-1. $$ The parameter $\Psi$ is the inverse-Wishart scale matrix. It appears in the trace with $X^{-1}$, not with $X$.

A frequent alternative is to use the shifted index $\kappa=\nu-p-1$. This is a reparameterization, not a different distribution. In terms of $\kappa$, the determinant exponent is $-(\kappa/2+p+1)$, and the multivariate gamma function is evaluated at $(\kappa+p+1)/2$. The same law has a finite mean when $\kappa>0$; in the degrees-of-freedom notation this condition is $\nu>p+1$. Its mean is $$ E[X]=\frac{\Psi}{\nu-p-1}=\frac{\Psi}{\kappa}, \qquad \nu>p+1. $$ The distinction between the two labels is consequential: inserting $\kappa$ into a formula that expects $\nu$ gives the wrong density and moment.

Worked Example: Translating an Inverse-Wishart Index

Let $p=2$, $\nu=6$, and $\Psi=I_2$. In the degrees-of-freedom convention, the determinant exponent is $-(6+2+1)/2=-9/2$, and the mean exists because $6>2+1$. The multivariate gamma value is $$ \Gamma_2(3)=\sqrt{\pi}\,\Gamma(3)\Gamma(5/2)=\frac{3\pi}{2}. $$ At $X=I_2$, the density is therefore $$ f_X(I_2)=\frac{1}{2^6\Gamma_2(3)}\exp\{-\tfrac12\operatorname{tr}(I_2)\} =\frac{e^{-1}}{96\pi}. $$

The shifted index is $\kappa=6-2-1=3$. In that notation the determinant exponent is $-(\kappa/2+p+1)=-9/2$, and the mean is $I_2/\kappa=I_2/3$. Both conventions give the same density and mean. A mistaken substitution of $\nu-1$ for the denominator would instead give $I_2/5$; it is wrong because the mean denominator is $\nu-p-1$, not $\nu-1$.

Inversion Connects the Two Families

A useful way to check the inverse-Wishart scale placement is to invert a Wishart matrix. For a differentiable inversion map on the positive-definite cone, the absolute Jacobian in the independent symmetric-matrix coordinates is $|X|^{-(p+1)}$ when $A=X^{-1}$. To see the power, the differential is $dA=-X^{-1}(dX)X^{-1}$. For a congruence map $H\mapsto BHB^\mathsf{T}$ on symmetric matrices, its absolute determinant is $|B|^{p+1}$: after diagonalizing $B$, the diagonal coordinates contribute $\prod_i b_i^2$, and the off-diagonal coordinates contribute $\prod_{i<j}b_ib_j$, giving $\prod_i b_i^{p+1}$. Taking $B=X^{-1}$ yields the stated inversion Jacobian.

Now let $A\sim W_p(\nu,\Sigma)$ and set $X=A^{-1}$. The transformed density is $f_A(X^{-1})|X|^{-(p+1)}$. Its determinant power is $$ |X^{-1}|^{(\nu-p-1)/2}|X|^{-(p+1)} =|X|^{-(\nu+p+1)/2}, $$ and its trace term is $\exp\{-\tfrac12\operatorname{tr}(\Sigma^{-1}X^{-1})\}$. The normalizing constant has $|\Sigma|^{-\nu/2}=|\Sigma^{-1}|^{\nu/2}$. These are exactly the inverse-Wishart density terms with scale $\Psi=\Sigma^{-1}$.

Theorem: If $A\sim W_p(\nu,\Sigma)$, then $A^{-1}\sim IW_p(\nu,\Sigma^{-1})$, under the density conventions given here. The inversion calculation proves the equality of the full densities, not just a correspondence between their names. $\square$

The corresponding moment check is equally useful. Since $X^{-1}\sim W_p(\nu,\Psi^{-1})$ when $X\sim IW_p(\nu,\Psi)$, the Wishart mean gives $E[X^{-1}]=\nu\Psi^{-1}$ for $\nu>p-1$. This does not imply that $E[X]$ exists under the same condition: the inverse-Wishart mean requires the stricter $\nu>p+1$. In dimension one, the density reduces to an inverse-gamma law with mean $\Psi/(\nu-2)$, confirming that the multivariate denominator $\nu-p-1$ has the correct scalar limit.

Common Parameterization Pitfalls

Calling the Wishart scale the mean. For $A\sim W_p(\nu,\Sigma)$, the mean is $\nu\Sigma$, not $\Sigma$.
Using sample size as centered-scatter degrees of freedom. With an estimated mean, centered scatter has $n-1$ degrees of freedom, not $n$.
Writing a nonsingular Wishart density for a rank-deficient matrix. For centered scatter, check $n>p$; when $n\leq p$, the matrix is singular.
Mixing inverse-Wishart indices. If the index is $\nu$, the mean denominator is $\nu-p-1$. If the index is $\kappa=\nu-p-1$, it is $\kappa$.
Placing the inverse-Wishart scale in the wrong trace term. The exponent contains $\operatorname{tr}(\Psi X^{-1})$, not $\operatorname{tr}(\Psi^{-1}X)$.

Key takeaway: To audit a Wishart or inverse-Wishart formula, identify its support, determinant exponent, trace term, and moment condition. Those features reveal the parameter convention and expose most errors in scale, degrees of freedom, and index shifts.

Check Your Understanding

Use the density and moment checks above to answer the following.

If $A\sim W_3(8,\Sigma)$, what is $E[A]$, and what is the scale matrix?
For centered Gaussian observations with $n=4$ and $p=4$, can the scatter have the nonsingular $W_p(n-1,\Sigma)$ density? Explain using rank and the degrees-of-freedom condition.
If $C\sim W_p(9,\Sigma)$, what is the Wishart law of $C/3$?
For $X\sim IW_3(10,\Psi)$, what is the condition for $E[X]$ to exist, and what is its value when it exists?
If an inverse-Wishart index is $\kappa=\nu-p-1$, express its mean using $\kappa$ and state how to recover $\nu$.