Noncentral Wishart Matrices From Shifted Gaussian Samples

Why Mean Shifts Change a Wishart Law

A Wishart matrix can arise from sums of outer products of Gaussian vectors whose means are not zero. The resulting matrix is still built from Gaussian sampling, but its distribution is not the central Wishart law: the mean shifts contribute a positive semidefinite noncentrality matrix. Identifying that matrix precisely is important, because “nonzero means” alone does not specify the size or direction of the departure from centrality.

The distinction also depends on which scatter matrix is being formed. The uncentered sum of outer products retains a common mean shift, whereas subtracting the sample mean removes a common shift. If the individual population means differ, however, centering need not remove all noncentrality.

Definition: Let $X_1,\ldots,X_n$ be independent $p$-dimensional Gaussian vectors with $X_i\sim N_p(\mu_i,\Sigma)$, where $\Sigma$ is positive definite and the mean vectors $\mu_i$ are fixed. Define $$ W=\sum_{i=1}^n X_iX_i',\qquad \Theta=\sum_{i=1}^n\mu_i\mu_i',\qquad \Lambda=\Sigma^{-1/2}\Theta\Sigma^{-1/2}. $$ Then $W$ has a noncentral Wishart distribution with $n$ degrees of freedom, scale matrix $\Sigma$, and whitened noncentrality matrix $\Lambda$, denoted $W\sim W_p(n,\Sigma,\Lambda)$. The symmetric matrix $\Lambda$ is positive semidefinite. When every $\mu_i=0$, $\Lambda=0$ and this is the central Wishart law.

Here $\Sigma^{-1/2}$ is the symmetric positive definite inverse square root. If $M=[\mu_1\ \cdots\ \mu_n]$, then $\Theta=MM'$ and $\Lambda=(\Sigma^{-1/2}M)(\Sigma^{-1/2}M)'$. This shows that $\Lambda$ records the aggregate mean shifts after whitening by the sampling covariance. Its rank is at most $\min(p,n)$. Some references use an unwhitened noncentrality parameter $\Theta$, or a nonsymmetric matrix such as $\Sigma^{-1}\Theta$. Here the parameter in the distribution notation is specifically the symmetric, whitened $\Lambda$.

Characterizing the Distribution

A useful way to characterize this law without needing its density is its Laplace transform. For a symmetric positive semidefinite $p\times p$ matrix $T$, the transform is

E\!\left[\exp\{-\operatorname{tr}(TW)\}\right] = |I_p+2\Sigma T|^{-n/2} \exp\!\left\{-\operatorname{tr}\!\left[ \Lambda B(I_p+2B)^{-1} \right]\right\}, \qquad B=\Sigma^{1/2}T\Sigma^{1/2}.

The determinant in this expression is positive; $I_p+2B$ is positive definite. The expression also makes the role of the parameters visible: the determinant term is the central Wishart contribution, and the exponential term involving $\Lambda$ carries the mean shifts.

Theorem: Under the Gaussian sampling assumptions in the definition, the displayed Laplace transform holds. In particular, the distribution of $W$ depends on the mean vectors through their aggregate $\Lambda$, not through their ordering across the $n$ observations.

Proof. Set $Y_i=\Sigma^{-1/2}X_i$ and $a_i=\Sigma^{-1/2}\mu_i$. Then $Y_i\sim N_p(a_i,I_p)$, and $\operatorname{tr}(TW)=\sum_i Y_i'BY_i$, with $B=\Sigma^{1/2}T\Sigma^{1/2}$. For any $Y\sim N_p(a,I_p)$, integrating its density and completing the square gives

\begin{aligned} E[\exp(-Y'BY)] &=(2\pi)^{-p/2}\int_{\mathbb R^p} \exp\!\left\{-\tfrac12(y-a)'(y-a)-y'By\right\}\,dy\\ &=|I_p+2B|^{-1/2} \exp\!\left\{-a'B(I_p+2B)^{-1}a\right\}. \end{aligned}

Indeed, the quadratic term in the exponent has precision matrix $I_p+2B$; completing its square yields the determinant factor shown, and the remaining constant is $-a'B(I_p+2B)^{-1}a$. Independence allows these expressions to be multiplied over $i$. The determinant factors give $|I_p+2B|^{-n/2}=|I_p+2\Sigma T|^{-n/2}$. The exponents sum to

-\sum_{i=1}^n a_i'B(I_p+2B)^{-1}a_i =-\operatorname{tr}\!\left[ \left(\sum_{i=1}^n a_i a_i'\right)B(I_p+2B)^{-1} \right] =-\operatorname{tr}\!\left[ \Lambda B(I_p+2B)^{-1} \right].

This proves the transform and shows that the mean vectors enter it only through $\Lambda$. $\square$

Worked Example: Computing Noncentrality From Mean Vectors

Let $p=2$, $n=2$, and $\Sigma=4I_2$. Take $\mu_1=(\sqrt{2},1/\sqrt{2})'$ and $\mu_2=(0,3/\sqrt{2})'$. Find $\Theta$, $\Lambda$, and $E[W]$.

First add the two outer products:

\Theta =\mu_1\mu_1'+\mu_2\mu_2' = \begin{pmatrix}2&1\\1&1/2\end{pmatrix} + \begin{pmatrix}0&0\\0&9/2\end{pmatrix} = \begin{pmatrix}2&1\\1&5\end{pmatrix}.

Since $\Sigma^{-1/2}=\tfrac12 I_2$, both the left and right multiplications contribute a factor of $1/2$ to every entry:

\Lambda =\frac12 \begin{pmatrix}2&1\\1&5\end{pmatrix} \frac12 = \begin{pmatrix}1/2&1/4\\1/4&5/4\end{pmatrix}.

In particular, each off-diagonal entry is $(1/2)(1)(1/2)=1/4$. The matrix is positive semidefinite: its determinant is $5/8-1/16=9/16>0$, and its first diagonal entry is positive. Finally, using $E[W]=n\Sigma+\Theta$,

E[W]=2(4I_2)+ \begin{pmatrix}2&1\\1&5\end{pmatrix} = \begin{pmatrix}10&1\\1&13\end{pmatrix}.

The noncentrality measures the whitened mean contribution; the expected uncentered sum includes that contribution in addition to the $n\Sigma$ contribution from Gaussian variation.

Mean, Covariance, and a Projection Result

Write $\Theta=\Sigma^{1/2}\Lambda\Sigma^{1/2}=\sum_i\mu_i\mu_i'$. The first two moments of $W$ make the contribution of mean shifts explicit.

Theorem: Under the assumptions in the definition, for indices $a,b,c,d\in\{1,\ldots,p\}$, $$ E[W]=n\Sigma+\Theta $$ and $$ \begin{aligned} \operatorname{Cov}(W_{ab},W_{cd}) ={}&n(\Sigma_{ac}\Sigma_{bd}+\Sigma_{ad}\Sigma_{bc})\\ &+\Sigma_{ac}\Theta_{bd}+\Sigma_{ad}\Theta_{bc} +\Sigma_{bc}\Theta_{ad}+\Sigma_{bd}\Theta_{ac}. \end{aligned} $$

Proof. For each observation write $X_i=\mu_i+\varepsilon_i$, where the independent $\varepsilon_i$ have mean zero and covariance $\Sigma$. Thus $E[X_iX_i']=\Sigma+\mu_i\mu_i'$; summing gives the stated expectation.

To obtain the covariance, independence across $i$ means that the covariance of sums is the sum of their covariances. For one observation, expand each product $X_aX_bX_cX_d$ using $X=\mu+\varepsilon$. Terms with an odd number of centered Gaussian factors have expectation zero. The fourth centered moment is $E[\varepsilon_a\varepsilon_b\varepsilon_c\varepsilon_d] =\Sigma_{ab}\Sigma_{cd}+\Sigma_{ac}\Sigma_{bd}+\Sigma_{ad}\Sigma_{bc}$; this identity follows by taking four derivatives of the Gaussian moment-generating function $\exp(t'\Sigma t/2)$ at $t=0$. Subtracting $E[X_aX_b]E[X_cX_d]$ cancels the $\Sigma_{ab}\Sigma_{cd}$ term and the pure-mean product. What remains is

\operatorname{Cov}(X_aX_b,X_cX_d) =\Sigma_{ac}\Sigma_{bd}+\Sigma_{ad}\Sigma_{bc} +\mu_a\mu_c\Sigma_{bd}+\mu_a\mu_d\Sigma_{bc} +\mu_b\mu_c\Sigma_{ad}+\mu_b\mu_d\Sigma_{ac}.

Summing over $i$ replaces each $\mu_{i,r}\mu_{i,s}$ by $\Theta_{rs}$, giving the covariance formula. $\square$

Worked Example: Checking a Scalar Noncentral Wishart Moment

Let $X_1,X_2$ be independent with $X_i\sim N(1,4)$, where the second normal parameter is the variance. For $W=X_1^2+X_2^2$, find its noncentrality, expectation, and variance.

Here $p=1$, $\Sigma=4$, and $\Theta=1^2+1^2=2$. Hence $\Lambda=\Sigma^{-1}\Theta=2/4=1/2$, and $W\sim W_1(2,4,1/2)$. Its expectation is

E[W]=n\Sigma+\Theta=2(4)+2=10.

For the variance formula, all indices are the single scalar coordinate, so the central term is $2n\Sigma^2=2(2)(16)=64$. The four noncentral terms together are $4\Sigma\Theta=4(4)(2)=32$. Therefore

\operatorname{Var}(W)=64+32=96.

As a check, $W/4$ is a noncentral chi-square variable with 2 degrees of freedom and noncentrality $1/2$; its variance is $2(2+2(1/2))=6$, so $\operatorname{Var}(W)=4^2(6)=96$, in agreement.

The same construction applies after an orthogonal projection on the observation index. This connects noncentrality to the part of the mean matrix retained by the projection.

Theorem: Let $X=[X_1\ \cdots\ X_n]=M+\Sigma^{1/2}Z$, where the columns of $Z$ are independent $N_p(0,I_p)$. If $P$ is a fixed symmetric idempotent $n\times n$ matrix of rank $r$, then $$ XPX'\sim W_p\!\left(r,\Sigma,\Lambda_P\right), \qquad \Lambda_P=\Sigma^{-1/2}MPM'\Sigma^{-1/2}. $$

Proof. Choose an $n\times r$ matrix $U$ with orthonormal columns spanning the range of $P$. Then $P=UU'$ and $XPX'=(XU)(XU)'$. The columns of $ZU$ are independent $N_p(0,I_p)$: each row of $Z$ is standard Gaussian in $\mathbb R^n$, and multiplication by $U$ gives standard Gaussian coordinates in an orthonormal basis. The columns of $XU=MU+\Sigma^{1/2}ZU$ are therefore independent Gaussian vectors with covariance $\Sigma$ and mean columns $MU$. Their aggregate whitened noncentrality is $(\Sigma^{-1/2}MU)(\Sigma^{-1/2}MU)'=\Sigma^{-1/2}MPM'\Sigma^{-1/2}$. The definition of the noncentral Wishart law now gives the result. $\square$

Worked Example: Centering a Sample With a Common Mean

Suppose $n=3$ observations are independent with $X_i\sim N_2(\mu,\Sigma)$, where $\mu=(1,-2)'$ and $\Sigma=\begin{pmatrix}1&0.3\\0.3&2\end{pmatrix}$. Let $H=I_3-\mathbf 1\mathbf 1'/3$, and define the centered scatter $S_c=XHX'$.

The centering matrix $H$ is symmetric and idempotent, with rank $2$. The mean matrix is $M=\mu\mathbf 1'$, so $M\mathbf 1=3\mu$, but $H\mathbf 1=0$. Consequently $MHM'=0$, and the projection result gives

S_c\sim W_2(2,\Sigma,0).

Thus centered scatter is central Wishart even though the observations have nonzero common mean. By contrast, the uncentered sum $XX'$ has $n=3$ degrees of freedom and $\Theta=3\mu\mu'=\begin{pmatrix}3&-6\\-6&12\end{pmatrix}$, so it is noncentral. The centering has removed the common mean component, not the random variation governed by $\Sigma$.

What Centering Does—and Does Not—Remove

For arbitrary individual means, centering replaces the aggregate mean contribution $\Theta=MM'$ by $MHM'$, where $H=I_n-\mathbf 1\mathbf 1'/n$. This matrix is zero when all columns of $M$ are the same, because then $M=\mu\mathbf 1'$ and $H\mathbf 1=0$. If the means differ, $MH$ need not be zero, and the centered scatter can still be noncentral. The relevant question is not whether the observations have nonzero means, but whether the mean matrix has a component in the observation directions retained by the scatter.

More generally, $XPX'$ keeps the mean component selected by $P$ and discards the component selected by $I_n-P$. Orthogonal projections therefore offer a direct way to see why different Gaussian scatter matrices can have different degrees of freedom and different noncentrality, even when formed from the same observations.

Common pitfall: “The observations have nonzero means, so their centered sample scatter must be noncentral Wishart.” This overlooks what centering subtracts. With a common mean, $MH=0$, and the centered scatter has the central $W_p(n-1,\Sigma)$ law. Conversely, subtracting the sample mean does not guarantee centrality when the population means vary across observations; then $MHM'$ may be nonzero.

A second parameterization mistake is to confuse $\Theta$ and $\Lambda$. They are related, but they are not interchangeable: $\Theta$ is the unwhitened sum of mean outer products, while the noncentrality parameter used here is $\Sigma^{-1/2}\Theta\Sigma^{-1/2}$. Both left and right factors must be applied to every matrix entry.

Check Your Understanding

Use the definitions and results above to answer the following questions.

For independent $X_i\sim N_p(\mu_i,\Sigma)$, write the definitions of $\Theta$ and the whitened noncentrality matrix $\Lambda$.
If every $\mu_i=0$, what Wishart law does $\sum_i X_iX_i'$ have?
For a fixed symmetric idempotent projection $P$, how do its rank and the mean matrix determine the degrees of freedom and noncentrality of $XPX'$?
Why is the centered scatter central Wishart when all observations share one common mean?
If the individual means differ, what matrix should be checked to determine whether centered scatter remains noncentral?

Noncentral Wishart Matrices From Shifted Gaussian Samples

What You'll Learn