From Gaussian Observations to a Matrix of Scatter
For a scalar observation, squared deviations summarize variation. With \(p\)-dimensional observations, each deviation is a vector, and its outer product records variation in every coordinate and cross-variation between coordinates. Adding these outer products produces a matrix: the scatter matrix.
The Wishart distribution describes a particularly important way to obtain such a matrix. Start with independent Gaussian vectors and add their outer products. The sample size and the covariance matrix of each vector determine the construction. Keeping track of whether vectors are centered at a known mean or at the sample mean is essential: it changes the scatter matrix and the relevant normalization.
In this construction, \(\Sigma\) is the covariance matrix of each Gaussian vector being sampled; it is not itself the covariance matrix of the random matrix \(W\). The integer \(\nu\) counts the independent outer products in the construction. This definition suffices for the sampling constructions considered here; it does not require a density formula.
The matrix \(W\) is symmetric because each \(Z_iZ_i^T\) is symmetric. It is also positive semidefinite: for any fixed \(u\in\mathbb{R}^p\), the quadratic form \(u^TWu\) is a sum of squared scalar projections. These properties are useful checks when calculating scatter matrices.
Scatter Matrices and Covariance Normalizations
Suppose \(X_1,\ldots,X_n\) are \(p\)-dimensional observations. If their population mean \(\mu\) is known, the scatter about that mean is $$ C_\mu=\sum_{i=1}^n (X_i-\mu)(X_i-\mu)^T. $$ If the observations are independent with \(X_i\sim N_p(\mu,\Sigma)\), then \(X_i-\mu\) are independent \(N_p(0,\Sigma)\) vectors. Thus the defining construction gives \(C_\mu\sim W_p(n,\Sigma)\). The scatter matrix is a sum, not an average; dividing by a sample-size quantity produces a covariance estimate instead.
When the mean is not treated as known, the data are often centered at their sample mean \(\bar X=n^{-1}\sum_{i=1}^n X_i\). The corresponding centered scatter is $$ C=\sum_{i=1}^n (X_i-\bar X)(X_i-\bar X)^T. $$ Two common covariance normalizations of this matrix are \(C/(n-1)\), the conventional sample covariance matrix, and \(C/n\), the maximum-likelihood covariance estimate in the Gaussian model when the mean is estimated as well. They share the same centered scatter but are not the same matrix. The distributional derivation for centered Gaussian scatter is distinct from the direct known-mean construction above.
A useful identity makes the role of centering explicit. For any fixed vector \(a\), scatter about \(a\) separates into scatter about the sample mean and a term measuring how far \(a\) is from that mean.
Proof. Write \(X_i-a=(X_i-\bar X)+(\bar X-a)\), and expand each outer product. Summing gives the centered-scatter term, the term \(n(\bar X-a)(\bar X-a)^T\), and two cross terms. Since \(\sum_{i=1}^n(X_i-\bar X)=0\), both cross terms are zero. The displayed identity follows. \(\square\)
This identity shows that centering at the data mean removes the component of scatter attributable to the displacement between \(a\) and \(\bar X\). In particular, taking \(a=\mu\) relates known-mean scatter to centered scatter, but does not make them interchangeable: the additional outer product is generally present.
Worked Example: Scatter About a Known Mean
Suppose the known mean is \(\mu=(0,0)^T\), and three observed deviations from that mean are \(x_1=(1,2)^T\), \(x_2=(-1,1)^T\), and \(x_3=(2,-1)^T\). Find the scatter matrix and the covariance estimate obtained by dividing by \(n=3\).
The three outer products are $$ x_1x_1^T= \begin{pmatrix}1&2\\2&4\end{pmatrix},\qquad x_2x_2^T= \begin{pmatrix}1&-1\\-1&1\end{pmatrix},\qquad x_3x_3^T= \begin{pmatrix}4&-2\\-2&1\end{pmatrix}. $$ Adding corresponding entries gives $$ C_\mu= \begin{pmatrix}1+1+4&2-1-2\\2-1-2&4+1+1\end{pmatrix} = \begin{pmatrix}6&-1\\-1&6\end{pmatrix}. $$
Because the mean is known, dividing this scatter by \(n=3\) gives $$ \frac{C_\mu}{3}= \begin{pmatrix}2&-1/3\\-1/3&2\end{pmatrix}. $$ The off-diagonal entries are the estimated cross-covariances under this known-mean normalization. The scatter itself is the unscaled sum and, under independent Gaussian sampling with covariance \(\Sigma\), has the \(W_2(3,\Sigma)\) construction.
Worked Example: Centering Changes the Scatter
Consider four observations in two dimensions: \(X_1=(2,1)^T\), \(X_2=(4,3)^T\), \(X_3=(3,2)^T\), and \(X_4=(3,2)^T\). Compute the centered scatter, then both common covariance normalizations.
The sample mean is $$ \bar X=\frac{1}{4}\begin{pmatrix}2+4+3+3\\1+3+2+2\end{pmatrix} =\begin{pmatrix}3\\2\end{pmatrix}. $$ The deviations are \((-1,-1)^T\), \((1,1)^T\), \((0,0)^T\), and \((0,0)^T\). Their outer products sum to $$ C= \begin{pmatrix}1&1\\1&1\end{pmatrix} + \begin{pmatrix}1&1\\1&1\end{pmatrix} = \begin{pmatrix}2&2\\2&2\end{pmatrix}. $$
The conventional sample covariance is $$ \frac{C}{n-1}=\frac{1}{3} \begin{pmatrix}2&2\\2&2\end{pmatrix} = \begin{pmatrix}2/3&2/3\\2/3&2/3\end{pmatrix}, $$ whereas the Gaussian maximum-likelihood normalization is $$ \frac{C}{n}=\frac{1}{4} \begin{pmatrix}2&2\\2&2\end{pmatrix} = \begin{pmatrix}1/2&1/2\\1/2&1/2\end{pmatrix}. $$ The difference is entirely the denominator: both estimates use the same centered scatter matrix.
Linear Transformations of the Construction
A change of coordinates transforms every observation vector and therefore transforms its scatter by multiplication on both sides. The following derivation makes this relationship explicit.
Proof. For each \(i\), the transpose of \(AZ_i\) is \(Z_i^TA^T\), so \((AZ_i)(AZ_i)^T=AZ_iZ_i^TA^T\). Summing this equality over \(i\) and factoring out the fixed matrices \(A\) and \(A^T\) gives the displayed identity. For nonsingular square \(A\), linear transformation of a Gaussian vector gives mean zero and covariance \(A I_p A^T=AA^T\); independence is preserved because the transformation is applied separately to each independent vector. This verifies the stated Wishart construction. \(\square\)
Worked Example: Transforming Standard Gaussian Scatter
Let $$ A=\begin{pmatrix}2&0\\1/2&\sqrt{3}/2\end{pmatrix}, \qquad z_1=\begin{pmatrix}1\\0\end{pmatrix}, \qquad z_2=\begin{pmatrix}0\\1\end{pmatrix}. $$ Calculate the transformed vectors and the sum of their outer products.
The transformed vectors are $$ Az_1=\begin{pmatrix}2\\1/2\end{pmatrix}, \qquad Az_2=\begin{pmatrix}0\\\sqrt{3}/2\end{pmatrix}. $$ Their outer products sum to $$ (Az_1)(Az_1)^T+(Az_2)(Az_2)^T = \begin{pmatrix}4&1\\1&1/4\end{pmatrix} + \begin{pmatrix}0&0\\0&3/4\end{pmatrix} = \begin{pmatrix}4&1\\1&1\end{pmatrix}. $$ This equals \(AA^T\), since direct multiplication gives $$ AA^T= \begin{pmatrix}4&1\\1&1/4+3/4\end{pmatrix} = \begin{pmatrix}4&1\\1&1\end{pmatrix}. $$ Thus this particular pair of transformed vectors produces the scale matrix itself as its scatter. Under the random construction with independent standard Gaussian \(Z_1,Z_2\), the corresponding sum is \(W_2(2,AA^T)\); the calculation above illustrates the matrix transformation, not a claim that every random sample has scatter \(AA^T\).
Checking Scatter and Avoiding Convention Errors
For any fixed \(u\in\mathbb{R}^p\), a scatter matrix about a vector \(a\) satisfies $$ u^T\left\{\sum_{i=1}^n(X_i-a)(X_i-a)^T\right\}u = \sum_{i=1}^n \{u^T(X_i-a)\}^2\geq 0. $$ This follows by distributing the matrix products term by term. It proves that scatter is positive semidefinite and supplies a practical calculation check: a computed scatter matrix must be symmetric and cannot have a negative quadratic form.
A common mistake is to call every covariance-related matrix a Wishart matrix with degrees of freedom equal to the denominator used in a covariance estimate. The Wishart construction is for the sum of independent Gaussian outer products. Dividing that sum by \(n\), for example, scales the matrix; it does not change how many outer products were summed in the construction. A second mistake is to use residuals from the sample mean as if they were independent centered Gaussian vectors. The sample-mean residuals are constrained to sum to zero, so the direct independent-vector construction applies immediately to deviations from a known mean, not automatically to centered residuals.
The distinctions can be summarized as follows:
| Matrix | Centering vector | Common normalization |
|---|---|---|
| Known-mean scatter \(C_\mu\) | Known population mean \(\mu\) | \(C_\mu/n\) |
| Centered scatter \(C\) | Sample mean \(\bar X\) | No division; it is the sum of outer products |
| Conventional sample covariance | Sample mean \(\bar X\) | \(C/(n-1)\) |
| Gaussian maximum-likelihood covariance estimate | Estimated sample mean \(\bar X\) | \(C/n\) |
The main discipline is to name the matrix before naming its distribution or normalization: specify the vectors being outer-producted, state their centering point, and distinguish the unscaled scatter from a covariance estimate. These choices prevent most sample-size convention errors.
Check Your Understanding
Use the definitions and calculations above to check your understanding.
- In the construction \(W=\sum_{i=1}^{\nu}Z_iZ_i^T\), what do \(\nu\) and \(\Sigma\) describe?
- For independent Gaussian observations with a known mean, why does the scatter about that mean match the Wishart construction directly?
- State the scatter decomposition about an arbitrary vector \(a\), and explain which term disappears when \(a=\bar X\).
- For a centered scatter matrix \(C\), distinguish the conventional sample covariance from the Gaussian maximum-likelihood covariance estimate.
- Why does dividing a scatter matrix by \(n\) not alter the number of outer products used to construct it?