Chi-Square Laws From Wishart Quadratic Forms

Why the Direction Matters

A Wishart matrix summarizes variation across several coordinates. One way to inspect that variation is to choose a direction $a$ and evaluate $a'Wa$. If $a$ is fixed in advance, this quantity has a simple distribution: after dividing by its scale, it is chi-square. But the conclusion relies on the direction being fixed independently of the observed matrix. Choosing a direction after seeing $W$, for example to maximize $a'Wa$, can change the distribution substantially.

The same distinction appears for scalar summaries that involve several directions at once. A general linear projection of the entries of a Wishart matrix is usually a weighted sum of chi-square variables, not necessarily one chi-square variable. The weights determine whether the sum simplifies.

Definition: Let $Z_1,\ldots,Z_\nu$ be independent $p$-dimensional random vectors with distribution $N_p(0,\Sigma)$, where $\Sigma$ is positive definite and $\nu$ is a positive integer. The central Wishart matrix with $\nu$ degrees of freedom and scale matrix $\Sigma$ is $$ W=\sum_{i=1}^{\nu} Z_i Z_i' \sim W_p(\nu,\Sigma). $$ For a fixed nonzero vector $a\in\mathbb{R}^p$, the directional quadratic form is $a'Wa$. Its natural scale is $a'\Sigma a>0$.

A Fixed Direction Gives a Chi-Square Law

The key is to examine the Gaussian observations one at a time. Each $a'Z_i$ is a scalar normal variable with mean zero and variance $a'\Sigma a$. Since the $Z_i$ are independent, these scalar projections are independent as well. The Wishart quadratic form is their sum of squares.

Theorem: If $W\sim W_p(\nu,\Sigma)$ is constructed from independent $N_p(0,\Sigma)$ vectors as above, and $a\ne 0$ is fixed, then $$ \frac{a'Wa}{a'\Sigma a}\sim\chi^2_\nu. $$ Equivalently, $a'Wa$ has the distribution of $(a'\Sigma a)\chi^2_\nu$, where the latter notation means a chi-square random variable multiplied by the stated positive constant.

Proof. By the Wishart construction,

a'Wa =a'\left(\sum_{i=1}^{\nu}Z_iZ_i'\right)a =\sum_{i=1}^{\nu}(a'Z_i)^2.

For each $i$, a linear projection of a multivariate normal vector is normal, so $a'Z_i\sim N(0,a'\Sigma a)$. Thus $U_i=(a'Z_i)/\sqrt{a'\Sigma a}\sim N(0,1)$. Independence of the $Z_i$'s implies independence of the $U_i$'s. Therefore

\frac{a'Wa}{a'\Sigma a} =\sum_{i=1}^{\nu}U_i^2 \sim\chi^2_\nu,

by the definition of the chi-square distribution as a sum of squares of $\nu$ independent standard normal variables. $\square$

Worked Example: A Fixed Linear Combination of Coordinates

Suppose $W\sim W_3(6,\Sigma)$, where $\Sigma=\operatorname{diag}(1,2,3)$, and take the fixed direction $a=(1,1,0)'$. Find the distribution of $a'Wa$.

First calculate its scale:

a'\Sigma a =(1,1,0) \begin{pmatrix}1&0&0\\0&2&0\\0&0&3\end{pmatrix} \begin{pmatrix}1\\1\\0\end{pmatrix} =1+2=3.

The fixed-direction theorem now gives

\frac{a'Wa}{3}\sim\chi^2_6, \qquad\text{or equivalently}\qquad a'Wa\overset{d}=3\chi^2_6.

For a check, $E[a'Wa]=3\cdot 6=18$, which also agrees with $E[W]=6\Sigma$ and $a'E[W]a=6(a'\Sigma a)=18$.

General Scalar Projections of a Wishart Matrix

The expression $a'Wa$ is one particular scalar summary of a matrix. Any symmetric matrix $C$ defines another scalar summary, $\operatorname{tr}(CW)$, which is a linear combination of the entries of $W$. A rank-one choice $C=aa'$ recovers the directional form, since $\operatorname{tr}(aa'W)=a'Wa$. The distribution for general $C$ can be read from the eigenvalues of a transformed version of $C$.

Theorem: Let $W\sim W_p(\nu,\Sigma)$, with $\Sigma$ positive definite, and let $C$ be a fixed real symmetric $p\times p$ matrix. Write $\lambda_1,\ldots,\lambda_p$ for the eigenvalues, including zeros, of $M=\Sigma^{1/2}C\Sigma^{1/2}$. Then $$ \operatorname{tr}(CW)\overset{d}=\sum_{j=1}^{p}\lambda_j Q_j, $$ where $Q_1,\ldots,Q_p$ are independent $\chi^2_\nu$ random variables. In particular, if $C$ is positive semidefinite, the $\lambda_j$'s are nonnegative. If its $r$ nonzero eigenvalues all equal the same positive value $\lambda$, then $\operatorname{tr}(CW)\sim\lambda\chi^2_{r\nu}$.

Derivation. Write $Z_i=\Sigma^{1/2}G_i$, where the vectors $G_i$ are independent $N_p(0,I_p)$. Collect the $G_i$'s as columns of a $p\times\nu$ matrix $G$. Then $W=\Sigma^{1/2}GG'\Sigma^{1/2}$, and the cyclic property of trace gives

\operatorname{tr}(CW) =\operatorname{tr}\!\left(G'\Sigma^{1/2}C\Sigma^{1/2}G\right) =\operatorname{tr}(G'MG).

Because $M$ is symmetric, the spectral theorem gives an orthogonal matrix $U$ such that $U'MU=\operatorname{diag}(\lambda_1,\ldots,\lambda_p)$. The matrix $U'G$ has independent standard normal entries: each column is a standard multivariate normal vector after an orthogonal transformation, and the columns remain independent. Denote its entries by $H_{ji}$. It follows that

\operatorname{tr}(G'MG) =\sum_{i=1}^{\nu}\sum_{j=1}^{p}\lambda_j H_{ji}^2 =\sum_{j=1}^{p}\lambda_j\left(\sum_{i=1}^{\nu}H_{ji}^2\right).

For each $j$, the inner sum is $\chi^2_\nu$. Different inner sums use different rows of $U'G$, so they are independent. This proves the weighted-sum representation. If exactly $r$ eigenvalues equal $\lambda>0$ and the rest are zero, the sum is $\lambda$ times the sum of $r$ independent $\chi^2_\nu$ variables, which is $\lambda\chi^2_{r\nu}$. $\square$

Worked Example: A Projection That Is a Scaled Chi-Square

Let $W\sim W_2(3,\Sigma)$, with $\Sigma=\operatorname{diag}(1,4)$, and set $C=\operatorname{diag}(1,1/4)$. Determine the distribution of $\operatorname{tr}(CW)$.

Here $\Sigma^{1/2}=\operatorname{diag}(1,2)$. Thus

M=\Sigma^{1/2}C\Sigma^{1/2} = \begin{pmatrix}1&0\\0&2\end{pmatrix} \begin{pmatrix}1&0\\0&1/4\end{pmatrix} \begin{pmatrix}1&0\\0&2\end{pmatrix} =I_2.

Both eigenvalues are $1$, so the theorem gives

\operatorname{tr}(CW)\overset{d}=Q_1+Q_2\sim\chi^2_6, \qquad Q_1,Q_2\ \text{independent and each }\chi^2_3.

The degrees of freedom are $6$, not $3$, because this projection combines two independent coordinate contributions, each with three degrees of freedom.

Worked Example: An Ordinary Trace Need Not Be Chi-Square

Let $W\sim W_2(4,\operatorname{diag}(1,4))$. Find a distributional representation for $\operatorname{tr}(W)$.

Take $C=I_2$, so $M=\Sigma=\operatorname{diag}(1,4)$, with eigenvalues $1$ and $4$. The projection theorem yields

\operatorname{tr}(W)\overset{d}=Q_1+4Q_2, \qquad Q_1,Q_2\ \text{independent and each }\chi^2_4.

This is a weighted sum, not the unweighted sum that would give a chi-square variable with eight degrees of freedom. Its mean is $4+4(4)=20$, and its variance is $2(4)+16\cdot 2(4)=136$. In contrast, a $\chi^2_8$ variable has mean $8$ and variance $16$. The scaling here is determined by $\Sigma$, and ignoring it would give the wrong law.

Independent Random Directions and Data-Dependent Directions

The fixed-direction result also applies to a random direction chosen independently of $W$. More precisely, suppose $A$ is a random nonzero vector independent of $W$. Conditional on $A=a$, the vector is fixed, so

\left.\frac{A'WA}{A'\Sigma A}\right|\,A=a\sim\chi^2_\nu.

This conditional distribution is the same for every nonzero $a$. Averaging over the distribution of $A$ therefore leaves the ratio marginally $\chi^2_\nu$ as well. Independence matters: if $A$ is computed from $W$, conditioning on $A$ does not leave the original fixed-direction argument intact.

Worked Example: Selecting the Leading Eigenvector

Let $W\sim W_2(3,I_2)$. Choose $A$ to be a unit eigenvector corresponding to the largest eigenvalue $\lambda_{\max}(W)$. Show that $A'WA$ does not have a $\chi^2_3$ distribution.

Since $A$ is a unit eigenvector for $\lambda_{\max}(W)$, its quadratic form is exactly

A'WA=\lambda_{\max}(W).

The matrix $W$ is positive definite with probability one because it is formed from three independent, nondegenerate Gaussian vectors in dimension two. Its Wishart distribution has a density on the positive-definite matrices. Repeated eigenvalues in dimension two require $(W_{11}-W_{22})^2+4W_{12}^2=0$, a set of Lebesgue measure zero. Therefore its eigenvalues are distinct with probability one.

For any positive-definite $2\times2$ matrix, the largest eigenvalue satisfies

\lambda_{\max}(W)\geq\frac{\lambda_{\max}(W)+\lambda_{\min}(W)}{2} =\frac{\operatorname{tr}(W)}{2},

with strict inequality when the eigenvalues are distinct. In this example, the inequality is strict with probability one. The difference is integrable because $0\leq\lambda_{\max}(W)\leq\operatorname{tr}(W)$, and $\operatorname{tr}(W)$ has finite expectation. Also, $E[\operatorname{tr}(W)]=\operatorname{tr}(3I_2)=6$. Hence

E[A'WA]=E[\lambda_{\max}(W)] >E\!\left[\frac{\operatorname{tr}(W)}{2}\right] =3.

A $\chi^2_3$ variable has mean $3$. Thus $A'WA$ cannot have the fixed-direction $\chi^2_3$ law. Selecting the direction to maximize the quadratic form has made its value larger, on average, than the corresponding average over two coordinates.

Common Pitfall and Practical Interpretation

Incorrect reasoning: “For every fixed unit vector $a$, $a'Wa$ has the same chi-square law, so the result still holds if I choose $a$ to be the leading eigenvector of $W$.” The fixed-vector theorem quantifies over vectors specified independently of the observed matrix. It does not assert that the conditional distribution remains the same after selecting a vector using $W$. The leading-eigenvector example shows the consequence: the selected quadratic form is $\lambda_{\max}(W)$, whose mean is strictly greater than the mean of the fixed-direction chi-square law in that example.

A second common error is to treat every scalar summary of a Wishart matrix as a single chi-square variable. For $\operatorname{tr}(CW)$, first calculate the eigenvalues of $\Sigma^{1/2}C\Sigma^{1/2}$. Equal positive eigenvalues give a scaled chi-square after combining the independent components; unequal eigenvalues generally give a weighted sum instead. A rank-one projection is especially simple because only one eigenvalue is nonzero, which recovers the fixed-direction result.

These distinctions are useful whenever a scalar statistic is extracted from a multivariate scatter matrix. A direction specified by the scientific question has the fixed-direction law, and an independent direction can be handled by conditioning. A direction estimated from the same scatter matrix requires a separate distributional argument; the fixed-direction theorem alone does not justify its law.

Key takeaway: For fixed nonzero $a$, $a'Wa/(a'\Sigma a)\sim\chi^2_\nu$. A general fixed symmetric projection $\operatorname{tr}(CW)$ is a weighted sum of independent chi-square variables, while a direction selected from $W$ need not obey either fixed-direction conclusion.

Check Your Understanding

Use the definitions and results above to answer these questions.

If $W\sim W_p(\nu,\Sigma)$ and $a\ne0$ is fixed, what is the scale needed to turn $a'Wa$ into a $\chi^2_\nu$ variable?
For a fixed symmetric $C$, which matrix's eigenvalues determine the distribution of $\operatorname{tr}(CW)$?
Suppose the nonzero eigenvalues of $\Sigma^{1/2}C\Sigma^{1/2}$ are all equal to $2$, with exactly three such eigenvalues. What is the resulting distribution of $\operatorname{tr}(CW)$?
Why does the fixed-direction theorem apply to a random direction independent of $W$, but not automatically to a direction chosen as a leading eigenvector of $W$?
For $W\sim W_2(3,I_2)$, explain why selecting a unit leading eigenvector gives a quadratic form whose expectation exceeds $3$.