Weighted Chi-Square Representation of Central Forms

From a Quadratic Form to a Distribution

A quadratic form $Z^{\mathsf T}AZ$ can look complicated when its matrix has off-diagonal entries. For a standard normal vector $Z$, the spectral decomposition provides a coordinate system in which the form becomes a sum of squares. Those squares are independent, and each has a chi-square distribution with one degree of freedom. The matrix’s eigenvalues determine the weights in that sum.

This representation applies whether the form is positive, negative, or indefinite. A zero eigenvalue contributes nothing; a negative eigenvalue gives a negative weight. The result is therefore more general than a sum of positively weighted squares.

Setup and Orthogonal Invariance

Definition: Let $Z=(Z_1,\ldots,Z_p)^{\mathsf T}$, where $Z_1,\ldots,Z_p$ are independent $N(0,1)$ random variables. For a real symmetric $p\times p$ matrix $A$, the random variable $Q=Z^{\mathsf T}AZ$ is a central Gaussian quadratic form. It is central because the normal vector has mean zero.

A matrix $U$ is orthogonal if $U^{\mathsf T}U=I_p$. Orthogonal matrices change coordinates while preserving Euclidean lengths. The corresponding distributional fact is just as important: applying an orthogonal matrix to a standard normal vector leaves its distribution unchanged.

Lemma (orthogonal invariance): If $Z\sim N_p(0,I_p)$ and $U$ is a fixed orthogonal $p\times p$ matrix, then $U^{\mathsf T}Z\sim N_p(0,I_p)$. In particular, the coordinates of $U^{\mathsf T}Z$ are independent $N(0,1)$ random variables.

Proof. The joint density of $Z$ at $z\in\mathbb R^p$ is

f_Z(z)=(2\pi)^{-p/2}\exp\left(-\frac{z^{\mathsf T}z}{2}\right).

Set $Y=U^{\mathsf T}Z$, so $Z=UY$. The absolute Jacobian determinant of this change of variables is $|\det(U)|=1$, since $U^{\mathsf T}U=I_p$. Also,

(Uy)^{\mathsf T}(Uy)=y^{\mathsf T}U^{\mathsf T}Uy=y^{\mathsf T}y.

Therefore the density of $Y$ at $y$ is

f_Y(y)=(2\pi)^{-p/2}\exp\left(-\frac{y^{\mathsf T}y}{2}\right) =\prod_{i=1}^p\left[(2\pi)^{-1/2}\exp\left(-\frac{y_i^2}{2}\right)\right].

This is the product of $p$ standard normal densities, proving both the claimed joint distribution and independence of the coordinates. $\square$

The Weighted Chi-Square Representation

Because $A$ is real and symmetric, the spectral theorem gives an orthogonal matrix $U$ and real eigenvalues $\lambda_1,\ldots,\lambda_p$ such that $A=U\Lambda U^{\mathsf T}$, where $\Lambda=\operatorname{diag}(\lambda_1,\ldots,\lambda_p)$. The eigenvalues are listed with multiplicity. Define $Y=U^{\mathsf T}Z$. By orthogonal invariance, the entries of $Y$ are independent standard normal variables.

Theorem (weighted chi-square representation): Let $Z\sim N_p(0,I_p)$, and let $A$ be a real symmetric $p\times p$ matrix with eigenvalues $\lambda_1,\ldots,\lambda_p$, counted with multiplicity. Then $$ Z^{\mathsf T}AZ\ \overset{d}{=}\ \sum_{i=1}^p \lambda_i X_i, $$ where $X_1,\ldots,X_p$ are independent chi-square random variables, each with one degree of freedom. Equivalently, $X_i=Y_i^2$ for independent $Y_i\sim N(0,1)$. A zero eigenvalue gives a zero term, and a negative eigenvalue gives a negatively weighted term.

Proof. Use the spectral decomposition and let $Y=U^{\mathsf T}Z$. Since $Z=UY$,

Z^{\mathsf T}AZ =Z^{\mathsf T}U\Lambda U^{\mathsf T}Z =Y^{\mathsf T}\Lambda Y =\sum_{i=1}^p \lambda_iY_i^2.

The lemma shows that $Y_1,\ldots,Y_p$ are independent $N(0,1)$ variables. By definition, the square of a standard normal variable has a chi-square distribution with one degree of freedom. Thus $X_i=Y_i^2$ are independent $\chi^2_1$ variables, and the displayed identity gives the claimed distribution. $\square$

The theorem is an equality in distribution, not a claim that the original coordinates $Z_i^2$ are weighted by the eigenvalues in their original order. The squares in the representation are the squares of the coordinates after rotating into an eigenvector basis.

Worked Representations

Worked Example: A Positive-Definite Form

Let $Z=(Z_1,Z_2)^{\mathsf T}\sim N_2(0,I_2)$, and let

A=\begin{pmatrix}2&1\\1&2\end{pmatrix}.

The characteristic polynomial is

\det(A-\lambda I_2) =(2-\lambda)^2-1 =(1-\lambda)(3-\lambda).

Thus the eigenvalues are $3$ and $1$. Corresponding unit eigenvectors are $(1,1)^{\mathsf T}/\sqrt{2}$ and $(1,-1)^{\mathsf T}/\sqrt{2}$. Take

U=\frac{1}{\sqrt{2}}\begin{pmatrix}1&1\\1&-1\end{pmatrix}, \qquad Y=U^{\mathsf T}Z =\begin{pmatrix}(Z_1+Z_2)/\sqrt{2}\\(Z_1-Z_2)/\sqrt{2}\end{pmatrix}.

The coordinates $Y_1,Y_2$ are independent $N(0,1)$, so

Z^{\mathsf T}AZ=2Z_1^2+2Z_1Z_2+2Z_2^2 =3Y_1^2+Y_2^2 \ \overset{d}{=}\ 3X_1+X_2,

where $X_1,X_2$ are independent $\chi^2_1$ variables. The coefficients are the eigenvalues $3$ and $1$, not the diagonal entries $2$ and $2$.

Worked Example: An Indefinite Form

Let

A=\begin{pmatrix}0&1\\1&0\end{pmatrix}, \qquad Z\sim N_2(0,I_2).

The characteristic polynomial is $\lambda^2-1$, so the eigenvalues are $1$ and $-1$. Unit eigenvectors for these eigenvalues are $(1,1)^{\mathsf T}/\sqrt{2}$ and $(1,-1)^{\mathsf T}/\sqrt{2}$, respectively. With the corresponding orthogonal matrix $U$, the transformed coordinates are $Y_1=(Z_1+Z_2)/\sqrt{2}$ and $Y_2=(Z_1-Z_2)/\sqrt{2}$. Hence

Z^{\mathsf T}AZ=2Z_1Z_2 =Y_1^2-Y_2^2 \ \overset{d}{=}\ X_1-X_2,

where $X_1$ and $X_2$ are independent $\chi^2_1$ variables. The negative eigenvalue produces a negative weight. The representation is valid even though the form is not nonnegative for every vector: for instance, $z=(1,-1)^{\mathsf T}$ gives $z^{\mathsf T}Az=-2$, while $z=(1,1)^{\mathsf T}$ gives $z^{\mathsf T}Az=2$.

Worked Example: A Singular Form With a Zero Weight

Let $Z=(Z_1,Z_2,Z_3)^{\mathsf T}\sim N_3(0,I_3)$ and

A=\begin{pmatrix}4&0&0\\0&0&0\\0&0&1\end{pmatrix}.

This matrix is already diagonal, so its eigenvalues are $4,0,1$, with the coordinate axes as eigenvectors. Direct substitution gives

Z^{\mathsf T}AZ=4Z_1^2+0Z_2^2+Z_3^2.

The variables $Z_1^2$ and $Z_3^2$ are independent $\chi^2_1$ variables. Therefore

Z^{\mathsf T}AZ\ \overset{d}{=}\ 4X_1+X_3,

where $X_1,X_3$ are independent $\chi^2_1$ variables. The term with coefficient zero is identically zero, so it can be omitted from the sum. The matrix is singular, but singularity does not prevent the representation.

What the Weights Tell You

The eigenvalues describe how the quadratic form acts along mutually perpendicular directions. In an eigenvector coordinate system, each direction contributes its squared standard normal coordinate multiplied by the eigenvalue for that direction. This is why the weights arise from the spectrum of $A$.

The representation also makes the role of repeated and zero eigenvalues explicit. If an eigenvalue occurs more than once, it appears once for each occurrence, with an independent $\chi^2_1$ term. If an eigenvalue is zero, its coordinate contributes nothing. No assumption that $A$ is invertible or positive definite is needed; symmetry is the condition that ensures an orthogonal diagonalization with real eigenvalues.

This form of the distribution is useful because it translates a matrix problem into a sum of familiar one-dimensional random variables. It is a starting point for further analysis of Gaussian quadratic forms. Here, the essential conclusion is the distributional identity itself, including the independence of its chi-square components.

Common Pitfall

Common mistake: Read the diagonal entries of $A$ as the weights of the chi-square terms, or assume that every term must have a positive weight.

Correction: The weights are the eigenvalues, which need not equal the diagonal entries in the original coordinates. Diagonalize $A$ orthogonally first. For example, in the first worked example the diagonal entries are $2,2$, but the eigenvalues are $3,1$. An indefinite matrix can also have negative eigenvalues, as the second example shows.

Check Your Understanding

For each question, use orthogonal diagonalization and keep track of the eigenvalues with their multiplicities.

What property of the transformed vector ensures that the squared coordinates in the representation are independent?
For a symmetric matrix with eigenvalues $5,2,0$, write the distributional representation of its central quadratic form in a standard normal vector.
Can a central quadratic form have a negative value? Explain what the eigenvalues imply.
Why are the diagonal entries of a symmetric matrix not generally the weights in the representation?
What contribution does a zero eigenvalue make to the weighted sum?