Spectral Decomposition of Noncentral Quadratic Forms

Why Spectral Decomposition Matters

A quadratic form in a Gaussian vector can look complicated when its matrix has off-diagonal entries. An orthogonal change of coordinates reveals its structure: each eigenvector gives one Gaussian coordinate, and its eigenvalue specifies the weight attached to that coordinate’s square. When the Gaussian vector has a nonzero mean, the coordinate mean determines the corresponding noncentrality.

The result is a distributional representation as a sum of independent, eigenvalue-weighted noncentral chi-square variables. This representation keeps two distinct features visible: the eigenvalues describe the quadratic form, while the projected mean describes its noncentrality.

Definition: A noncentral chi-square random variable with \(r\) degrees of freedom and noncentrality parameter \(\delta\), written \(\chi^2_r(\delta)\), has the distribution of \( \sum_{j=1}^r (Z_j+a_j)^2 \), where \(Z_1,\ldots,Z_r\) are independent \(N(0,1)\) random variables and \(\sum_{j=1}^r a_j^2=\delta\). The degrees of freedom satisfy \(r\geq 1\), and \(\delta\geq 0\). When \(\delta=0\), this is a central chi-square distribution.

The Spectral Representation

Let \(\mathbf X\) be a \(p\)-dimensional Gaussian vector with mean vector \(\boldsymbol\mu\) and identity covariance matrix, and let \(A\) be a real symmetric \(p\times p\) matrix. The identity covariance assumption means that an orthogonal rotation preserves the covariance matrix. The spectral theorem supplies the orthogonal rotation that diagonalizes \(A\).

Theorem: Suppose \(\mathbf X\sim N_p(\boldsymbol\mu,I_p)\), where \(\boldsymbol\mu\in\mathbb R^p\), and \(A\) is a real symmetric \(p\times p\) matrix. Let \(u_1,\ldots,u_p\) be an orthonormal set of eigenvectors of \(A\), with corresponding eigenvalues \(\lambda_1,\ldots,\lambda_p\), and define \( \eta_i=u_i^{\mathsf T}\boldsymbol\mu \). Then the quadratic form \(Q=\mathbf X^{\mathsf T}A\mathbf X\) has the distribution \( Q\ \overset{d}{=}\ \sum_{i=1}^p \lambda_i W_i, \) where \(W_1,\ldots,W_p\) are independent, \(W_i\sim\chi^2_1(\eta_i^2)\). A term with \(\lambda_i=0\) is identically zero after weighting and may be omitted.

In particular, the noncentrality in direction \(u_i\) is the squared mean coordinate \((u_i^{\mathsf T}\boldsymbol\mu)^2\). It is not the eigenvalue, nor the unsquared mean coordinate.

Proof by Orthogonal Coordinates

Form the orthogonal matrix \(U=[u_1\ \cdots\ u_p]\), whose columns are the eigenvectors. Then \(U^{\mathsf T}U=I_p\), and the diagonalization is \( A=U\Lambda U^{\mathsf T}, \) where \(\Lambda=\operatorname{diag}(\lambda_1,\ldots,\lambda_p)\). Define the rotated vector \(\mathbf Y=U^{\mathsf T}\mathbf X\) and the rotated mean \(\boldsymbol\eta=U^{\mathsf T}\boldsymbol\mu\). Its coordinates are \(Y_i=u_i^{\mathsf T}\mathbf X\) and \(\eta_i=u_i^{\mathsf T}\boldsymbol\mu\).

To verify the distribution and independence of the rotated coordinates, the density of \(\mathbf X\) is \( (2\pi)^{-p/2}\exp\{-\tfrac12(\mathbf x-\boldsymbol\mu)^{\mathsf T}(\mathbf x-\boldsymbol\mu)\}. \) Under the change of variables \(\mathbf x=U\mathbf y\), the absolute Jacobian determinant is \(|\det U|=1\), and orthogonality gives \( (U\mathbf y-\boldsymbol\mu)^{\mathsf T}(U\mathbf y-\boldsymbol\mu) =\sum_{i=1}^p(y_i-\eta_i)^2. \) Thus the density of \(\mathbf Y\) factors as \( \prod_{i=1}^p (2\pi)^{-1/2}\exp\{-\tfrac12(y_i-\eta_i)^2\}. \) Each \(Y_i\) is \(N(\eta_i,1)\), and the factorization proves that the coordinates are independent.

The quadratic form in these coordinates is \( Q=\mathbf X^{\mathsf T}U\Lambda U^{\mathsf T}\mathbf X =\mathbf Y^{\mathsf T}\Lambda\mathbf Y =\sum_{i=1}^p\lambda_iY_i^2. \) By the definition of a noncentral chi-square variable, \(Y_i^2\sim\chi^2_1(\eta_i^2)\). Since the \(Y_i\) are independent, their squares are independent as well. Substitution proves the stated representation. \(\square\)

Worked Example: A Diagonal Form With a Zero Eigenvalue

Let \(\mathbf X\sim N_3((1,-4,2)^{\mathsf T},I_3)\), and take \(A=\operatorname{diag}(3,0,2)\). Find the weighted noncentral chi-square representation of \(Q=\mathbf X^{\mathsf T}A\mathbf X\).

The coordinate eigenvectors are \(u_1=(1,0,0)^{\mathsf T}\), \(u_2=(0,1,0)^{\mathsf T}\), and \(u_3=(0,0,1)^{\mathsf T}\), with eigenvalues \(3,0,2\). Their projected mean coordinates are \( \eta_1=1,\qquad \eta_2=-4,\qquad \eta_3=2. \) Therefore, the three noncentrality parameters are \(1^2=1\), \((-4)^2=16\), and \(2^2=4\), respectively.

The middle coordinate has eigenvalue zero, so its squared value makes no contribution to \(Q\), despite its nonzero noncentrality. The remaining independent terms give \( Q\ \overset{d}{=}\ 3W_1+2W_3,\qquad W_1\sim\chi^2_1(1),\quad W_3\sim\chi^2_1(4), \) with \(W_1\) and \(W_3\) independent.

Directly, \(Q=3X_1^2+2X_3^2\). The representation has the same two coordinate squares, with the eigenvalues as their weights and the squared means as their noncentralities.

Combining Repeated Eigenvalues

When an eigenvalue is repeated, its eigenspace has several orthogonal coordinates with the same weight. Their squared coordinates can be combined into a single noncentral chi-square variable. The degrees of freedom equal the dimension of the eigenspace, and the noncentrality is the squared length of the projection of the mean onto that eigenspace.

Result: Let \(\lambda\) be an eigenvalue of \(A\), let \(E_\lambda\) be its eigenspace, and let \(m_\lambda=\dim(E_\lambda)\). Write \(P_\lambda\) for orthogonal projection onto \(E_\lambda\). The sum of the quadratic-form terms from an orthonormal basis of \(E_\lambda\) has distribution \( \lambda\,V_\lambda,\qquad V_\lambda\sim\chi^2_{m_\lambda}\bigl(\|P_\lambda\boldsymbol\mu\|^2\bigr). \) The variables \(V_\lambda\) for distinct eigenvalues are independent.

To see why the noncentrality has this form, choose an orthonormal eigenbasis \(u_1,\ldots,u_{m_\lambda}\) for \(E_\lambda\). The contribution before weighting is \(\sum_{j=1}^{m_\lambda}Y_j^2\), with independent \(Y_j\sim N(u_j^{\mathsf T}\boldsymbol\mu,1)\). By the definition of the noncentral chi-square distribution, this sum has \(m_\lambda\) degrees of freedom and noncentrality \( \sum_{j=1}^{m_\lambda}(u_j^{\mathsf T}\boldsymbol\mu)^2 =\|P_\lambda\boldsymbol\mu\|^2. \) Multiplication by the common eigenvalue gives the stated weighted contribution. Eigenspaces for distinct eigenvalues use disjoint coordinates in an orthonormal eigenbasis; those coordinates are independent by the density factorization in the proof above. Hence the grouped variables are independent.

Worked Example: Grouping a Repeated Eigenvalue

Suppose \(\mathbf X\sim N_4((1,2,1,3)^{\mathsf T},I_4)\) and \(A=\operatorname{diag}(4,4,-1,0)\). Group the terms by eigenvalue.

The eigenvalue \(4\) has multiplicity two. Its two coordinate means are \(1\) and \(2\), so the combined noncentrality is \(1^2+2^2=5\). The contribution from these coordinates is therefore \(4V\), where \(V\sim\chi^2_2(5)\).

The eigenvalue \(-1\) has multiplicity one and coordinate mean \(1\), giving an independent term \(-W\), where \(W\sim\chi^2_1(1)\). The eigenvalue zero multiplies the final coordinate square by zero, so the mean \(3\) in that direction does not affect the form.

Consequently, \( Q\ \overset{d}{=}\ 4V-W,\qquad V\sim\chi^2_2(5),\quad W\sim\chi^2_1(1), \) where \(V\) and \(W\) are independent. The two degrees of freedom for \(V\) come from the dimension of the eigenvalue-\(4\) eigenspace, not from the size of its noncentrality.

Worked Example: Diagonalizing a Nondiagonal Matrix

Let \(\mathbf X\sim N_2((2,0)^{\mathsf T},I_2)\) and \( A=\begin{pmatrix}2&1\\1&2\end{pmatrix}. \) Find the representation using the eigenvectors of \(A\).

The unit eigenvectors \(u_1=(1,1)^{\mathsf T}/\sqrt2\) and \(u_2=(1,-1)^{\mathsf T}/\sqrt2\) have eigenvalues \(3\) and \(1\), respectively. Projecting the mean onto these directions gives \( \eta_1=u_1^{\mathsf T}\boldsymbol\mu=\sqrt2,\qquad \eta_2=u_2^{\mathsf T}\boldsymbol\mu=\sqrt2. \) Both noncentrality parameters are therefore \(2\). The rotated coordinates are independent, so \( Q\ \overset{d}{=}\ 3W_1+W_2,\qquad W_1,W_2\ \text{independent},\quad W_1,W_2\sim\chi^2_1(2). \)

For a direct check, the original form is \(Q=2X_1^2+2X_2^2+2X_1X_2\). Substituting \(Y_1=(X_1+X_2)/\sqrt2\) and \(Y_2=(X_1-X_2)/\sqrt2\) gives \(Q=3Y_1^2+Y_2^2\), as predicted by the eigenvalues.

Common Pitfalls and Interpretation

The spectral representation separates the geometry of the quadratic form from the location of the Gaussian distribution. The eigenvalues are weights: they multiply the squared Gaussian coordinates. The noncentrality parameters are determined by the mean’s projections onto the corresponding eigendirections. Keeping these roles separate makes the representation useful in calculations and in interpreting which directions affect the form.

Using the projected mean instead of its square. If \(\eta_i=u_i^{\mathsf T}\boldsymbol\mu\), the noncentrality is \(\eta_i^2\), which is nonnegative even when the coordinate mean is negative.
Putting the eigenvalue inside the noncentrality. The term is \(\lambda_i W_i\), with \(W_i\sim\chi^2_1(\eta_i^2)\). The eigenvalue weights the random variable; it does not alter its noncentrality parameter.
Ignoring multiplicity. A repeated eigenvalue contributes one chi-square variable with degrees of freedom equal to the eigenspace dimension when its terms are grouped.
Keeping a zero-eigenvalue term as a contribution. Its coordinate can have a nonzero mean and a noncentral chi-square square, but multiplication by zero makes its contribution to the form exactly zero.

The resulting representation can include positive, negative, or zero weights, depending on the eigenvalues. The decomposition itself does not require positive definiteness; it requires symmetry so that an orthonormal eigenbasis exists.

Key takeaway: In orthogonal eigen-coordinates, \(Q=\mathbf X^{\mathsf T}A\mathbf X\) is a sum of independent weighted squares. Each direction contributes its eigenvalue as the weight and the squared projected mean as the noncentrality; repeated eigenvalues can be grouped by eigenspace.

Check Your Understanding

Use the orthogonal eigen-coordinates to answer each question.

If an eigenvector \(u_i\) has eigenvalue \(\lambda_i\), what are the weight and noncentrality of its contribution?
For a repeated eigenvalue, how are the degrees of freedom and noncentrality of the grouped variable determined?
Why are the chi-square variables associated with distinct eigenvectors independent in this setting?
A zero eigenvalue has an eigenvector whose mean projection is nonzero. Does its coordinate contribute to the quadratic form? Explain.
For a two-dimensional eigenspace with an orthonormal basis \(u_1,u_2\), write the noncentrality in terms of the two mean projections.