Diagnosing Matrix Errors in Gaussian Quadratic Forms

Why Matrix Bookkeeping Matters

A Gaussian quadratic form can look simple—$X'AX$—while its distribution depends on several different matrices: the matrix actually represented by the quadratic form, the covariance of $X$, and the matrix obtained after transforming to standard normal coordinates. Confusing these objects can produce an incorrect chi-square degree of freedom, the wrong weighted sum, or an unjustified claim of independence.

The central discipline is to keep the transformations explicit. First identify the symmetric matrix that determines the scalar form. Then account for the covariance before using eigenvalues. Finally, distinguish a covariance calculation from an independence argument.

Definition: For a real $p\times p$ matrix $A$, the quadratic form in a real random vector $X\in\mathbb{R}^p$ is the scalar $Q=X'AX$. Its symmetric part is $A_s=(A+A')/2$. The matrix $A_s$, not the skew-symmetric part of $A$, determines $Q$.

Error 1: Treating a Nonsymmetric Matrix as the Quadratic-Form Matrix

The expression $X'AX$ is a scalar, so transposing it does not change it: $X'AX=(X'AX)'=X'A'X$. Averaging these two expressions gives $X'AX=X'A_sX$. Consequently, the eigenvalues and rank relevant to the quadratic form are those of $A_s$, not necessarily those of the displayed matrix $A$.

Result: For every real vector $x$ and real square matrix $A$, $$ x'Ax=x'\frac{A+A'}{2}x. $$

Proof. The quantity $x'Ax$ is a scalar, hence equals its transpose $x'A'x$. Adding the two equal expressions and dividing by two gives $x'[(A+A')/2]x$, as required. $\square$

Worked Example: A Full-Rank Matrix That Gives the Zero Form

Let $$ A=\begin{pmatrix}0&1\\-1&0\end{pmatrix},\qquad X=\begin{pmatrix}X_1\\X_2\end{pmatrix}. $$ The matrix is nonsingular because $\det(A)=1$. But its symmetric part is $$ A_s=\frac12\begin{pmatrix}0&1\\-1&0\end{pmatrix} +\frac12\begin{pmatrix}0&-1\\1&0\end{pmatrix} =\begin{pmatrix}0&0\\0&0\end{pmatrix}. $$ Directly, $X'AX=X_1X_2-X_2X_1=0$ for every $X$. Thus the quadratic form has no nonzero eigenvalues and is identically zero, despite the original matrix having rank two. Using $\operatorname{rank}(A)=2$ as a chi-square degree of freedom would be an error.

Error 2: Diagonalizing Before Accounting for Covariance

Eigenvalues of $A_s$ describe the form in the coordinates of $X$. If $X$ has a nonidentity covariance matrix, those are generally not the weights in the distribution. Suppose $X\sim N(\mu,\Sigma)$, with $\Sigma$ positive definite. Let $\Sigma^{1/2}$ denote its symmetric positive-definite square root, and set $Z=\Sigma^{-1/2}(X-\mu)$. Then $Z\sim N(0,I_p)$. The matrix that governs the standardized quadratic form is $C=\Sigma^{1/2}A_s\Sigma^{1/2}$.

Theorem: If $X\sim N(\mu,\Sigma)$ with $\Sigma$ positive definite, then for $Q=X'AX$, define $A_s=(A+A')/2$, $C=\Sigma^{1/2}A_s\Sigma^{1/2}$, and $m=\Sigma^{-1/2}\mu$. If $\lambda_1,\ldots,\lambda_p$ are the eigenvalues of $C$, and $\alpha_1,\ldots,\alpha_p$ are the coordinates of $m$ in an orthonormal eigenbasis of $C$, then $$ Q\ \overset{d}{=}\ \sum_{i=1}^p \lambda_i(U_i+\alpha_i)^2, $$ where $U_1,\ldots,U_p$ are independent $N(0,1)$ random variables. In particular, if $\mu=0$, then $Q\overset{d}{=}\sum_{i=1}^p\lambda_iU_i^2$.

Proof. The symmetric-part identity gives $Q=X'A_sX$. Write $X=\mu+\Sigma^{1/2}Z=\Sigma^{1/2}(Z+m)$, where $Z\sim N(0,I_p)$. It follows that $$ Q=(Z+m)'\Sigma^{1/2}A_s\Sigma^{1/2}(Z+m)=(Z+m)'C(Z+m). $$ By the spectral theorem, $C=V\Lambda V'$, where $V$ is orthogonal and $\Lambda$ is diagonal with entries $\lambda_i$. The vector $U=V'Z$ is again standard multivariate normal, so its coordinates are independent standard normal variables; set $\alpha=V'm$. Substitution yields $Q=(U+\alpha)'\Lambda(U+\alpha)=\sum_i\lambda_i(U_i+\alpha_i)^2$. When $\mu=0$, $m=0$ and each $\alpha_i=0$. $\square$

Worked Example: A Missing Covariance Transformation Changes the Weights

Let $X\sim N(0,\Sigma)$, where $\Sigma=\operatorname{diag}(4,1)$, and let $A=\operatorname{diag}(1,1)$. The tempting calculation uses the two eigenvalues of $A$, both equal to one, and concludes that $X'AX\sim\chi^2_2$. But $A_s=A$ and $\Sigma^{1/2}=\operatorname{diag}(2,1)$, so $$ C=\Sigma^{1/2}A_s\Sigma^{1/2} =\begin{pmatrix}2&0\\0&1\end{pmatrix} \begin{pmatrix}1&0\\0&1\end{pmatrix} \begin{pmatrix}2&0\\0&1\end{pmatrix} =\operatorname{diag}(4,1). $$ Therefore $Q=X_1^2+X_2^2\overset{d}{=}4U_1^2+U_2^2$, not $U_1^2+U_2^2$. The first coordinate has variance four, so its square contributes four times a one-degree-of-freedom chi-square variable. The covariance transformation is essential even though $A$ is already symmetric.

Error 3: Reading Degrees of Freedom from the Wrong Rank

For a central Gaussian vector with positive-definite covariance, the rank of $C=\Sigma^{1/2}A_s\Sigma^{1/2}$ is the number of nonzero weights in the weighted chi-square representation. Because $\Sigma^{1/2}$ is invertible in this setting, $\operatorname{rank}(C)=\operatorname{rank}(A_s)$. If the covariance is singular, however, that equality can fail: directions absent from the support of $X$ cannot contribute to the form.

Even the correct rank does not by itself imply a chi-square distribution. With $\mu=0$, the form is a standard $\chi^2_r$ variable when the standardized matrix $C$ is symmetric and idempotent with rank $r$. Its nonzero eigenvalues are then all one, and its remaining eigenvalues are zero. If those nonzero eigenvalues are unequal, the result is a weighted sum instead.

Worked Example: Singular Covariance Reduces the Effective Rank

Let $X=(Z,0)'$, where $Z\sim N(0,1)$. Its covariance is $\Sigma=\operatorname{diag}(1,0)$. For $A=I_2$, the original matrix has rank two, but $$ Q=X'AX=Z^2+0^2=Z^2\sim\chi^2_1. $$ The covariance has symmetric square root $\Sigma^{1/2}=\operatorname{diag}(1,0)$, giving $$ C=\Sigma^{1/2}A\Sigma^{1/2}=\operatorname{diag}(1,0), $$ which has rank one. The second coordinate is always zero, so it contributes neither a random square nor a degree of freedom. This example shows why the rank of $A$ cannot be used without regard to the covariance and its support.

Error 4: Declaring Independence from Zero Covariance

For standard normal $Z\in\mathbb{R}^p$ and symmetric matrices $A$ and $B$, the covariance of the quadratic forms is $$ \operatorname{Cov}(Z'AZ,Z'BZ)=2\operatorname{tr}(AB). $$ This follows by expanding both forms and using the standard normal fourth-moment identity $E[Z_iZ_jZ_kZ_l]=\delta_{ij}\delta_{kl}+\delta_{ik}\delta_{jl}+\delta_{il}\delta_{jk}$, where $\delta_{ij}$ is one when $i=j$ and zero otherwise. The first pairing gives the product of the expectations; the other two pairings give $2\operatorname{tr}(AB)$. Thus a zero trace product proves zero covariance, not independence.

For two real quadratic forms in a standard normal vector, Craig’s criterion states that they are independent if and only if $AB=0$, when their representing matrices $A$ and $B$ are symmetric. The criterion is stronger than a zero-covariance check. When the original Gaussian vector has positive-definite covariance $\Sigma$, apply the criterion to the standardized symmetric matrices $\Sigma^{1/2}A_s\Sigma^{1/2}$ and $\Sigma^{1/2}B_s\Sigma^{1/2}$, not automatically to the untransformed matrices.

Worked Example: Zero Covariance but Dependent Quadratic Forms

Take independent $Z_1,Z_2\sim N(0,1)$, and define $Q_1=Z_1^2$ and $Q_2=Z_1Z_2$. Both are quadratic forms: $$ Q_1=Z'AZ,\quad A=\begin{pmatrix}1&0\\0&0\end{pmatrix}; \qquad Q_2=Z'BZ,\quad B=\begin{pmatrix}0&1/2\\1/2&0\end{pmatrix}. $$ Their covariance is $$ \operatorname{Cov}(Q_1,Q_2)=E[Z_1^3Z_2]-E[Z_1^2]E[Z_1Z_2]=0-1\cdot0=0, $$ since independence gives $E[Z_1^3Z_2]=E[Z_1^3]E[Z_2]=0$. Yet the forms are dependent: conditional on $Q_1=q$, $Q_2$ has a normal distribution with mean zero and variance $q$. Indeed, given $Z_1$, $Z_1Z_2$ is normal with variance $Z_1^2=Q_1$, and that conditional distribution depends on $q$. Equivalently, $AB=\begin{pmatrix}0&1/2\\0&0\end{pmatrix}\ne0$, so Craig’s criterion does not declare independence. Zero covariance alone is insufficient.

A Practical Diagnostic Sequence

When a calculation involving Gaussian quadratic forms seems to produce a familiar distribution or an independence claim, check the steps in this order:

Symmetrize.
Replace every matrix $A$ by $A_s=(A+A')/2$, since the skew-symmetric part contributes zero to $X'AX$.

Standardize the Gaussian vector.
For positive-definite covariance, use $C=\Sigma^{1/2}A_s\Sigma^{1/2}$ before reading off the eigenvalue weights.

Check rank and eigenvalues separately.
Count nonzero eigenvalues of the transformed matrix for the number of weighted terms; require the nonzero weights to be one before calling a central form an ordinary chi-square.

Test independence with the right condition.
Zero covariance is only a moment statement. For standard normal quadratic forms with symmetric matrices, check the matrix-product condition in Craig’s criterion.

Common Mistakes

Using the rank of a nonsymmetric matrix. The quadratic form depends on its symmetric part; a full-rank skew-symmetric matrix can produce the zero form.
Using eigenvalues of $A_s$ under general covariance. The relevant weights come from $\Sigma^{1/2}A_s\Sigma^{1/2}$.
Equating rank with chi-square degrees of freedom in every case. Rank counts nonzero weighted terms; it does not make unequal weights equal to one.
Inferring independence from zero covariance. The example above has zero covariance but dependent forms; apply the appropriate matrix criterion instead.

Key takeaway: For Gaussian quadratic forms, the reliable object is the symmetric matrix after the covariance transformation. Its eigenvalues determine the weights, its nonzero eigenvalues determine the effective number of terms, and independence requires more than a zero covariance calculation.

Check Your Understanding

For each question, identify which matrix or condition controls the conclusion.

Why does replacing $A$ by $(A+A')/2$ leave $X'AX$ unchanged?
If $X\sim N(0,\operatorname{diag}(9,1))$ and $A=I_2$, what are the weights in the distribution of $X'AX$?
Can a quadratic form have a chi-square distribution if its standardized matrix has a nonzero eigenvalue equal to two? Explain.
What does the rank of the covariance-transformed matrix count in a weighted chi-square representation?
For standard normal $Z$ and symmetric $A,B$, what additional matrix condition does Craig’s criterion require for independence?