Integrated Gaussian Quadratic-Form Problems

A Unified Strategy for Quadratic-Form Problems

A Gaussian quadratic-form problem often asks several questions at once: what is the distribution of the form, what are its moments, is it chi-square, and is it independent of another form? The same reduction answers all of them. First express the Gaussian vector in standardized coordinates, then diagonalize the resulting symmetric matrix, and finally interpret each coordinate contribution.

Throughout, let $X$ be a $p$-dimensional Gaussian vector with mean $\mu$ and positive-definite covariance matrix $\Sigma$, and let $A$ be a real symmetric matrix. We study $Q=X'AX$. Positive definiteness makes whitening direct; the singular-covariance case is handled by reducing to the Gaussian vector’s support.

Definition: Let $Z=\Sigma^{-1/2}(X-\mu)$, so $Z\sim N_p(0,I_p)$. Define the symmetric matrix $C=\Sigma^{1/2}A\Sigma^{1/2}$. If $U$ is orthogonal and $U'CU=\operatorname{diag}(\lambda_1,\ldots,\lambda_p)$, set $Y=U'Z$ and $d=U'\Sigma^{1/2}A\mu$. Then the eigenvalue reduction of $Q$ is $$ Q=\mu'A\mu+\sum_{i=1}^p\lambda_iY_i^2+2\sum_{i=1}^p d_iY_i, $$ where the $Y_i$ are independent $N(0,1)$ variables.

The reduction follows by substituting $X=\mu+\Sigma^{1/2}Z$ into $X'AX$, and then rotating $Z$ by $U$. Orthogonal rotation preserves the standard multivariate normal law, so the coordinates $Y_i$ are independent standard normal variables. If $d_i=0$ whenever $\lambda_i=0$, each nonzero-eigenvalue term can also be written as $\lambda_i(Y_i+d_i/\lambda_i)^2-d_i^2/\lambda_i$. Thus the eigenvalues determine the weights, while the linear terms and constant determine any shifts.

Moments from the Reduced Form

The reduction provides a direct route to the first two moments, including when the quadratic form is not chi-square. This is useful because moments can be checked independently of any distributional label.

Theorem: For $X\sim N_p(\mu,\Sigma)$ with $\Sigma$ positive definite and $A=A'$, $$ E[Q]=\operatorname{tr}(A\Sigma)+\mu'A\mu,\qquad \operatorname{Var}(Q)=2\operatorname{tr}(A\Sigma A\Sigma)+4\mu'A\Sigma A\mu. $$

Proof. In the eigenvalue reduction, write $c=\mu'A\mu$. Since $E[Y_i]=0$ and $E[Y_i^2]=1$, $$ E[Q]=c+\sum_{i=1}^p\lambda_i=c+\operatorname{tr}(C). $$ The cyclic property of the trace gives $\operatorname{tr}(C)=\operatorname{tr}(A\Sigma)$. For the variance, independence gives $$ \operatorname{Var}(Q) =\sum_{i=1}^p\operatorname{Var}(\lambda_iY_i^2+2d_iY_i). $$ For a standard normal variable, $E[Y_i^4]=3$ and $E[Y_i^3]=0$. Therefore $\operatorname{Var}(Y_i^2)=2$, and $\operatorname{Cov}(Y_i^2,Y_i)=0$. Expanding each variance yields $$ \operatorname{Var}(Q)=2\sum_{i=1}^p\lambda_i^2+4\sum_{i=1}^p d_i^2. $$ Now $\sum_i\lambda_i^2=\operatorname{tr}(C^2)=\operatorname{tr}(A\Sigma A\Sigma)$, and $\sum_i d_i^2=\mu'A\Sigma A\mu$. Substitution proves both formulas. $\square$

Worked Example: A Weighted Chi-Square Form and Its Moments

Let $X\sim N_3(0,\Sigma)$, where $\Sigma=\operatorname{diag}(4,1,1)$, and let $A=\operatorname{diag}(1/4,2,0)$. Find the distributional representation, mean, and variance of $Q=X'AX$.

Since $\Sigma^{1/2}=\operatorname{diag}(2,1,1)$, the reduced matrix is $$ C=\Sigma^{1/2}A\Sigma^{1/2}=\operatorname{diag}(1,2,0). $$ Writing $Z=\Sigma^{-1/2}X\sim N_3(0,I_3)$, we obtain $$ Q=Z_1^2+2Z_2^2. $$ Thus $Q$ is a weighted sum of two independent $\chi^2_1$ variables, not an ordinary chi-square variable: its nonzero weights are 1 and 2.

Directly, $E[Q]=1+2=3$, and $\operatorname{Var}(Q)=2(1^2+2^2)=10$. The matrix formulas agree: $\operatorname{tr}(A\Sigma)=3$, while $2\operatorname{tr}(A\Sigma A\Sigma)=2(1+4)=10$. The mean vector is zero, so the additional noncentral variance term is zero.

Recognizing Chi-Square Forms

A chi-square variable is a sum of equally weighted squares of independent standard normal variables. The eigenvalue reduction makes this structure visible. The following conditions provide a useful sufficient check, including for a shifted Gaussian vector.

Theorem: In the eigenvalue reduction above, suppose every $\lambda_i$ is either 0 or 1, $d_i=0$ whenever $\lambda_i=0$, and $$ \mu'A\mu=\sum_{\{i:\lambda_i=1\}}d_i^2. $$ Then $Q\sim\chi^2_r(\eta)$, a noncentral chi-square distribution with $r=\#\{i:\lambda_i=1\}$ degrees of freedom and noncentrality parameter $\eta=\sum_{\{i:\lambda_i=1\}}d_i^2$. If all $d_i=0$ and $\mu'A\mu=0$, this reduces to the central law $Q\sim\chi^2_r$.

Proof. Under these conditions, the zero-eigenvalue coordinates contribute neither quadratic nor linear terms. For the remaining coordinates, complete the square: $$ Q=\sum_{\{i:\lambda_i=1\}}(Y_i^2+2d_iY_i)+\mu'A\mu =\sum_{\{i:\lambda_i=1\}}(Y_i+d_i)^2. $$ The final equality uses the stated condition on the constant. By definition, the sum of squares of $r$ independent unit-variance normal variables with means $d_i$ has a noncentral chi-square distribution with $r$ degrees of freedom and noncentrality parameter $\sum_i d_i^2$. When every $d_i=0$, the distribution is central. $\square$

Worked Example: Checking a Noncentral Chi-Square Criterion

Let $X\sim N_3(\mu,I_3)$, with $\mu=(1,2,0)'$, and let $A=\operatorname{diag}(1,1,0)$. Determine the distribution, mean, and variance of $Q=X'AX$.

Here $C=A$, so its eigenvalues are $1,1,0$. The reduced coordinates are the original coordinates, and the linear coefficients are $d=(1,2,0)'$. Also, $\mu'A\mu=1^2+2^2=5=\sum_{\lambda_i=1}d_i^2$. The criterion applies: $$ Q=(X_1)^2+(X_2)^2=(Z_1+1)^2+(Z_2+2)^2 \sim\chi^2_2(5). $$ Its mean is $2+5=7$, and its variance is $2(2+2\cdot5)=24$. These values also follow from the moment formulas: $\operatorname{tr}(A)=2$, $\mu'A\mu=5$, and $\mu'A\mu=5$ gives the quadratic contribution $2\operatorname{tr}(A^2)=4$ and the shift contribution $4\mu'A\mu=20$, for variance $24$.

The eigenvalue condition is a check, not a license to label every sum of squares “chi-square.” For example, a coefficient of 2 produces a weighted chi-square term; it cannot be absorbed into the degrees of freedom. Likewise, for a shifted form, the constant and linear terms must match the completed-square expression. If the check fails, the eigenvalue reduction still describes the form and its moments, but a different distributional analysis may be required.

Testing Independence of Two Forms

For two quadratic forms in the same centered Gaussian vector, Craig’s criterion gives a matrix test for independence. In the positive-definite covariance setting, for symmetric $A$ and $D$, the forms $X'AX$ and $X'DX$ are independent if and only if $$ A\Sigma D=0. $$ The matrix order matters: this condition is equivalent to $(\Sigma^{1/2}A\Sigma^{1/2})(\Sigma^{1/2}D\Sigma^{1/2})=0$. In particular, it implies that the two transformed forms use orthogonal eigenspaces. Independence is stronger than zero covariance; checking only a scalar covariance is not a substitute for the matrix condition.

For the sufficiency direction, put $C_A=\Sigma^{1/2}A\Sigma^{1/2}$ and $C_D=\Sigma^{1/2}D\Sigma^{1/2}$. If $A\Sigma D=0$, then $C_AC_D=0$. Since both matrices are symmetric, the range of $C_D$ is contained in the null space of $C_A$, and their ranges are orthogonal. For a centered Gaussian vector, $Z=\Sigma^{-1/2}X$ has independent standard normal coordinates in any orthonormal basis adapted to these orthogonal spaces. The first quadratic form depends only on coordinates in the range of $C_A$, and the second only on coordinates in the range of $C_D$; any remaining coordinates contribute nothing. Functions of these disjoint independent coordinate sets are independent. $\square$

Worked Example: Applying the Independence Matrix Test

Let $X\sim N_3(0,\Sigma)$, with $\Sigma=\operatorname{diag}(4,1,9)$, and define $Q_1=X_1^2/4+X_2^2$ and $Q_2=X_3^2/9$. Verify their distributions and independence.

The matrices are $A=\operatorname{diag}(1/4,1,0)$ and $D=\operatorname{diag}(0,0,1/9)$. Whitening gives $$ C_A=\Sigma^{1/2}A\Sigma^{1/2}=\operatorname{diag}(1,1,0),\qquad C_D=\Sigma^{1/2}D\Sigma^{1/2}=\operatorname{diag}(0,0,1). $$ Thus $Q_1=Z_1^2+Z_2^2\sim\chi^2_2$ and $Q_2=Z_3^2\sim\chi^2_1$, with independent standard normal coordinates $Z_1,Z_2,Z_3$. The matrix test also verifies independence: $$ A\Sigma D=\operatorname{diag}(0,0,0). $$ Hence $Q_1$ and $Q_2$ are independent. Their expectations, 2 and 1, are a useful check, but those expectations alone do not establish independence.

Common Pitfalls and a Practical Workflow

A reliable solution keeps the matrix calculation, distributional conclusion, and independence claim separate until each has been checked. The following sequence is effective for a new problem.

Check the setup.
Confirm that the quadratic-form matrix is symmetric. Record the Gaussian mean and covariance, and identify whether the covariance is positive definite or whether the problem requires reduction to its support.

Whiten and diagonalize.
Form $C=\Sigma^{1/2}A\Sigma^{1/2}$, find its eigenvalues, and rotate the standardized Gaussian coordinates using an orthogonal eigenvector matrix.

Interpret the reduced form.
Keep track of the eigenvalue weights, linear coefficients, and constant. Apply the chi-square check only if its eigenvalue and shift conditions hold.

Check moments and independence separately.
Use the moment formulas as an arithmetic check. For two centered Gaussian quadratic forms, apply Craig’s criterion to the matrices; do not infer independence from zero covariance alone.

Using $A$'s eigenvalues instead of the whitened matrix's. Unless $\Sigma=I_p$, the relevant eigenvalues come from $\Sigma^{1/2}A\Sigma^{1/2}$, not from $A$ alone.
Calling a weighted sum chi-square. A coefficient other than 0 or 1 does not give an ordinary chi-square law under the criterion above.
Dropping the mean terms. When $\mu\ne0$, both the linear coefficients and the constant affect the law and the moments.
Using zero covariance as an independence proof. Independence requires an appropriate joint-distribution or matrix argument, such as Craig’s criterion under its stated Gaussian assumptions.

Key takeaway: Reduce the form in standardized coordinates before naming its distribution. The eigenvalues identify its quadratic weights; the mean terms determine shifts; the trace formulas check its moments; and a matrix criterion, rather than zero covariance alone, supports an independence conclusion.

Check Your Understanding

Use the reduction and checks developed above to work through these questions.

For a centered Gaussian vector, why are the eigenvalues of $\Sigma^{1/2}A\Sigma^{1/2}$, rather than those of $A$ alone, relevant to the distribution of $X'AX$?
Suppose the reduced eigenvalues are $1,0,0$, the linear coefficients are all zero, and the constant is zero. What is the distribution of the form?
In the noncentral example, identify the degrees of freedom and noncentrality parameter, and verify the mean from the chi-square formula.
For two symmetric matrices $A,D$ and a centered Gaussian covariance $\Sigma$, what matrix condition does Craig’s criterion use to test independence?
Why does finding zero covariance between two forms not, by itself, justify an independence claim?