Orthogonal Coordinates for Standard Normal Quadratic Forms

Why Change Coordinates?

A quadratic form in a random vector can contain products of different coordinates. For example, $Z_1^2+2Z_1Z_2+Z_2^2$ includes a cross term, which can make its structure hard to see. When the matrix defining the form is symmetric, an orthogonal change of coordinates removes all such cross terms at once. If the original coordinates are independent standard normals, the transformed coordinates remain independent standard normals, so the change simplifies the algebra without changing the underlying coordinate distributions.

This tutorial establishes that coordinate representation. The eigenvalues determine the coefficients in the diagonal form; the transformed random variables are its coordinates. Here the goal is to derive this representation carefully, not to classify the resulting distribution.

Orthogonal Diagonalization and the Coordinate Change

Definition: A real $p\times p$ matrix $Q$ is orthogonal if $Q'Q=QQ'=I_p$, where $Q'$ denotes the transpose and $I_p$ the $p\times p$ identity matrix. Its columns form an orthonormal basis of $\mathbb{R}^p$. The spectral theorem states that, for every real symmetric matrix $A$, there are an orthogonal $Q$ and a real diagonal matrix $\Lambda=\operatorname{diag}(\lambda_1,\ldots,\lambda_p)$ such that $A=Q\Lambda Q'$. The columns of $Q$ are eigenvectors of $A$, and the corresponding diagonal entries of $\Lambda$ are their eigenvalues.

Suppose $Z=(Z_1,\ldots,Z_p)'$ has independent standard normal components, and let $Y=Q'Z$. The coordinates of $Y$ are the projections of $Z$ onto the columns of $Q$. The key fact is that $Y$ has the same joint distribution as $Z$.

Theorem: If $Z_1,\ldots,Z_p$ are independent $N(0,1)$ random variables and $Q$ is a fixed orthogonal $p\times p$ matrix, then the coordinates $Y=Q'Z$ are also independent $N(0,1)$ random variables.

Proof. The joint density of $Z$ at $z\in\mathbb{R}^p$ is

f_Z(z)=(2\pi)^{-p/2}\exp\!\left(-\frac{z'z}{2}\right).

The transformation is $y=Q'z$, so its inverse is $z=Qy$. Its Jacobian determinant in absolute value is $|\det(Q)|=1$, since $Q'Q=I_p$ implies $(\det Q)^2=1$. Also,

$$ z'z=(Qy)'(Qy)=y'Q'Qy=y'y. $$

The change-of-variables formula therefore gives

f_Y(y)=f_Z(Qy)|\det Q| =(2\pi)^{-p/2}\exp\!\left(-\frac{y'y}{2}\right) =\prod_{i=1}^{p}\frac{1}{\sqrt{2\pi}}\exp\!\left(-\frac{y_i^2}{2}\right).

This joint density factors into the standard normal density for each coordinate. Hence $Y_1,\ldots,Y_p$ are independent and each has distribution $N(0,1)$. $\square$

The Diagonal-Coordinate Representation

For a vector $z$, the quadratic form defined by $A$ is the scalar $z'Az$. Apply the spectral theorem to a real symmetric $A$, writing $A=Q\Lambda Q'$, and use the transformed coordinates $Y=Q'Z$. Then

Z'AZ=Z'Q\Lambda Q'Z=Y'\Lambda Y =\sum_{i=1}^{p}\lambda_iY_i^2.

Theorem: Let $A$ be a fixed real symmetric $p\times p$ matrix, and let $Z$ have independent $N(0,1)$ components. Choose an orthogonal diagonalization $A=Q\Lambda Q'$, with $\Lambda=\operatorname{diag}(\lambda_1,\ldots,\lambda_p)$, and define $Y=Q'Z$. Then $$ Z'AZ=\sum_{i=1}^{p}\lambda_iY_i^2, $$ where $Y_1,\ldots,Y_p$ are independent $N(0,1)$ random variables.

Proof. The orthogonal-invariance theorem shows that the coordinates of $Y=Q'Z$ are independent standard normals. Substituting $A=Q\Lambda Q'$ into the quadratic form gives $Z'AZ=(Q'Z)'\Lambda(Q'Z)=Y'\Lambda Y$. Since $\Lambda$ is diagonal, matrix multiplication gives $Y'\Lambda Y=\sum_{i=1}^{p}\lambda_iY_i^2$. This proves both the algebraic representation and the stated properties of its coordinates. $\square$

The eigenvalues are real, but they need not all be positive. A zero eigenvalue contributes no term, while a negative eigenvalue contributes a negative multiple of a squared coordinate. The formula itself requires neither positive definiteness nor invertibility; it requires symmetry for orthogonal diagonalization and standard independent normal coordinates for the invariance result.

Worked Coordinate Changes

Worked Example: A Two-Dimensional Form With a Cross Term

Let $Z=(Z_1,Z_2)'$ have independent $N(0,1)$ components, and set

A=\begin{pmatrix}2&1\\1&2\end{pmatrix}.

The vectors $q_1=(1,1)'/\sqrt{2}$ and $q_2=(1,-1)'/\sqrt{2}$ are orthonormal. Direct multiplication gives $Aq_1=3q_1$ and $Aq_2=q_2$. Thus, with $Q=(q_1\ q_2)$,

Q=\frac{1}{\sqrt{2}}\begin{pmatrix}1&1\\1&-1\end{pmatrix}, \qquad Q'AQ=\begin{pmatrix}3&0\\0&1\end{pmatrix}.

The transformed coordinates are $Y_1=(Z_1+Z_2)/\sqrt{2}$ and $Y_2=(Z_1-Z_2)/\sqrt{2}$. Both are independent $N(0,1)$ variables by orthogonal invariance. The original form is

$$ Z'AZ=2Z_1^2+2Z_1Z_2+2Z_2^2 =3Y_1^2+Y_2^2. $$

For the last equality, substitute $Z_1=(Y_1+Y_2)/\sqrt{2}$ and $Z_2=(Y_1-Y_2)/\sqrt{2}$, or use $Z'AZ=Y'(Q'AQ)Y$. The cross term in the original coordinates disappears in the eigenvector coordinates.

Worked Example: A Zero Eigenvalue Removes a Direction

Let

A=\begin{pmatrix}1&-1\\-1&1\end{pmatrix}, \qquad Z=(Z_1,Z_2)',

where $Z_1,Z_2$ are independent $N(0,1)$. The orthonormal eigenvectors $q_1=(1,-1)'/\sqrt{2}$ and $q_2=(1,1)'/\sqrt{2}$ have eigenvalues $2$ and $0$, respectively. With $Y=Q'Z$, this gives

Y_1=\frac{Z_1-Z_2}{\sqrt{2}}, \qquad Y_2=\frac{Z_1+Z_2}{\sqrt{2}}, \qquad Z'AZ=2Y_1^2+0Y_2^2.

A direct check in the original coordinates yields $Z'AZ=Z_1^2-2Z_1Z_2+Z_2^2=(Z_1-Z_2)^2=2Y_1^2$. The second coordinate makes no contribution because its eigenvalue is zero. The transformed coordinates are nevertheless both independent standard normals; it is the coefficient, not the coordinate's distribution, that removes this direction from the form.

Worked Example: A Negative Eigenvalue

Consider independent standard normal coordinates $Z=(Z_1,Z_2,Z_3)'$ and the symmetric matrix

A=\begin{pmatrix}3&1&0\\1&3&0\\0&0&-2\end{pmatrix}.

The first two orthonormal eigenvectors are $q_1=(1,1,0)'/\sqrt{2}$ and $q_2=(1,-1,0)'/\sqrt{2}$, with eigenvalues $4$ and $2$. The vector $q_3=(0,0,1)'$ has eigenvalue $-2$. These vectors form the columns of an orthogonal matrix $Q$. Therefore the coordinates $Y=Q'Z$ are

Y_1=\frac{Z_1+Z_2}{\sqrt{2}}, \qquad Y_2=\frac{Z_1-Z_2}{\sqrt{2}}, \qquad Y_3=Z_3.

They are independent standard normals, and the diagonal-coordinate form is

$$ Z'AZ=4Y_1^2+2Y_2^2-2Y_3^2. $$

Expanding in the original coordinates checks the result: the first two coordinates contribute $3Z_1^2+2Z_1Z_2+3Z_2^2$, and the third contributes $-2Z_3^2$. The negative eigenvalue produces a negative term, so the quadratic form is not constrained to be nonnegative.

Repeated Eigenvalues and Coordinate Choices

When an eigenvalue is repeated, its eigenspace has dimension greater than one, and there is more than one possible orthonormal basis for that space. This does not change the quadratic form's coordinate structure. If a value $\lambda$ occurs for coordinates $Y_i,\ldots,Y_j$, their contribution is $\lambda\sum_{k=i}^{j}Y_k^2$. Replacing the basis within that eigenspace applies another orthogonal transformation to those coordinates; their squared sum is unchanged because $R'R=I$ implies $(Ry)'(Ry)=y'y$.

Thus the individual coordinates can depend on the chosen eigenbasis, while the eigenvalues and the diagonal representation as a whole do not depend on arbitrary choices of basis within repeated-eigenvalue spaces.

Common Mistakes and Why the Coordinates Matter

Changing the matrix but not the vector. If $Y=Q'Z$, the matching identity is $Z'AZ=Y'(Q'AQ)Y$. Keeping $Z$ in the expression while replacing $A$ by $Q'AQ$ generally mixes two coordinate systems.
Using a general change of basis as though it were orthogonal. A symmetric matrix admits an orthogonal diagonalization. A non-orthogonal transformation does not preserve the standard normal density in the way proved above.
Assuming every eigenvalue is positive. Symmetry guarantees real eigenvalues, not positive ones. Positivity would require the additional condition that $A$ be positive definite.
Assuming transformed coordinates are independent just because they are uncorrelated. Here independence follows from the transformed joint density, which factors into standard normal densities; the conclusion uses the normal model and orthogonality.

The practical value of the change of coordinates is that it isolates the contribution along each eigenvector direction. Cross terms vanish, and zero or negative directions become explicit. This is the algebraic foundation for subsequent distributional results about Gaussian quadratic forms.

Check Your Understanding

Use the orthogonal-coordinate representation to answer these questions.

For a real symmetric matrix $A=Q\Lambda Q'$, which transformed vector should be used to express $Z'AZ$ in diagonal coordinates?
Why are the coordinates of $Q'Z$ independent standard normals when $Z$ has independent standard normal components?
If one eigenvalue of $A$ is zero, what happens to its coordinate in the expression for $Z'AZ$?
Can a symmetric matrix with a negative eigenvalue define a quadratic form that takes negative values? Explain using the diagonal-coordinate expression.
Why does choosing a different orthonormal basis within a repeated-eigenvalue space leave that space's contribution unchanged?