Simultaneous Diagonalization of Commuting Quadratic Forms

Why a Common Basis Matters

A single symmetric matrix can be diagonalized by choosing coordinates along its eigenvectors. When several quadratic forms are being studied together, diagonalizing each matrix separately is not enough: the resulting coordinates may differ from one form to another. The key question is whether one orthonormal coordinate system diagonalizes all the matrices at once.

Commutation provides the answer. If a finite collection of real symmetric matrices commutes pairwise, they have a common orthonormal eigenbasis. For Gaussian vectors, this shared basis is especially useful because an orthogonal change of coordinates preserves a standard multivariate normal distribution. Consequently, all the quadratic forms can be written using the same independent standard normal coordinates. The shared coordinates also make clear why simultaneous diagonalization does not, by itself, imply independence of the forms.

Simultaneous Diagonalization

For the forward direction, suppose \(U'A_rU=D_r\) and \(U'A_sU=D_s\) are diagonal. Diagonal matrices commute, so \(D_rD_s=D_sD_r\). Since \(UU'=I\), \(A_rA_s=UD_rD_sU'=UD_sD_rU'=A_sA_r\).

For the reverse direction, use induction on the number \(k\) of matrices. The case \(k=1\) is the spectral theorem for real symmetric matrices. Suppose the result holds for \(k-1\) pairwise commuting real symmetric matrices, and consider \(A_1,\ldots,A_k\). By the spectral theorem, the space is an orthogonal direct sum of the eigenspaces \(E_\lambda\) of \(A_1\). Each \(E_\lambda\) is invariant under every \(A_r\), \(r\geq 2\): if \(v\in E_\lambda\), then \(A_1v=\lambda v\), and commutation gives \(A_1(A_rv)=A_r(A_1v)=\lambda A_rv\), so \(A_rv\in E_\lambda\).

On each \(E_\lambda\), the restrictions of \(A_2,\ldots,A_k\) are symmetric and commute pairwise. By the induction hypothesis, they have a common orthonormal eigenbasis on that eigenspace. Combining these bases across the mutually orthogonal eigenspaces gives an orthonormal basis of the full space. Its vectors are eigenvectors of \(A_1\) and of every other \(A_r\), so it diagonalizes the whole family. Thus pairwise commutation is sufficient as well. \(\square\)

What the Shared Basis Does to Gaussian Forms

For a vector \(z\in\mathbb{R}^p\), the quadratic form associated with a symmetric matrix \(A\) is \(z'Az\). Suppose \(A_1,\ldots,A_k\) commute pairwise and are real symmetric. Choose a common orthogonal diagonalizer \(U\), and write \(U'A_rU=\operatorname{diag}(\lambda_{r1},\ldots,\lambda_{rp})\). For \(Z\sim N_p(0,I)\), define \(Y=U'Z\). Orthogonal invariance gives \(Y\sim N_p(0,I)\), whose coordinates \(Y_1,\ldots,Y_p\) are independent standard normal random variables. Substitution then gives the joint representation

The same \(Y_j\)'s appear in every form. This is the central feature of the joint representation: each form has its own eigenvalues, but the random coordinates are shared. The statement does not say that the forms are independent.

The representation also yields exact first and second moments. Since a standard normal variable has \(E[Y_j^2]=1\) and \(\operatorname{Var}(Y_j^2)=2\), independence of the coordinates gives

For the covariance, terms with different coordinate indices have zero covariance, while the \(j\)th shared coordinate contributes \(\lambda_{rj}\lambda_{sj}\operatorname{Var}(Y_j^2)=2\lambda_{rj}\lambda_{sj}\). The trace identity follows because both matrices are diagonal in the common basis.

Independence Depends on Which Coordinates Are Used

The common-basis representation separates the question of diagonalization from the question of independence. If two forms involve disjoint sets of the independent coordinates \(Y_1,\ldots,Y_p\), then they are independent: each is a function of a different group of independent random variables. If they share coordinates with nonzero coefficients, they need not be independent. The preceding example shares \(Y_1^2\), and its covariance is nonzero.

A useful special case is when the diagonal matrices have disjoint nonzero coordinate positions. In that case their forms are functions of disjoint coordinate groups. Commutation alone does not impose this disjointness; it only ensures that a shared orthonormal basis exists.

Shifted Gaussian Vectors

The same coordinates work when the Gaussian vector has a nonzero mean. If \(Z\sim N_p(m,I)\), then \(Y=U'Z\sim N_p(U'm,I)\), with independent coordinates. Writing \(\nu=U'm\), each form becomes

The coefficients are still the eigenvalues in the common basis; the shift changes the coordinate distributions, not the diagonalization. The shared coordinates again determine how the forms are related.

Using the Result With a General Covariance Matrix

If \(X\sim N_p(\mu,\Sigma)\) with positive definite covariance matrix \(\Sigma\), set \(Z=\Sigma^{-1/2}(X-\mu)\). Then \(Z\sim N_p(0,I)\). For centered forms in \(X-\mu\),

Each \(B_r\) is real symmetric when \(A_r\) is real symmetric. The common-basis result applies if these whitened matrices \(B_r\) commute pairwise. Commutation of the original \(A_r\)'s alone does not generally establish that condition when \(\Sigma\) is not the identity. If the forms are uncentered, \(X'A_rX\), substituting \(X=\mu+\Sigma^{1/2}Z\) also produces linear terms in \(Z\); diagonalizing the quadratic parts does not remove those terms automatically.

Common Pitfalls and Practical Meaning

• Assuming separate diagonalizations are enough. The eigenvectors must be shared. Pairwise commutation is the condition that guarantees a common orthonormal basis for real symmetric matrices.
• Confusing simultaneous diagonalization with independence. A common basis expresses all forms using the same independent normal coordinates; overlapping nonzero coefficients can leave dependence, as the first example shows.
• Changing the matrices but not the random vector. If \(U'A_rU\) is used to diagonalize the matrices, the corresponding coordinates in the form are \(Y=U'Z\). This keeps the quadratic-form identity exact.
• Applying the identity without checking the covariance. For a nonstandard Gaussian covariance, first account for the covariance by whitening. The relevant matrices for centered forms are \(\Sigma^{1/2}A_r\Sigma^{1/2}\).

The practical payoff is a shared coordinate description of several forms. It reveals their individual eigenvalue weights, which normal coordinates contribute to each form, and where dependence can arise. Those structural facts are often more informative than analyzing each quadratic form in isolation.