Invariance of MANOVA Roots Under Response Transformations

Why Response Coordinates Do Not Change the Roots

A MANOVA analysis can describe the same responses in different coordinates. For example, a response measured in one set of units can be rescaled, or several responses can be replaced by linear combinations of one another. Such a change alters the entries of the hypothesis and error matrices. It should not, however, alter the generalized roots that describe their relative geometry: an invertible relabelling of response coordinates does not create or remove a hypothesis direction.

This tutorial makes that statement precise. Throughout, let \(H\) and \(E\) be real symmetric \(p\times p\) matrices, with \(H\) positive semidefinite and \(E\) positive definite. Positive definiteness of \(E\) ensures that the generalized eigenvalue problem is well defined and has \(p\) real roots, counted with multiplicity.

Definition: The generalized roots of \(H\) and \(E\) are the roots \(\lambda\) of \(\det(H-\lambda E)=0\), counted with their algebraic multiplicities. Equivalently, they are the generalized eigenvalues for which there is a nonzero vector \(v\) satisfying \(Hv=\lambda Ev\). When \(E\) is positive definite and \(H\) is positive semidefinite, all these roots are real and nonnegative.

Let \(Y\) denote a data matrix with \(p\) response columns. A change of response coordinates can be written \(Y^*=YA\), where \(A\) is a nonsingular \(p\times p\) matrix. Under this change, sums-of-squares-and-cross-products matrices formed from the responses transform by congruence:

H^*=A^{T}HA,\qquad E^*=A^{T}EA.

For instance, if an error matrix is \(E=R^TR\), where the rows of \(R\) are residual response vectors, then the transformed residual matrix is \(RA\), and its error matrix is \((RA)^T(RA)=A^TEA\). The same reasoning applies to the hypothesis matrix. Since \(A\) is nonsingular, \(E^*\) remains positive definite.

The Invariance Theorem

Theorem: Suppose \(H\) is real symmetric and positive semidefinite, \(E\) is real symmetric and positive definite, and \(A\) is a real nonsingular \(p\times p\) matrix. Then \(H\) and \(E\), and their congruence transforms \(H^*=A^THA\) and \(E^*=A^TEA\), have the same generalized roots, including multiplicities.

The determinant identity proves the result directly. For every scalar \(\lambda\),

\begin{aligned} \det(H^*-\lambda E^*) &=\det\!\left(A^T(H-\lambda E)A\right)\\ &=\det(A^T)\det(H-\lambda E)\det(A)\\ &=\det(A)^2\det(H-\lambda E). \end{aligned}

The factor \(\det(A)^2\) is nonzero and does not depend on \(\lambda\). Thus the two characteristic polynomials differ only by a nonzero constant factor, so they have the same roots with the same multiplicities. This proves the theorem. \(\square\)

The same invariance has an eigenvector interpretation. If \(v^*\ne 0\) solves \(H^*v^*=\lambda E^*v^*\), then

A^THAv^*=\lambda A^TEAv^*.

Multiplying by \((A^T)^{-1}\) gives \(H(Av^*)=\lambda E(Av^*)\). Since \(A\) is nonsingular, \(Av^*\ne 0\). Conversely, a generalized eigenvector \(v\) for \(H,E\) corresponds to \(v^*=A^{-1}v\) for \(H^*,E^*\). Thus the transformation changes the coordinates used to describe a direction, not its generalized root.

Worked Example: Checking Invariance by Determinants

Let

H=\begin{pmatrix}8&0\\0&2\end{pmatrix}, \qquad E=\begin{pmatrix}2&0\\0&1\end{pmatrix}, \qquad A=\begin{pmatrix}1&1\\0&1\end{pmatrix}.

The matrix \(A\) is nonsingular because \(\det(A)=1\). Before transforming, the generalized roots solve

\det(H-\lambda E) =(8-2\lambda)(2-\lambda) =2(4-\lambda)(2-\lambda)=0.

Therefore the roots are \(4\) and \(2\). Now calculate the transformed matrices. The columns of \(A\) are \((1,0)^T\) and \((1,1)^T\), so

H^*=A^THA=\begin{pmatrix}8&8\\8&10\end{pmatrix}, \qquad E^*=A^TEA=\begin{pmatrix}2&2\\2&3\end{pmatrix}.

Their determinant polynomial is

\begin{aligned} \det(H^*-\lambda E^*) &=\det\begin{pmatrix}8-2\lambda&8-2\lambda\\ 8-2\lambda&10-3\lambda\end{pmatrix}\\ &=(8-2\lambda)(10-3\lambda)-(8-2\lambda)^2\\ &=(8-2\lambda)(2-\lambda)\\ &=2(4-\lambda)(2-\lambda). \end{aligned}

The transformed roots are again \(4\) and \(2\). The matrices have changed, but the roots have not.

Whitening the Error Matrix

A useful coordinate choice makes the error matrix the identity. Since \(E\) is symmetric positive definite, its spectral decomposition gives a symmetric positive-definite square root \(E^{1/2}\) and its inverse \(E^{-1/2}\). Choose \(A=E^{-1/2}\). Then

E^*=E^{-1/2}EE^{-1/2}=I, \qquad H^*=E^{-1/2}HE^{-1/2}.

The following result explains why this choice is useful.

Theorem: If \(E\) is symmetric positive definite and \(H\) is symmetric positive semidefinite, the generalized roots of \(H,E\) are the ordinary eigenvalues of the symmetric positive-semidefinite matrix \(K=E^{-1/2}HE^{-1/2}\).

To prove the claim, factor \(H-\lambda E\) as

H-\lambda E =E^{1/2}\left(E^{-1/2}HE^{-1/2}-\lambda I\right)E^{1/2}.

Taking determinants yields \(\det(H-\lambda E)=\det(E)\det(K-\lambda I)\). Since \(\det(E)>0\), the two determinants have the same roots, with multiplicities. Also, for every vector \(x\),

x^TKx=(E^{-1/2}x)^TH(E^{-1/2}x)\geq 0.

Thus \(K\) is symmetric positive semidefinite, and its ordinary eigenvalues are real and nonnegative. This proves the result. \(\square\)

Worked Example: Whitening a Nonidentity Error Matrix

Consider

H=\begin{pmatrix}4&2\\2&2\end{pmatrix}, \qquad E=\begin{pmatrix}4&0\\0&1\end{pmatrix}.

The error matrix is positive definite, and \(E^{-1/2}=\begin{pmatrix}1/2&0\\0&1\end{pmatrix}\). Whitening gives

K=E^{-1/2}HE^{-1/2} =\begin{pmatrix}1&1\\1&2\end{pmatrix}.

The characteristic polynomial of \(K\) is

\det(K-\lambda I)=(1-\lambda)(2-\lambda)-1 =\lambda^2-3\lambda+1.

Solving the quadratic gives the roots \(\lambda=(3+\sqrt{5})/2\) and \(\lambda=(3-\sqrt{5})/2\). Directly from the original pair,

\begin{aligned} \det(H-\lambda E) &=(4-4\lambda)(2-\lambda)-4\\ &=4\left((1-\lambda)(2-\lambda)-1\right)\\ &=4(\lambda^2-3\lambda+1). \end{aligned}

The same two roots result. Whitening has reduced the generalized eigenvalue calculation to an ordinary eigenvalue calculation for a symmetric matrix.

Choosing Coordinates to Simplify the Calculation

Whitening is one convenient choice, not the only one. Once the error matrix is the identity, an orthogonal response transformation can diagonalize the symmetric matrix \(K\). This follows from the spectral theorem: a real symmetric matrix has an orthonormal basis of eigenvectors. In those coordinates, the hypothesis matrix is diagonal and the error matrix remains the identity. Each diagonal entry is a generalized root.

Worked Example: Rotating to Diagonal Coordinates

Suppose the error matrix is already \(E=I\), and

H=\begin{pmatrix}5&2\\2&2\end{pmatrix}.

The roots solve

\det(H-\lambda I)=(5-\lambda)(2-\lambda)-4 =\lambda^2-7\lambda+6 =(\lambda-6)(\lambda-1).

Thus the roots are \(6\) and \(1\). Corresponding unit eigenvectors are \((2,1)^T/\sqrt{5}\) and \((1,-2)^T/\sqrt{5}\). Put them in the columns of

Q=\frac{1}{\sqrt{5}}\begin{pmatrix}2&1\\1&-2\end{pmatrix}.

The columns are orthonormal, so \(Q^TQ=I\). Direct multiplication gives

Q^THQ=\begin{pmatrix}6&0\\0&1\end{pmatrix}, \qquad Q^TEQ=I.

The roots are now visible on the diagonal. Since \(Q\) is nonsingular, the invariance theorem guarantees that this rotation has not changed them.

What the Invariance Does—and Does Not—Say

The invariance is useful whenever response coordinates are chosen for clarity or numerical convenience. Rescaling responses to use more manageable units, taking invertible linear combinations, or whitening \(E\) can change the entries of \(H\) and \(E\) substantially. Yet the generalized roots remain identical. The hypothesis and error matrices describe a pair of quadratic forms, and a nonsingular coordinate change rewrites both forms in the same new coordinates.

A transformation must be nonsingular. If \(A\) is singular, at least one response direction is discarded or collapsed. In that case \(\det(A)=0\), the determinant argument no longer gives a nonzero constant multiple of the original polynomial, and preservation of all roots is not guaranteed. Likewise, the congruence rule requires transforming both matrices consistently; changing \(H\) while leaving \(E\) unchanged is not, in general, a response-coordinate transformation.

Common pitfall: It is tempting to compare the entries, traces, or determinants of \(H\) and \(E\) before and after a response transformation and conclude that the roots must have changed because those quantities changed. That reasoning confuses matrix representation with the generalized eigenvalue problem. The relevant comparison is \(\det(H-\lambda E)\): under a nonsingular congruence it is multiplied by the nonzero constant \(\det(A)^2\), so its roots are unchanged. Conversely, a singular transformation can lose directions, so the invariance result cannot be invoked.

Key takeaway: For a nonsingular response transformation, \(H^*=A^THA\) and \(E^*=A^TEA\) have exactly the same generalized roots as \(H,E\). Whitening the error matrix or rotating into eigenvector coordinates is therefore a change in representation, not a change in the roots.

Check Your Understanding

Use the determinant identity and the coordinate interpretation to answer these questions.

Why does \(\det(A)^2\) not affect the roots of the determinant polynomial when \(A\) is nonsingular?
If \(E\) is positive definite, what response transformation makes the transformed error matrix the identity?
For a generalized eigenvector \(v^*\) after transformation, how is the corresponding original-coordinate vector obtained?
Why does the invariance proof fail to guarantee the same roots when \(A\) is singular?
What property of a symmetric matrix allows an orthogonal transformation to diagonalize the whitened hypothesis matrix?