Generalized Roots of Hypothesis and Error Matrices

Why Generalized Roots Matter

In a multivariate linear test, the hypothesis and error sums-of-squares-and-cross-products matrices, $H$ and $E$, describe variation in the same response space. A MANOVA root compares these matrices along a particular response direction. The roots therefore provide a compact description of how the hypothesis variation sits relative to the error variation—not merely how large either matrix is by itself.

The first issue is whether this comparison is well defined. The standard MANOVA roots require $E$ to be positive definite. Under that condition, the number of nonzero roots is determined by the rank of $H$, while their numerical values depend on both matrices. If $E$ is singular, the ordinary calculation needs additional care.

Definition: Let $H$ and $E$ be symmetric positive-semidefinite $p \times p$ matrices, and suppose $E$ is positive definite. The generalized roots (or MANOVA roots) are the nonzero generalized eigenvalues $\lambda$ satisfying $$ Hv=\lambda Ev $$ for some nonzero vector $v$. Roots are counted with multiplicity. Equivalently, they are the nonzero eigenvalues of the symmetric matrix $E^{-1/2}HE^{-1/2}$.

Here $v$ is a response-space direction. For this direction, $v^{\mathsf T}Hv$ is the hypothesis quadratic form and $v^{\mathsf T}Ev$ is the error quadratic form. When $E$ is positive definite, the latter is positive for every $v\ne0$, so the ratio $v^{\mathsf T}Hv/(v^{\mathsf T}Ev)$ is well defined. The generalized eigenvalue equation identifies directions in which that ratio takes an eigenvalue.

Whitening the Error Matrix

The symmetric formulation is useful both conceptually and computationally. Since $E$ is positive definite, it has a symmetric positive-definite square root $E^{1/2}$, with inverse $E^{-1/2}$. Set $y=E^{1/2}v$. Then the generalized eigenvalue equation can be written as an ordinary symmetric eigenvalue equation.

Theorem: If $E$ is positive definite and $H$ is positive semidefinite, then all generalized roots are real and nonnegative. The number of nonzero roots, counted with multiplicity, is $\operatorname{rank}(H)$. The remaining $p-\operatorname{rank}(H)$ eigenvalues are zero.

Proof. Let $C=E^{-1/2}HE^{-1/2}$. This matrix is symmetric because $H$ and $E^{-1/2}$ are symmetric. For every vector $y$, $$ y^{\mathsf T}Cy=(E^{-1/2}y)^{\mathsf T}H(E^{-1/2}y)\geq 0, $$ so $C$ is positive semidefinite. By the spectral theorem, its eigenvalues are real and nonnegative. Because multiplication by the invertible matrix $E^{-1/2}$ on either side does not change rank, $\operatorname{rank}(C)=\operatorname{rank}(H)$. A symmetric positive-semidefinite matrix has exactly its rank many nonzero eigenvalues, and all other eigenvalues are zero.

It remains to connect these eigenvalues to the generalized equation. If $Hv=\lambda Ev$, multiply on the left by $E^{-1/2}$, and set $y=E^{1/2}v$. This gives $Cy=\lambda y$. Conversely, if $Cy=\lambda y$, set $v=E^{-1/2}y$ and multiply by $E^{1/2}$ to recover $Hv=\lambda Ev$. The change of variables is invertible, so the eigenvalues, including multiplicities, agree. Thus the generalized roots have the claimed properties. $\square$

One immediate consequence is that, when $E$ is positive definite, the number of nonzero roots cannot exceed $\min(p,\operatorname{rank}(H))$, which is simply $\operatorname{rank}(H)$ since $H$ is $p \times p$. Rank determines how many positive roots there are, not their sizes. The values reflect the relative magnitudes and directions encoded jointly by $H$ and $E$.

Worked Example: Reading the Roots From Diagonal Matrices

Let $$ H=\begin{pmatrix}4&0&0\\0&0&0\\0&0&9\end{pmatrix}, \qquad E=\begin{pmatrix}2&0&0\\0&3&0\\0&0&1\end{pmatrix}. $$ Find the generalized roots and check the rank-based count.

The error matrix is positive definite because its diagonal entries are all positive. The generalized eigenvalue equation is diagonal: $$ \begin{pmatrix}4&0&0\\0&0&0\\0&0&9\end{pmatrix}v = \lambda \begin{pmatrix}2&0&0\\0&3&0\\0&0&1\end{pmatrix}v. $$ For a vector in the first coordinate direction, the equation gives $4v_1=2\lambda v_1$, so $\lambda=2$. In the second coordinate direction, it gives $0=3\lambda v_2$, so the corresponding eigenvalue is $0$. In the third direction, $9v_3=\lambda v_3$, so $\lambda=9$. The roots are therefore $9$ and $2$, with one zero eigenvalue.

The rank of $H$ is $2$, so the theorem predicts two nonzero roots and $3-2=1$ zero eigenvalue. The calculation agrees. In this diagonal example the roots are the nonzero diagonal entries of $H$ divided by the corresponding entries of $E$; that shortcut does not apply to general nondiagonal matrices.

Ranks Do Not Determine Root Values

The rank result is a counting statement. Two pairs of matrices may have identical ranks but different roots. Even when both matrices are diagonal, changing their relative scales changes the roots. With nondiagonal matrices, the alignment of the hypothesis and error variation across response directions also matters.

Worked Example: Nondiagonal Hypothesis Variation

Take $$ H=\begin{pmatrix}2&1\\1&2\end{pmatrix}, \qquad E=\begin{pmatrix}1&0\\0&4\end{pmatrix}. $$ Both matrices are positive definite, so their ranks are both $2$. Find the roots.

The inverse square root of $E$ is $\operatorname{diag}(1,\tfrac12)$, so $$ E^{-1/2}HE^{-1/2} = \begin{pmatrix}1&0\\0&\tfrac12\end{pmatrix} \begin{pmatrix}2&1\\1&2\end{pmatrix} \begin{pmatrix}1&0\\0&\tfrac12\end{pmatrix} = \begin{pmatrix}2&\tfrac12\\\tfrac12&\tfrac12\end{pmatrix}. $$ Its characteristic equation is $$ \lambda^2-\frac52\lambda+\frac34=0. $$ The discriminant is $\frac{25}{4}-3=\frac{13}{4}$, so the roots are $$ \lambda_1=\frac{5+\sqrt{13}}{4}\approx 2.151, \qquad \lambda_2=\frac{5-\sqrt{13}}{4}\approx 0.349. $$ Both are positive, as expected, and there are two because $\operatorname{rank}(H)=2$. The roots are not obtained by dividing corresponding diagonal entries of $H$ and $E$: $H$ is not diagonal in these coordinates. The rank pair $(2,2)$ establishes the count, but not either value.

For any generalized eigenvector $v\ne0$, multiplying $Hv=\lambda Ev$ by $v^{\mathsf T}$ gives $$ \lambda=\frac{v^{\mathsf T}Hv}{v^{\mathsf T}Ev}. $$ The denominator is positive and the numerator is nonnegative, which also shows directly why a generalized root cannot be negative. This quotient gives a useful directional interpretation, but it does not mean that every direction $v$ is an eigenvector or that each root can be read from a single diagonal entry.

What Changes When the Error Matrix Is Singular?

If $\operatorname{rank}(E)<p$, then $E^{-1/2}$ does not exist on the whole response space, and the standard definition above cannot be applied directly. This situation can arise when the error information does not span all response directions. It is not valid simply to replace $E^{-1}$ by a generalized inverse and assume that the resulting values are the ordinary MANOVA roots.

A useful special case occurs when every direction with zero error variation also has zero hypothesis variation. For symmetric matrices, this condition can be stated as $\operatorname{range}(H)\subseteq\operatorname{range}(E)$. The matrices can then be restricted to the range of $E$, where the restricted error matrix is positive definite. The generalized roots are calculated on that support. If the condition fails, a direction can have positive hypothesis variation and zero error variation; the pencil may then have an infinite generalized eigenvalue rather than only ordinary finite roots.

Worked Example: Restricting to the Error Support

Consider $$ E=\begin{pmatrix}3&0&0\\0&0&0\\0&0&2\end{pmatrix}, \qquad H=\begin{pmatrix}6&0&0\\0&0&0\\0&0&4\end{pmatrix}. $$ Here $\operatorname{rank}(E)=2$ and $\operatorname{rank}(H)=2$. Both matrices vanish in the second coordinate, so the hypothesis range is contained in the error range. Restrict to coordinates 1 and 3, where the matrices are $\operatorname{diag}(6,4)$ and $\operatorname{diag}(3,2)$. The generalized equations give $6v_1=3\lambda v_1$ and $4v_3=2\lambda v_3$, hence both roots are $2$.

The second coordinate satisfies $0=\lambda\cdot0$ for every $\lambda$. That equation does not supply an additional root: it reflects a direction excluded from the error support. On the two-dimensional support, there are two nonzero roots, matching the rank of the restricted hypothesis matrix.

Worked Example: Positive Hypothesis Variation With Zero Error

Now let $$ E=\begin{pmatrix}2&0\\0&0\end{pmatrix}, \qquad H=\begin{pmatrix}2&0\\0&1\end{pmatrix}. $$ The first coordinate has generalized root $1$, since $2v_1=\lambda(2v_1)$. In the second coordinate, however, the equation would require $v_2=0$, because it reads $v_2=\lambda\cdot0$. Thus no finite $\lambda$ accounts for the second direction when $v_2\ne0$.

Indeed, $\operatorname{range}(H)$ is not contained in $\operatorname{range}(E)$: the second coordinate has positive hypothesis variation but zero error variation. The determinant of the pencil is $\det(H-\lambda E)=2(1-\lambda)$, which has only one finite root, $1$. The other direction is associated with an infinite generalized eigenvalue. This is why the rank of $H$ alone does not give the number of finite ordinary roots when $E$ is singular.

Common Mistakes and Practical Checks

The safest workflow is to check the error matrix before calculating roots, then use ranks to check the answer. In the usual full-rank setting, $\operatorname{rank}(E)=p$ and $\operatorname{rank}(H)=r$ imply exactly $r$ positive roots. The ranks do not supply their numerical values; those require the actual matrices.

Using diagonal ratios without diagonal structure. Dividing entries of $H$ by entries of $E$ is justified in a shared diagonal basis, not for arbitrary matrices. Use the generalized eigenvalue equation or the symmetric matrix $E^{-1/2}HE^{-1/2}$.
Assuming a full-rank hypothesis matrix is required. If $H$ is positive semidefinite but rank deficient, it simply has fewer nonzero roots; zero eigenvalues are not counted as roots under the definition here.
Ignoring a singular error matrix. A singular $E$ invalidates the ordinary full-space whitening step. Check whether hypothesis variation lies in the error support before interpreting a reduced calculation.
Inferring sizes from ranks. Ranks determine the nonzero-root count under the stated conditions, not the strengths of those roots. Different matrices with the same ranks can yield different values.

Key takeaway: With $E$ positive definite, the MANOVA roots are the nonzero eigenvalues of $E^{-1/2}HE^{-1/2}$, and their number is exactly $\operatorname{rank}(H)$. A singular $E$ requires support or singular-pencil analysis; it cannot be treated as an ordinary full-rank error matrix.

Check Your Understanding

Use the definitions and rank conditions to answer the following questions.

If $H$ is a $5 \times 5$ positive-semidefinite matrix of rank $3$ and $E$ is positive definite, how many nonzero roots are there? How many zero eigenvalues?
Why does $E^{-1/2}HE^{-1/2}$ have only nonnegative eigenvalues when $H$ is positive semidefinite?
Two MANOVA problems have the same ranks for $H$ and $E$, with both error matrices positive definite. What can you conclude about the root counts, and what can you not conclude about the root values?
For singular $E$, what does the condition $\operatorname{range}(H)\subseteq\operatorname{range}(E)$ allow you to do?
Why is replacing a singular $E$ by a generalized inverse not, by itself, enough to justify calling the resulting eigenvalues the ordinary MANOVA roots?