Tutorials › Multivariate Statistics (Graduate) › Roy’s Largest Root as the Strongest Hypothesis Direction

Home  /  Tutorials  /  MANOVA Criteria and Distributions

MANOVA Criteria and Distributions · Tutorial 339 of 975

Roy's Largest Root as the Strongest Hypothesis Direction

Learn how Roy’s largest root identifies the response contrast with the greatest hypothesis-to-error ratio, and how to find and interpret its maximizing direction.

Advanced 10 min read

What You'll Learn

  • Express Roy’s largest root as a maximum generalized Rayleigh quotient.
  • Connect each response contrast to its hypothesis and error sums of squares.
  • Find the maximizing direction by whitening the error matrix.
  • Characterize all maximizing directions and their uniqueness.
  • Distinguish a strongest hypothesis-to-error direction from the direction with the largest hypothesis sum of squares alone.

Why a Largest Root Selects a Direction

A multivariate hypothesis can affect several responses, but its evidence need not be equally strong in every linear combination of them. For example, a treatment effect might be modest for each response considered separately yet pronounced for a particular contrast, such as one response minus a weighted combination of others. Roy’s largest root focuses on the contrast for which the hypothesis evidence is strongest relative to its error variation.

Let \(p\) be the number of responses, and let \(H\) and \(E\) be the \(p\times p\) hypothesis and error sum-of-squares-and-cross-products matrices for a MANOVA test. In this tutorial, assume that \(H\) is symmetric positive semidefinite and that \(E\) is symmetric positive definite. The latter full-rank condition ensures that every nonzero response contrast has positive error sum of squares and that the ratios below are finite.

Definition: For a nonzero vector \(v\in\mathbb{R}^p\), the generalized Rayleigh quotient of \(H\) relative to \(E\) is $$ R(v)=\frac{v^{T}Hv}{v^{T}Ev}. $$ Roy’s largest root is the largest generalized eigenvalue of the pair \((H,E)\). Under the stated conditions, it is also $$ \lambda_{\max}=\max_{v\ne 0}\frac{v^{T}Hv}{v^{T}Ev}. $$ Any nonzero maximizing vector \(v\) specifies a response contrast with the greatest hypothesis-to-error ratio.

The quotient is unchanged by multiplying \(v\) by any nonzero scalar, so a direction matters, not its length. In the MANOVA construction, \(v^{T}Hv\) is the hypothesis sum of squares for the scalar response formed by applying \(v\) to the response vector, and \(v^{T}Ev\) is the corresponding error sum of squares. Thus \(R(v)\) compares hypothesis variation with error variation for that particular contrast. The maximizing contrast is not necessarily one of the original responses: its coefficients can combine several responses.

Finding the Maximizing Direction

The denominator depends on \(v\), so this is not generally an ordinary eigenvalue problem for \(H\). A useful change of variables turns it into one. Since \(E\) is positive definite, it has a symmetric positive-definite square root \(E^{1/2}\) and inverse square root \(E^{-1/2}\). Define \(x=E^{1/2}v\) and \(A=E^{-1/2}HE^{-1/2}\). Then \(v=E^{-1/2}x\), and direct substitution gives $$ v^{T}Ev=x^{T}x,\qquad v^{T}Hv=x^{T}Ax,\qquad R(v)=\frac{x^{T}Ax}{x^{T}x}. $$ The matrix \(A\) is symmetric and positive semidefinite. The ordinary Rayleigh quotient of a symmetric matrix is maximized by a unit eigenvector for its largest eigenvalue. Translating that eigenvector back to \(v\) gives the generalized eigenvector formulation.

Theorem: If \(H\) is symmetric positive semidefinite and \(E\) is symmetric positive definite, then $$ \max_{v\ne 0}\frac{v^{T}Hv}{v^{T}Ev} =\lambda_{\max}\!\left(E^{-1/2}HE^{-1/2}\right), $$ where \(\lambda_{\max}(A)\) denotes the largest eigenvalue of a symmetric matrix \(A\). A vector \(v\ne 0\) attains the maximum if and only if \(E^{1/2}v\) is an eigenvector of \(E^{-1/2}HE^{-1/2}\) for its largest eigenvalue. Equivalently, every maximizing \(v\) satisfies $$ Hv=\lambda_{\max}Ev. $$

Proof. By the spectral theorem, \(A=E^{-1/2}HE^{-1/2}\) has an orthonormal eigenbasis \(q_1,\ldots,q_p\), with eigenvalues \(\lambda_1\ge\cdots\ge\lambda_p\ge 0\). For any nonzero \(x=\sum_{j=1}^{p}c_jq_j\), $$ \frac{x^{T}Ax}{x^{T}x} =\frac{\sum_{j=1}^{p}\lambda_jc_j^2}{\sum_{j=1}^{p}c_j^2} \le \lambda_1. $$ Equality holds exactly when every \(c_j\) corresponding to an eigenvalue strictly less than \(\lambda_1\) is zero. Such an \(x\) is an eigenvector for \(\lambda_1\), and every eigenvector for \(\lambda_1\) attains equality. The invertible change of variables \(x=E^{1/2}v\) maps all nonzero \(v\) to nonzero \(x\), so the same maximum and equality characterization hold for \(R(v)\). If \(Ax=\lambda_1x\) and \(x=E^{1/2}v\), multiplication by \(E^{1/2}\) gives \(Hv=\lambda_1Ev\). Conversely, if \(Hv=\lambda_1Ev\), multiplication by \(E^{-1/2}\) shows that \(E^{1/2}v\) is an eigenvector of \(A\) for \(\lambda_1\). \(\square\)

Worked Example: A Diagonal Pair of Matrices

Let $$ H=\begin{pmatrix}8&0\\0&2\end{pmatrix}, \qquad E=\begin{pmatrix}2&0\\0&1\end{pmatrix}. $$ For \(v=(v_1,v_2)^T\ne 0\), $$ R(v)=\frac{8v_1^2+2v_2^2}{2v_1^2+v_2^2}. $$ The quotient can be written as a weighted average of \(4\) and \(2\): $$ R(v)= \frac{4(2v_1^2)+2(v_2^2)}{2v_1^2+v_2^2} \le 4. $$ The bound is attained when \(v_2=0\) and \(v_1\ne 0\). Indeed, \(R((1,0)^T)=8/2=4\), whereas \(R((0,1)^T)=2/1=2\). Thus Roy’s largest root is \(4\), and the strongest contrast is proportional to the first response alone.

What the Maximizing Direction Means

The eigenvector supplies the coefficients of a response contrast. If the response vector is \(Y=(Y_1,\ldots,Y_p)^T\), the contrast is \(v^TY\). The maximizing direction selects the linear combination whose hypothesis sum of squares, relative to its error sum of squares, is largest. It does not necessarily select the response with the largest marginal hypothesis sum of squares, nor does it identify a causal mechanism. It describes the strongest direction in the specified MANOVA comparison, measured relative to the error matrix.

A second property clarifies when this direction is uniquely determined. Directions are understood up to multiplication by a nonzero scalar, since that operation does not change the contrast ratio.

Theorem: If the largest eigenvalue of \(E^{-1/2}HE^{-1/2}\) has multiplicity one, the maximizing contrast direction is unique up to nonzero scaling. If its multiplicity is \(k\), the maximizing directions are exactly the nonzero vectors obtained by mapping its \(k\)-dimensional largest-eigenvalue eigenspace back through \(E^{-1/2}\).

Proof. Write \(A=E^{-1/2}HE^{-1/2}\), and let \(Q\) be its eigenspace for the largest eigenvalue. The first theorem shows that \(v\) maximizes the quotient exactly when \(x=E^{1/2}v\) belongs to \(Q\) and is nonzero. Equivalently, the set of maximizing vectors is $$ \{E^{-1/2}x:x\in Q,\ x\ne 0\}. $$ Because \(E^{-1/2}\) is invertible, this set has the same dimension as \(Q\). If that dimension is one, all its nonzero vectors are scalar multiples of each other. If it is \(k\), the maximizing directions comprise the image of that \(k\)-dimensional eigenspace, as claimed. \(\square\)

Worked Example: A Mixed Contrast Can Be Strongest

Consider $$ E=\begin{pmatrix}2&0\\0&1\end{pmatrix}, \qquad H=\begin{pmatrix}6&\sqrt{2}\\\sqrt{2}&3\end{pmatrix}. $$ Here \(E^{1/2}=\operatorname{diag}(\sqrt{2},1)\) and \(E^{-1/2}=\operatorname{diag}(1/\sqrt{2},1)\). Therefore $$ A=E^{-1/2}HE^{-1/2} =\begin{pmatrix}3&1\\1&3\end{pmatrix}. $$ The eigenvectors \((1,1)^T\) and \((1,-1)^T\) have eigenvalues \(4\) and \(2\), respectively. The largest root is \(4\); a maximizing direction satisfies \(E^{1/2}v\propto(1,1)^T\), so \(v\propto(1/\sqrt{2},1)^T\).

Check the quotient directly using \(v=(1/\sqrt{2},1)^T\). Its denominator is $$ v^TEv=2(1/2)+1=2, $$ and its numerator is $$ v^THv=6(1/2)+2\sqrt{2}(1/\sqrt{2})(1)+3=3+2+3=8. $$ Hence \(R(v)=8/2=4\). The strongest contrast combines both responses; it is not simply either coordinate response.

Worked Example: Why the Largest Entry of \(H\) Is Not Enough

Take $$ H=\begin{pmatrix}9&0\\0&4\end{pmatrix}, \qquad E=\begin{pmatrix}9&0\\0&1\end{pmatrix}. $$ The first response has the larger hypothesis sum of squares, since \(9>4\). But for \(v=(1,0)^T\), the quotient is \(9/9=1\); for \(v=(0,1)^T\), it is \(4/1=4\). More generally, $$ R(v)=\frac{9v_1^2+4v_2^2}{9v_1^2+v_2^2}\le 4, $$ because \(9v_1^2+4v_2^2\le 4(9v_1^2+v_2^2)\), with equality when \(v_1=0\). Thus the second response is the strongest direction even though its hypothesis sum of squares is smaller. Its error sum of squares is smaller still, producing the larger ratio.

Stationarity and Interpretation in Practice

The generalized eigenvalue equation can also be obtained by optimizing directly under a normalization. Since \(R(v)\) is scale-invariant and \(v^TEv>0\) for \(v\ne 0\), the maximization can be restricted to \(v^TEv=1\). At a maximizing vector, a Lagrange multiplier condition yields \(Hv=\lambda Ev\). This condition identifies candidate stationary directions; selecting the largest generalized eigenvalue, rather than an arbitrary eigenvalue, is essential for finding the global maximum.

Result: Every stationary direction of \(R(v)\) subject to \(v^TEv=1\) satisfies \(Hv=\lambda Ev\), with \(\lambda=R(v)\). The global maximizing directions are those associated with the largest generalized eigenvalue.

Derivation. Define the Lagrangian \(L(v,\lambda)=v^THv-\lambda(v^TEv-1)\). Since \(H\) and \(E\) are symmetric, its gradient with respect to \(v\) is \(2Hv-2\lambda Ev\). At a constrained stationary point this gradient is zero, so \(Hv=\lambda Ev\). Premultiplying by \(v^T\) and using \(v^TEv=1\) gives \(v^THv=\lambda\), hence \(\lambda=R(v)\). The first theorem establishes that the largest of these generalized eigenvalues is the global maximum. \(\square\)

In applications, Roy’s criterion is especially interpretable when a single response contrast carries a strong effect relative to error. The maximizing direction provides an exploratory description of that concentration, while the value of the root summarizes its ratio. If multiple largest eigenvalues are tied, there is no single uniquely preferred contrast: every direction in the corresponding maximizing subspace has the same greatest ratio.

Common Mistakes

  • Maximizing \(v^THv\) without accounting for \(E\). The relevant objective is the ratio \(v^THv/(v^TEv)\), not hypothesis variation alone. The third example shows that a response with smaller hypothesis sum of squares can have the larger ratio.
  • Treating \(H\)'s ordinary eigenvalues as Roy’s roots. Roy’s largest root is a generalized eigenvalue of \((H,E)\), or an ordinary eigenvalue of \(E^{-1/2}HE^{-1/2}\). It equals an eigenvalue of \(H\) only in special cases, such as \(E=I\).
  • Reporting an eigenvector without recognizing its scale is arbitrary. Any nonzero scalar multiple gives the same contrast direction and quotient. Interpret coefficients relative to one another, not as a uniquely scaled vector.
  • Assuming the maximizing direction must be unique. It is unique up to scaling only when the largest transformed eigenvalue is simple. A repeated largest eigenvalue gives a whole maximizing subspace.
Key takeaway: Roy’s largest root is the maximum hypothesis-to-error sum-of-squares ratio over all nonzero response contrasts. Whitening by \(E\) converts the problem to an ordinary symmetric eigenvalue problem, and the top eigenvector, mapped back to response-contrast coordinates, gives the strongest direction.

Check Your Understanding

Use the quotient and maximizing-direction results to answer the following.

  1. Why does multiplying a nonzero contrast vector by a scalar leave its hypothesis-to-error ratio unchanged?
  2. For positive-definite \(E\), what symmetric matrix has largest ordinary eigenvalue equal to Roy’s largest root?
  3. If the largest eigenvalue of \(E^{-1/2}HE^{-1/2}\) has multiplicity greater than one, what does that imply about the maximizing contrasts?
  4. In the third worked example, why does the second response maximize the ratio even though its hypothesis sum of squares is smaller?
  5. What generalized eigenvalue equation is satisfied by a maximizing contrast direction?

Previous

Next

Back to Tutorials
Clinical Biostats