Common Errors in MANOVA Criterion Interpretation

Why Interpretation Errors Matter

A MANOVA analysis can report several criteria for the same multivariate null hypothesis. Their values are not on a common scale: one criterion decreases as evidence against the null grows, while the others increase. Further, their null distributions and calibrations are not interchangeable. Correct interpretation therefore requires keeping three things distinct: the generalized roots, the criterion calculated from them, and the reference distribution used to obtain a test decision.

Use the standard MANOVA setting in which \(H\) and \(E\) are the hypothesis and error sum-of-products matrices for \(p\) responses, and \(E\) is positive definite. The generalized roots are the eigenvalues of \(E^{-1}H\). Since this matrix is similar to the symmetric positive semidefinite matrix \(E^{-1/2}HE^{-1/2}\), the roots are real and nonnegative. Write them in descending order as \(\lambda_1\geq\lambda_2\geq\cdots\geq\lambda_p\geq0\); roots equal to zero are included when needed. There are at most \(\min(p,h)\) positive roots, where \(h\) is the numerator degrees of freedom of the hypothesis.

Deriving the Criteria and Reading Their Directions

These expressions are linked, but they summarize the roots differently. Wilks’ lambda measures a determinant ratio, Pillai’s trace sums bounded contributions from all roots, the Hotelling–Lawley trace sums the roots directly, and Roy’s criterion uses only the largest root.

For a fixed root spectrum, the direction of each criterion follows directly from its formula. Increasing any root while holding the others fixed decreases Wilks’ lambda, since \(1/(1+\lambda_i)\) decreases. It increases the corresponding Pillai contribution because the derivative of \(\lambda_i/(1+\lambda_i)\) is \(1/(1+\lambda_i)^2>0\), and increases the Hotelling–Lawley sum. Increasing the largest root cannot decrease Roy’s criterion. Thus small \(\Lambda\), but large \(V\), \(U\), or \(\Theta\), points toward greater separation of the hypothesis and error variation. This direction statement does not by itself determine a p-value or guarantee that two different spectra will be ranked alike by every criterion.

Exact Null Laws Are Not the Same as Approximate Calibrations

A criterion is a statistic; its null distribution is a separate part of a test. Under the usual Gaussian MANOVA assumptions, with the model and dimensions specified, some null distributions or reductions are exact in particular settings. For example, the scalar-response reduction below has an exact \(F\) law under the null. Other familiar procedures use approximations: Bartlett’s chi-square approximation for Wilks’ lambda and F calibrations for multivariate criteria are not made exact merely by being expressed using an \(F\) or chi-square reference distribution.

The word “exact” refers to the distribution under stated assumptions and dimensions, not to the number of decimal places displayed or the accuracy of a numerical computation. Conversely, an approximation can be useful and well calibrated without being an exact null law. The exact distribution of one criterion also does not make the calibration of another criterion exact. For the special exact laws and approximations covered in earlier material on MANOVA criteria, use the conditions and degrees of freedom attached to that particular result.

Degrees of Freedom: Keep the Dimensions Separate

Three quantities are especially easy to conflate: \(p\), the number of responses; \(h\), the numerator degrees of freedom for the hypothesis; and \(\nu\), the error degrees of freedom. The number of positive roots is at most \(\min(p,h)\), but this count is not a general substitute for the degrees of freedom of an approximate F calibration. The transformation and its reference degrees of freedom must come from the particular calibration being used, which can involve more than one of these dimensions.

A useful check is the scalar-response case. When \(p=1\), under the null hypothesis and the standard Gaussian MANOVA model, \(H/\sigma^2\) and \(E/\sigma^2\) are independent chi-square random variables with \(h\) and \(\nu\) degrees of freedom, respectively. Therefore

This is an exact reduction, not a general recipe for assigning degrees of freedom to multivariate F approximations. It follows from the definition of the \(F\) distribution: the ratio of independent chi-square variables divided by their respective degrees of freedom has an \(F\) distribution.

Different Criteria Need Not Give Identical Tests

All four criteria address the same null hypothesis, but they are different summaries of the root spectrum. Wilks, Pillai, and Hotelling–Lawley use all roots with different weights; Roy uses only the largest. Their numerical values are consequently not comparable as though they were alternative readings of one common scale. Their test statistics are also calibrated using criterion-specific null laws or approximations. Different p-values or decisions are therefore possible, especially when effects are spread across several roots rather than concentrated in one.

Common Mistakes and a Practical Check

• Using the wrong root for Roy. The mistaken reasoning is “Roy uses the first root as printed.” The correction is to order the roots descending and use \(\lambda_1\), the largest root.
• Reversing the evidence direction. The mistaken reasoning is “a larger criterion always means stronger evidence.” The correction is that Wilks’ lambda is evidence in the small direction; Pillai, Hotelling–Lawley, and Roy are evidence in the large direction.
• Calling an approximation exact. The mistaken reasoning is “the output says chi-square or F, so the null law must be exact.” The correction is to check the assumptions and the named calibration: an approximate reference distribution remains approximate.
• Substituting the number of roots for degrees of freedom. The mistaken reasoning is “there are \(s\) positive roots, so the test must have \(s\) numerator degrees of freedom.” The correction is to distinguish \(p\), \(h\), and \(\nu\), and to use the degrees of freedom specified by the particular exact reduction or approximation.
• Expecting all criteria to agree. The mistaken reasoning is “the null hypothesis is the same, so every criterion must produce the same ranking and decision.” The correction is that different summaries emphasize different aspects of the root spectrum and have different calibrations.

Before interpreting output, write down the ordered roots, calculate or verify the criterion, state its evidence direction, and identify the null calibration and its degrees of freedom. This short sequence catches most interpretation errors before they become claims about a scientific effect.