Choosing an Approximation by Design Dimensions

Why the Design Dimensions Matter

MANOVA criteria summarize the same set of hypothesis roots in different ways, but their null calibrations do not respond identically to the design. A chi-square approximation for Wilks' lambda, for example, uses a degrees-of-freedom count tied to both the response dimension and the hypothesis rank. F calibrations also depend on the error degrees of freedom, and their formulas can involve the number of available roots. Choosing a calibration therefore starts with the design, not with whichever test statistic happens to look most familiar.

This tutorial compares the main calibration dimensions and gives a practical way to assess them. The comparison does not produce a universal ranking: an approximation can be more or less credible depending on the design and the null distribution it is intended to represent. In particular, an exact reduction in a special design should take precedence over a general approximation.

Design Dimensions and Root Count

Let \(p\) be the response dimension, \(q\) the rank of the tested hypothesis, and \(v\) the error degrees of freedom. Under the central MANOVA null, assume the usual multivariate normal linear model, with independent hypothesis and error sums-of-squares-and-cross-products matrices and a nonsingular error matrix. The MANOVA roots are the \(s\) eigenvalues of \(E^{-1}H\), counted with zeros when necessary, where \(s=\min(p,q)\). At most \(s\) of these roots can be nonzero.

The distinction matters because the root count in a particular data set can be smaller than its design maximum. For example, if \(H=0\), every root is zero, even though \(s\) may be positive. Conversely, when the hypothesis matrix has its generic rank, the design may permit all \(s\) roots to be nonzero. The calibration formulas are defined by the model dimensions, not by counting the roots that happen to appear numerically in one sample.

To verify this, write \(E^{-1}H=E^{-1/2}(E^{-1/2}HE^{-1/2})E^{1/2}\). Thus \(E^{-1}H\) is similar to the symmetric positive semidefinite matrix \(E^{-1/2}HE^{-1/2}\); similarity preserves eigenvalues. The latter matrix has nonnegative eigenvalues. Multiplication on either side by the invertible matrix \(E^{-1/2}\) preserves rank, so its rank is \(\operatorname{rank}(H)\). A positive semidefinite matrix has exactly its rank many nonzero eigenvalues. Finally, \(H\) is formed from a hypothesis of rank \(q\) in a \(p\)-dimensional response space, so \(\operatorname{rank}(H)\leq\min(p,q)\). \(\square\)

What the Standard Calibrations Use

Write the roots in decreasing order as \(\theta_1\geq\cdots\geq\theta_s\geq0\), including zeros when needed. The four criteria are Wilks' lambda, Pillai's trace, the Hotelling–Lawley trace, and Roy's largest root. Their standard null calibrations use different summaries of these roots. The following parameterizations make the design dependence explicit.

The degrees of freedom for Pillai and Hotelling–Lawley are not generally common. In the usual parameterization, let \(m=(|p-q|-1)/2\) and \(n=(v-p-1)/2\). Pillai's F degrees of freedom are \(s(2m+s+1)\) and \(s(2n+s+1)\). Since \(2m+s+1=|p-q|+s=\max(p,q)\), the first is \(s\max(p,q)=pq\); since \(2n+s+1=v-p+s\), the second is \(s(v-p+s)\). Hotelling–Lawley's F degrees of freedom are \(s(2m+s+1)\) and \(2(sn+1)\). These formulas apply provided the design has the positive error degrees of freedom required by the underlying MANOVA calibration.

These formulas also show why a single dimension is not enough to choose a method. The Bartlett degrees of freedom are \(pq\), while its multiplier depends on \(v,p,q\). The Pillai and Hotelling–Lawley F calibrations generally use different degrees of freedom and transform different criteria. Roy's calibration has numerator degrees of freedom \(\max(p,q)\), not \(pq\), and focuses on one root. A large response dimension can therefore affect calibrations differently from a large hypothesis rank.

A Design-Based Choice

A defensible choice can be made in a fixed order. First use the actual rank of the tested hypothesis, after removing redundant hypothesis directions; a nominal count of contrasts is not necessarily the rank \(q\). Next record \(p\) and \(v\), checking that the error matrix is nonsingular and that the correction or F degrees of freedom are admissible. Then compute \(s=\min(p,q)\), the maximum possible number of nonzero roots. Finally ask whether the design has a known exact reduction before relying on an approximation.

When \(p=1\), the problem is univariate and an exact F test is available under the classical model. When \(q=1\), the rank-one exact reduction applies. Outside these cases, Bartlett's chi-square and the named F calibrations are approximations or, for Roy, an upper-bound calibration. The formulas alone do not establish that one will be more accurate for every choice of dimensions.

The error degrees of freedom \(v\) deserve particular attention. In the Bartlett expression they affect the scale \(b\); in the F calibrations they affect the denominator degrees of freedom and scaling. Limited \(v\) means the calibration is being used with less error information, so interpreting a reference tail probability as exact is especially unwarranted. Meanwhile \(p\) and \(q\) determine the scale of the hypothesis: their product appears in the Wilks chi-square degrees of freedom and in the Pillai and Hotelling–Lawley transformations.

A second pitfall is to call the Bartlett chi-square or a general F approximation “exact” because the formula is explicit. Exactness is a distributional claim, not a consequence of having a closed-form statistic. Use an exact reduction when the design supplies one; otherwise state which approximation is being used and retain its assumptions and degrees of freedom. This makes the chosen calibration auditable and prevents a design feature such as small \(v\) or a high response dimension from being hidden behind a single reported p-value.