Worked Calibration of MANOVA Criteria

From Observed Roots to a Calibrated Test

A MANOVA calculation does not end when the hypothesis and error sums-of-squares-and-cross-products matrices have been reduced to their roots. Those roots summarize the separation attributed to the hypothesis relative to residual variation; a criterion such as Wilks’ lambda or Pillai’s trace then compresses the root spectrum to one number. To assess that number, one needs a reference distribution and its degrees of freedom.

This tutorial works through that calibration. The F transformations below are approximations in the general multivariate case, not a claim that every criterion has an exact F null distribution. The exact reductions in special cases are distinct. Throughout, let \(p\) be the response dimension, \(h\) the rank of the tested hypothesis, \(v\) the error degrees of freedom, and \(\lambda_1,\ldots,\lambda_s\) the MANOVA roots, counted with multiplicity and including any zero roots, where \(s=\min(p,h)\). Under the usual nonsingular error-matrix setup, these roots are nonnegative.

Reference F Calibrations

For the common F approximations below, define \(s=\min(p,h)\), \(r=\max(p,h)\), \(m=(|p-h|-1)/2\), and \(n=(v-p-1)/2\). The formulas require positive reference degrees of freedom; for Wilks’ Rao approximation, the displayed \(t\) must also be defined and positive. These transformations are used with the observed criteria to obtain an F statistic and its reference degrees of freedom.

The formulas are not interchangeable: each uses its own criterion and, in general, its own degrees of freedom. In particular, do not infer a reference distribution from the numerical size of a criterion alone. Once a transformation gives \(F_{\mathrm{obs}}\) with degrees of freedom \(d_1,d_2\), the approximate p-value is the upper-tail probability \(P(F_{d_1,d_2}\geq F_{\mathrm{obs}})\).

Turning an F Statistic into a p-Value

The F tail can be written using the regularized incomplete beta function \(I_x(a,b)\). This provides a precise expression for the p-value and is also a useful check on numerical software.

Proof. The standard F-to-beta transformation says that \(U=d_1F/(d_1F+d_2)\) has a beta distribution with parameters \(d_1/2,d_2/2\). The event \(F\geq f\) is equivalent, since the transformation is increasing, to \(U\geq u\), where \(u=d_1f/(d_1f+d_2)\). Therefore its probability is \(1-I_u(d_1/2,d_2/2)\). The beta symmetry identity \(1-I_u(a,b)=I_{1-u}(b,a)\) gives the result because \(1-u=d_2/(d_2+d_1f)=x\). \(\square\)

The beta expression is an exact way to evaluate the tail of the chosen F reference distribution. It does not make the MANOVA calibration itself exact: when the F transformation is approximate, the resulting MANOVA p-value remains approximate.

Worked Calibration From One Root Spectrum

Worked Tail Calculations and Numerical Checks

For \(F_{4,40}\), the beta-tail expression simplifies because the beta parameters in the upper-tail form are \(20\) and \(2\). Specifically, \(I_z(20,2)=z^{20}(21-20z)\), where \(z=40/(40+4f)\). This closed form makes a useful check on reported probabilities.

What Calibration Does—and Does Not—Say

Calibration translates a criterion into a statistic that can be compared with a reference distribution. It does not change the data, the hypothesis, or the meaning of the original criterion. Nor does a small approximate p-value prove that the null hypothesis is false; it quantifies how extreme the calculated statistic would be under the stated reference distribution and calibration.

A useful check is that increasing any one root while holding the others fixed decreases Wilks’ lambda and does not decrease the other three criteria. For Wilks’ lambda, each factor \((1+\lambda_i)^{-1}\) decreases. Each summand \(\lambda_i/(1+\lambda_i)\) in Pillai’s trace increases because its derivative is \(1/(1+\lambda_i)^2>0\). The Hotelling–Lawley trace increases by the same root increment, and the maximum root cannot decrease when a root increases. This verifies the direction of evidence in the criteria, though their calibrated p-values need not coincide.