Interpreting and Reporting General Linear Tests

What a Multivariate Test Report Must Say

A multivariate test can answer a focused scientific question while still being easy to misreport. A small p-value for an overall hypothesis does not, by itself, say which response or coefficient differs. A large p-value does not establish that the coefficients are equal. A useful report therefore makes the hypothesis and the calibration visible, then limits its conclusion to what that test actually addresses.

Consider the multivariate linear model \(Y=XB+U\), where \(Y\) is an \(n\times q\) response matrix, \(X\) is an \(n\times p\) design matrix, and \(B\) is a \(p\times q\) coefficient matrix. A general linear hypothesis can be written \(LBA=C\), where \(L\) selects or combines predictor-space coefficients, \(A\) selects or combines response-space directions, and \(C\) specifies the hypothesized value. The dimensions are \(L:s\times p\), \(A:q\times t\), and \(C:s\times t\). The null hypothesis is the set of coefficient matrices satisfying this equation.

The number of rows of \(L\), or the number of entries in \(C\), is not automatically the effective rank. Some restrictions may repeat information already contained in others. The effective rank matters because it describes the dimension of the tested hypothesis and, for example, determines the degrees of freedom of an asymptotic Wald reference distribution.

Why the Effective Rank Is a Rank

For full-column-rank \(X\), all entries of \(B\) are identifiable. Using column-wise vectorization, the identity \(\operatorname{vec}(LBA)=(A^\mathsf{T}\otimes L)\operatorname{vec}(B)\) converts the matrix restrictions into a linear system. This gives a direct way to count independent restrictions.

Proof. Vectorization writes the restriction map as multiplication by \(A^\mathsf{T}\otimes L\). For any matrices \(G\) and \(H\), \(\operatorname{rank}(G\otimes H)=\operatorname{rank}(G)\operatorname{rank}(H)\). To see this, write each matrix in a rank factorization: \(G=P_G Q_G\), \(H=P_H Q_H\), where the columns of each \(P\) and rows of each \(Q\) are linearly independent. Then

The first factor has full column rank and the second has full row rank, so their product has rank equal to the inner dimension, namely \(\operatorname{rank}(G)\operatorname{rank}(H)\). Applying this to \(G=A^\mathsf{T}\) and \(H=L\) gives the result. Since \(X\) has full column rank, this restriction map acts on an identifiable coefficient space. \(\square\)

If \(X\) is rank deficient, this product is not by itself enough: the restriction map must be considered only on the identifiable coefficient space. A report should then make clear that the tested restrictions are estimable. Redundant rows also need careful handling: removing a repeated equation preserves the hypothesis only when its right-hand side obeys the same linear dependence.

Proof. Write the restrictions as \(D\beta=d\), where \(\beta=\operatorname{vec}(B)\). If \(S\) is invertible, \(D\beta=d\) holds exactly when \(SD\beta=Sd\) holds, because multiplication by \(S\) is reversible. Also, \(\operatorname{rank}(SD)=\operatorname{rank}(D)\), since an invertible transformation preserves linear dependence among rows. Thus the hypothesis and its effective rank are unchanged. \(\square\)

Worked Examples: From Restrictions to a Report

Choosing and Reporting the Calibration

A test statistic has no interpretable p-value without a reference distribution and calibration. State both, including degrees of freedom where applicable. For an asymptotic Wald test, report the chi-square statistic, its degrees of freedom (the effective restriction rank under the usual regularity conditions), and that the calibration is asymptotic. The examples above use this form of report.

Classical multivariate analyses may instead report Wilks’ lambda, Pillai’s trace, the Hotelling–Lawley trace, or Roy’s largest root, together with the reference calibration used by the software. These statistics are different summaries of the same hypothesis and error information; their reported p-values need not agree exactly. Their finite-sample reference distributions or approximate \(F\) calibrations depend on the design, ranks, and assumptions. A careful report identifies the statistic actually used and states whether its calibration is exact under the stated model or approximate. Do not label an approximate \(F\) calibration as exact.

For classical finite-sample multivariate calibrations, the model assumptions must justify the chosen method; a common setting assumes independent multivariate normal errors with a common response covariance matrix and appropriate full-rank design and hypothesis matrices. An asymptotic Wald calibration instead requires the relevant regularity conditions, including estimability and a consistent covariance estimate for the unrestricted estimator. The test name alone does not establish that its calibration is justified.

A complete report ordinarily gives the scientific question in context, the tested restrictions, the effective rank or the degrees of freedom that reflect it, the chosen statistic and observed value, the reference calibration, and the p-value. It should also state the scope of the conclusion. If estimates or confidence intervals are important to interpretation, report them separately; a test p-value is not an effect-size estimate.

Common Reporting Pitfalls

• Counting equations instead of independent restrictions. A matrix equation can contain redundant rows or entries. Check the rank of the restriction map on the identifiable coefficient space.
• Reporting only a p-value. Without the hypothesis, statistic, and reference calibration, a reader cannot tell what was tested or how the p-value was obtained.
• Turning an omnibus rejection into a coefficient-level claim. A joint test can reject even when it does not identify a particular response or coefficient. Follow-up tests or contrasts must be specified and their multiplicity addressed.
• Treating nonrejection as proof of equality. A nonsignificant result is not an equivalence conclusion. Evidence for practical similarity requires a suitable equivalence hypothesis and design.
• Calling an approximation exact. Name the calibration accurately and state the assumptions that support it.

These distinctions matter because a general linear test summarizes evidence about a specified collection of restrictions. It is not a substitute for interpreting the estimated coefficients, checking the model assumptions, or designing follow-up comparisons. Keeping the hypothesis and calibration explicit makes the report reproducible and keeps the conclusion within the evidence.