Common Errors In Covariance Hypothesis Tests

Why Covariance-Test Errors Are Easy to Make

Multivariate covariance tests combine several kinds of reasoning: the hypothesis concerns a matrix, the statistic often depends on determinants, and its reference distribution depends on model assumptions. A calculation can look plausible while answering the wrong question. In particular, equal correlations do not require equal covariances, a pooled estimate is not a substitute for each group’s estimate, and the degrees of freedom are not simply the number of variables.

Let \(p\) be the number of variables and \(g\) the number of independent groups. For group \(i\), write \(n_i\) for its sample size, \(S_i\) for its usual unbiased sample covariance matrix, and \(N=\sum_{i=1}^g n_i\). The pooled within-group covariance estimate is

This estimate combines within-group scatter. It is useful in procedures whose model assumes a common covariance, but it does not describe the separate covariance estimates under a hypothesis that allows each group to have its own matrix.

Covariance Equality Is Not Correlation Equality

For a positive-definite covariance matrix \(\Sigma\), let \(D\) be the diagonal matrix of marginal standard deviations. Its correlation matrix is \(R=D^{-1}\Sigma D^{-1}\). The diagonal entries of \(\Sigma\) are variances; the off-diagonal entries are covariances. Correlations standardize those off-diagonal entries by the marginal standard deviations.

The distinction follows directly from the scaling relation. If \(D\) is positive diagonal, then

For the first identity, the diagonal standard-deviation matrix of \(D\Sigma D\) is \(D\) times the diagonal standard-deviation matrix of \(\Sigma\). Substitution in the definition of the correlation matrix cancels the factors of \(D\). For the second, use multiplicativity of the determinant: \(|D\Sigma D|=|D||\Sigma||D|\). Thus changing measurement scales can leave correlations unchanged while changing both covariances and determinants.

This scaling also explains a common determinant error. A determinant is scale-dependent: it changes when variables are measured in different units. Consequently, a determinant calculation based on raw covariance matrices cannot automatically be interpreted as a test of correlation equality. The hypothesis and the statistic must match.

Group-Specific Estimates and the Pooled Estimate

For the likelihood-ratio comparison of equal group covariance matrices with unrestricted group covariance matrices, the uncorrected Box’s \(M\) statistic is

The pooled determinant and the group-specific determinants have different roles in this expression. Substituting \(S_p\) for every \(S_i\) erases the differences the statistic is meant to assess. Conversely, using an unweighted average of the \(S_i\) in place of \(S_p\) ignores the different within-group degrees of freedom. The weights are \(n_i-1\), not \(n_i\), because each group mean has been estimated.

A negative value beyond numerical rounding is therefore a warning to check the weights, group degrees of freedom, determinant calculations, and whether each matrix is positive definite. When a sample covariance matrix is singular, its log determinant is not finite, so the usual determinant-based calculation cannot be carried out as written.

Count Degrees of Freedom From the Restrictions

A frequent mistake is to use \(p\), or the number of off-diagonal entries, as the degrees of freedom for every covariance test. The appropriate count is the difference in the number of free parameters in the alternative and null models, provided the models are regular and the restrictions are locally independent.

For equality of \(g\) unrestricted covariance matrices, the alternative has \(gm\) parameters and the common-covariance null has \(m\). The difference is \((g-1)p(p+1)/2\), which is the usual chi-square degrees of freedom for the likelihood-ratio comparison. For equality of correlation matrices with group-specific variances, each correlation matrix has \(q=p(p-1)/2\) free correlations. The alternative has \(g(p+q)\) parameters; the null has \(gp+q\), since variances remain group-specific but correlations are common. The difference is \((g-1)q\).

These counts are derived by listing the free parameters under each model and subtracting. They also show why the two hypotheses have different degrees of freedom: equality of covariance matrices restricts variances as well as correlations, while equality of correlation matrices does not.

When a Chi-Square Approximation Is Justified

A chi-square reference distribution is not a consequence of writing down a likelihood-ratio statistic. Wilks’ theorem supplies the usual asymptotic calibration under regularity conditions. In this setting, the basic conditions include independent observations within and across groups, multivariate normal sampling for the classical covariance likelihood, positive-definite population covariance matrices, fixed \(p\) and \(g\), and sample sizes that grow so the null and alternative parameters are identifiable. The null restrictions must be smooth and locally independent, and the true parameter must be an interior point of the parameter space.

Under these conditions, the likelihood-ratio statistic, expressed on the usual \(-2\log\)-likelihood scale, converges in distribution under the null to a chi-square random variable with degrees of freedom given by the dimension difference. For Box’s \(M\), a finite-sample correction is commonly applied to improve the approximation; the corrected statistic and its calibration should not be confused with the uncorrected \(M\) value. The specific approximation may still be poor for small samples, especially when dimension is large relative to group sizes.

Normality matters because the classical likelihood and its chi-square calibration are derived under a distributional model, not simply from the fact that \(S_i\) are sample covariance matrices. Strong nonnormality, influential observations, dependence, near-singular estimates, or dimensions that increase with sample size can undermine the approximation. A chi-square result should therefore be reported as an approximation whose conditions have been assessed, not as an exact finite-sample fact.

Common Mistakes and Corrections

• Wrong: “The groups have equal correlations, so they have equal covariance matrices.” Correction: Equal correlations permit group-specific marginal variances. State which matrix is under test.
• Wrong: “The pooled covariance is the estimate for each group under the alternative.” Correction: The pooled estimate represents a common within-group covariance; the separate \(S_i\) retain the group-specific information.
• Wrong: “The degrees of freedom are \(p\).” Correction: Count free parameters in the specified alternative and null, then take their difference under regularity conditions.
• Wrong: “Equal determinants mean equal covariance matrices.” Correction: A determinant is one scalar summary. Different matrices can have the same determinant, as the worked example illustrates.
• Wrong: “Every likelihood-ratio statistic has an exact chi-square distribution.” Correction: The familiar chi-square calibration is generally asymptotic and depends on model and regularity assumptions.
• Wrong: “A negative Box’s \(M\) value is just a possible sampling outcome.” Correction: With positive-definite inputs and the stated formula, concavity of log determinant implies \(M\geq0\); check the computation or matrix inputs.