Robust Tests of Equality Across Covariance Matrices

What Is Being Compared?

When several independent groups are measured on the same multivariate variables, a natural question is whether they have the same scatter. Under nonnormality or contamination, however, the usual sample covariance can be highly sensitive to a small number of unusual observations. A robust comparison instead tests equality of specified robust scatter functionals. Its uncertainty must also be estimated separately for each group: a noisier scatter estimate should contribute less evidence than a more precise one.

The inferential target is the scatter functional evaluated at each group’s distribution. This is not automatically the covariance matrix of a hypothetical uncontaminated population. That distinction is central when interpreting a rejection.

Definition: Let $F_g$ be the distribution in independent group $g$, for $g=1,\ldots,G$, and let $S(F_g)$ be a specified robust scatter functional taking values in the symmetric $p \times p$ matrices. The equality hypothesis is $$ H_0:S(F_1)=S(F_2)=\cdots=S(F_G). $$ Write $q=p(p+1)/2$, and let $\operatorname{vech}(A)$ denote the vector of the distinct entries of a symmetric matrix $A$, in a fixed order. Set $\theta_g=\operatorname{vech}\{S(F_g)\}$ and let $\hat\theta_g$ be the corresponding estimate.

The robust scatter functional and its estimator must be specified for the application. For example, a robust M-estimator of scatter targets the functional defined by its estimating equations, with any required consistency correction applied. The asymptotic covariance used below must match that estimator and its actual estimating procedure, including the effect of estimating location when location is a nuisance parameter.

Contrasts and Their Uncertainty

A convenient way to express equality is to compare every group with group 1. Stack the estimates as $\hat\theta=(\hat\theta_1^\mathsf{T},\ldots,\hat\theta_G^\mathsf{T})^\mathsf{T}$, and define the contrast matrix $C$ by $$ C\theta= \begin{pmatrix} \theta_2-\theta_1\\ \theta_3-\theta_1\\ \vdots\\ \theta_G-\theta_1 \end{pmatrix}. $$ Thus $C$ has $(G-1)q$ rows, and the null hypothesis is equivalent to $C\theta=0$. Suppose group $g$ has sample size $n_g$, and its robust estimate satisfies $$ \sqrt{n_g}(\hat\theta_g-\theta_g)\xrightarrow{d}N_q(0,V_g), $$ where $V_g$ is positive definite and consistently estimated by $\hat V_g$. The groups are independent, so the estimated covariance matrix of the stacked estimates is block diagonal, with group $g$'s block equal to $\hat V_g/n_g$.

Theorem: Under the null, suppose $G$ and $p$ are fixed, $n_g/n\to\lambda_g>0$ for $n=\sum_g n_g$, the groups are independent, and the preceding asymptotic normality and covariance consistency conditions hold. If every $V_g$ is positive definite, then $$ W=(C\hat\theta)^\mathsf{T} \left\{C\,\operatorname{blockdiag}\left(\frac{\hat V_1}{n_1},\ldots,\frac{\hat V_G}{n_G}\right)C^\mathsf{T}\right\}^{-1} (C\hat\theta) \xrightarrow{d}\chi^2_{(G-1)q}. $$

Proof. Independence and the groupwise asymptotic normal limits imply that the stacked estimation error has asymptotic block-diagonal covariance. More explicitly, with $n=\sum_g n_g$, $\sqrt n(\hat\theta-\theta)$ converges in distribution to a mean-zero normal vector with covariance $$ D=\operatorname{blockdiag}\left(\frac{V_1}{\lambda_1},\ldots,\frac{V_G}{\lambda_G}\right). $$ Consequently, under $H_0$, $\sqrt n C\hat\theta$ converges to a mean-zero normal vector with covariance $\Omega=CDC^\mathsf{T}$. This matrix is positive definite: $D$ is positive definite, and $C$ has full row rank. Also, $$ nC\,\operatorname{blockdiag}\left(\frac{\hat V_1}{n_1},\ldots,\frac{\hat V_G}{n_G}\right)C^\mathsf{T} \xrightarrow{p}\Omega. $$ Slutsky’s theorem therefore gives the limit of $W$ as $Z^\mathsf{T}\Omega^{-1}Z$, where $Z\sim N_{(G-1)q}(0,\Omega)$. This quadratic form has a $\chi^2_{(G-1)q}$ distribution, proving the result. $\square$

Why the Covariance Has Off-Diagonal Blocks

Contrasts that share a reference group are correlated. In particular, both $\hat\theta_2-\hat\theta_1$ and $\hat\theta_3-\hat\theta_1$ contain the same estimate $\hat\theta_1$. The covariance must retain that shared uncertainty; treating the contrasts as independent generally gives the wrong test.

Result: For the baseline contrasts, the block in the covariance matrix corresponding to contrasts $g$ and $h$, where $g,h\in\{2,\ldots,G\}$, is $$ \operatorname{Cov}(\hat\theta_g-\hat\theta_1,\hat\theta_h-\hat\theta_1) = \begin{cases} V_g/n_g+V_1/n_1,&g=h,\\ V_1/n_1,&g\ne h, \end{cases} $$ to first order.

Derivation. Write $e_g=\hat\theta_g-\theta_g$. Independence across groups gives $\operatorname{Cov}(e_g,e_h)=0$ when $g\ne h$, while $\operatorname{Cov}(e_g)=V_g/n_g$ to first order. For $g\ne h$, $$ \operatorname{Cov}(e_g-e_1,e_h-e_1) =0-0-0+\operatorname{Cov}(e_1) =V_1/n_1. $$ For $g=h$, the same expansion gives $$ \operatorname{Var}(e_g-e_1) =\operatorname{Var}(e_g)+\operatorname{Var}(e_1) =V_g/n_g+V_1/n_1. $$ The formula follows. $\square$

Worked Example: Two Groups and Two Variables

Suppose two independent groups have $p=2$, so $q=3$. Using the order $(11,21,22)$ for $\operatorname{vech}$, suppose the estimated robust scatter matrices are $$ \hat S_1=\begin{pmatrix}4&1\\1&2\end{pmatrix}, \qquad \hat S_2=\begin{pmatrix}5&1.5\\1.5&3\end{pmatrix}. $$ Take $n_1=40$, $n_2=50$, and suppose the estimated group-specific asymptotic covariance matrices for the vectorized estimates are $$ \hat V_1=\operatorname{diag}(4,1,2), \qquad \hat V_2=\operatorname{diag}(9,4,8). $$

The estimated contrast is $$ \hat d=\operatorname{vech}(\hat S_2-\hat S_1)=(1,0.5,1)^\mathsf{T}. $$ For two groups, its estimated covariance is $\hat V_1/n_1+\hat V_2/n_2$, hence $$ \operatorname{diag}(4/40+9/50,\;1/40+4/50,\;2/40+8/50) =\operatorname{diag}(0.28,0.105,0.21). $$ The Wald statistic is therefore $$ W=\frac{1^2}{0.28}+\frac{0.5^2}{0.105}+\frac{1^2}{0.21} =3.5714+2.3810+4.7619 =10.7143. $$ Under the null, the reference distribution is $\chi^2_3$. The statistic is evidence against equality at conventional significance levels (its upper-tail probability is about $0.013$). The calculation uses uncertainty for each group’s robust estimate, not the scatter matrices themselves as though they were known without error.

Worked Example: Three Groups With Unequal Precision

For a one-dimensional scatter functional, $q=1$. Suppose three independent groups have estimated functional values $\hat\theta_1=2$, $\hat\theta_2=3$, and $\hat\theta_3=4$. Let their sample sizes be $40,50,60$, and their estimated asymptotic variances be $\hat V_1=4$, $\hat V_2=9$, and $\hat V_3=16$. The two baseline contrasts are $\hat d=(1,2)^\mathsf{T}$.

The covariance matrix, including the shared uncertainty from group 1, is $$ \hat\Omega= \begin{pmatrix} 4/40+9/50&4/40\\ 4/40&4/40+16/60 \end{pmatrix} = \begin{pmatrix} 0.28&0.10\\ 0.10&0.3667 \end{pmatrix}. $$ Its determinant is $0.28(0.3667)-0.10^2=0.09267$. For a symmetric matrix with entries $a,b,b,c$, the quadratic form for $(1,2)^\mathsf{T}$ is $(c-4b+4a)/(ac-b^2)$. Thus $$ W=\frac{0.3667-4(0.10)+4(0.28)}{0.09267} \approx 11.73. $$ The null reference distribution is $\chi^2_2$, whose upper-tail probability here is about $0.0028$. Ignoring the off-diagonal $0.10$ would discard the dependence induced by using group 1 in both contrasts and would change the test.

What Rejection Means Under Contamination

A robust procedure limits the effect of some unusual observations, but it does not make its target independent of the group distributions. If $F_g$ is a contaminated distribution, the tested quantity is still $S(F_g)$. If groups have different contamination mechanisms, their robust scatter functionals may differ even when the uncontaminated components are identical. Conversely, robust scatter equality does not establish that the full distributions are equal: differences in tails, skewness, or contamination may remain.

Worked Example: Equal Clean Distributions, Different Robust Targets

Consider a univariate robust scatter functional based on the median absolute deviation. For a distribution $F$ with unique median $m_F$, define $$ s(F)=c\,\operatorname{median}_F(|X-m_F|),\qquad S(F)=s(F)^2, $$ where $c=1/\Phi^{-1}(0.75)\approx1.4826$ and $\Phi$ is the standard normal distribution function. This calibration makes $S(F)=1$ when $F=N(0,1)$.

Let group 1 have distribution $N(0,1)$. Let group 2 have distribution $$ F_2=0.95N(0,1)+0.025\delta_{-10}+0.025\delta_{10}, $$ where $\delta_a$ is a point mass at $a$. This mixture is symmetric, so its median is zero. For $0\le r<10$, its probability of lying within distance $r$ of zero is $0.95\{2\Phi(r)-1\}$. The median absolute deviation $r$ therefore solves $$ 0.95\{2\Phi(r)-1\}=0.5, \qquad \Phi(r)=\frac{1+0.5/0.95}{2}\approx0.76316. $$ Thus $r\approx0.716$, $s(F_2)\approx1.4826(0.716)\approx1.062$, and $S(F_2)\approx1.13$, rather than 1. The robust scatter targets differ although both groups share the same clean normal component. A test rejection would indicate evidence that these specified robust functionals differ; it would not, by itself, show that the clean components have different scatter.

This example also illustrates why robust scatter should not be casually called “the covariance.” For group 2, the ordinary variance is $0.95(1)+0.05(100)=5.95$, whereas the robust scatter functional is about $1.13$. These quantities answer different questions under contamination.

Conditions, Diagnostics, and a Common Pitfall

The chi-square calibration is an asymptotic result. It relies on independent groups, fixed $G$ and $p$, adequate sample sizes in every group, asymptotic normality of each estimator, and consistent estimation of its asymptotic covariance. The matrix being inverted must be nonsingular. If a chosen estimator has a singular asymptotic covariance or the hypotheses contain redundant contrasts, the test must instead be formulated on the estimable subspace, with degrees of freedom equal to the rank of the tested covariance; one should not invert a singular matrix as if it were nonsingular.

Common mistake: pooling uncertainty across groups. Using one pooled covariance estimate assumes a shared uncertainty structure that need not hold. Use the group-specific $\hat V_g/n_g$ blocks unless a justified model says otherwise.
Common mistake: treating baseline contrasts as independent. Contrasts that share group 1 have covariance $V_1/n_1$ to first order. Include these off-diagonal blocks in the Wald statistic.
Common mistake: interpreting rejection as a test of clean-population covariance. The null concerns equality of the specified robust scatter functionals at the actual group distributions. A different contamination pattern can change those targets.
Common mistake: using the wrong covariance estimator. The $V_g$ must correspond to the robust estimator and its nuisance-parameter estimation and calibration. A covariance formula for a different estimator does not justify this Wald test.

Key takeaway: Compare the vectorized robust scatter functionals with a Wald statistic built from the full covariance of the group contrasts. The conclusion concerns equality of those functionals for the observed group distributions, not equality of every distributional feature or of an unobserved clean covariance.

Check Your Understanding

Use the definitions and results above to check that you can construct and interpret the test.

For $G$ groups and $p$ variables, how many baseline contrast coordinates are tested?
Why do contrasts against the same reference group have nonzero covariance?
What group-specific quantities enter the estimated covariance of $\hat\theta_g-\hat\theta_1$?
State the conditions under which the Wald statistic has its chi-square limit.
If the test rejects when groups have different contamination patterns, what does rejection establish—and what does it not establish?