Unequal-Covariance Wald Tests for Two Mean Vectors

Why Use a Separate-Covariance Wald Test?

A two-sample comparison concerns the difference between two mean vectors, but the uncertainty in that difference depends on both groups’ covariance matrices. If the groups may have different covariance structures, pooling their sample covariances imposes a restriction that the analysis need not make. A separate-covariance Wald test instead estimates the variance of the mean difference using each group’s covariance estimate.

The statistic is a squared, covariance-adjusted distance from the observed mean difference to the null value. Its construction is direct; its calibration requires care. In particular, the statistic does not generally have the exact Hotelling distribution available for the pooled common-covariance test.

Definition and Construction

Let $X_{ij}$ be the $p$-dimensional observation for individual $j$ in group $i$, for $i=1,2$. Assume the two samples are independent, observations are independent and identically distributed within each group, and group $i$ has mean vector $\mu_i$ and covariance matrix $\Sigma_i$. The two covariance matrices are not required to be equal. Write $n_i$ for the sample size, $\bar X_i$ for the sample mean vector, and $S_i$ for the usual unbiased sample covariance matrix in group $i$.

Definition: To test $H_0:\mu_1-\mu_2=\delta_0$, where $\delta_0$ is a specified $p$-vector, define $$ \widehat V=\frac{S_1}{n_1}+\frac{S_2}{n_2}, \qquad W=(\bar X_1-\bar X_2-\delta_0)^\mathsf{T} \widehat V^{-1} (\bar X_1-\bar X_2-\delta_0). $$ When the null hypothesis is equality of the mean vectors, $\delta_0=0$. The inverse is assumed to exist. A large value of $W$ is evidence against the null hypothesis.

The estimated covariance $\widehat V$ is the estimated covariance of the difference of the two sample means: independence of the groups gives $\operatorname{Cov}(\bar X_1-\bar X_2)=\Sigma_1/n_1+\Sigma_2/n_2$. The quadratic form therefore measures the discrepancy in units of estimated sampling variability, accounting for correlation among coordinates as well as their different scales.

Worked Example: Diagonal Covariance Estimates

Suppose $p=2$, $n_1=20$, $n_2=30$, $\bar X_1-\bar X_2=(1,0.5)^\mathsf{T}$, and the null is $\delta_0=0$. Let $$ S_1=\begin{pmatrix}4&0\\0&1\end{pmatrix}, \qquad S_2=\begin{pmatrix}9&0\\0&4\end{pmatrix}. $$ Calculate $W$, and give its large-sample reference probability under the null.

First form the estimated covariance of the mean difference: $$ \widehat V=\frac{S_1}{20}+\frac{S_2}{30} =\begin{pmatrix}4/20+9/30&0\\0&1/20+4/30\end{pmatrix} =\begin{pmatrix}0.5&0\\0&11/60\end{pmatrix}. $$ Its inverse is $\operatorname{diag}(2,60/11)$. Thus $$ W=1^2(2)+(0.5)^2(60/11) =2+\frac{15}{11} =\frac{37}{11}\approx 3.364. $$ For $p=2$, the large-sample reference distribution is $\chi^2_2$. Since $P(\chi^2_2\geq w)=\exp(-w/2)$, the approximate upper-tail probability is $\exp(-37/22)\approx 0.186$. This is an asymptotic calibration, not an exact finite-sample p-value.

Large-Sample Calibration

For a fixed number $p$ of coordinates, the separate-covariance Wald statistic has a chi-square limit under standard sampling conditions. The result does not require multivariate normal observations, but it does require enough regularity for the sample means to satisfy a central limit theorem and the sample covariance matrices to be consistent.

Theorem: Suppose the two samples are independent, observations are identically distributed within each group, both group covariance matrices $\Sigma_i$ are finite and positive definite, and each group has finite second moments. Let $N=n_1+n_2$, with $n_1/N\to\lambda$ for some $0<\lambda<1$. Under $H_0:\mu_1-\mu_2=\delta_0$, the statistic $W$ satisfies $$ W\xrightarrow{d}\chi^2_p $$ as $N\to\infty$, for fixed $p$.

Proof. Define $D=\bar X_1-\bar X_2-\delta_0$, which under the null has mean zero. By the multivariate central limit theorem applied to the independent samples, $$ \sqrt{N}D\xrightarrow{d}N_p(0,\Omega), \qquad \Omega=\frac{\Sigma_1}{\lambda}+\frac{\Sigma_2}{1-\lambda}. $$ The matrix $\Omega$ is positive definite. The law of large numbers applied to the sample second moments gives $S_i\xrightarrow{p}\Sigma_i$. Since $N/n_1\to1/\lambda$ and $N/n_2\to1/(1-\lambda)$, it follows that $$ N\widehat V=\frac{N}{n_1}S_1+\frac{N}{n_2}S_2\xrightarrow{p}\Omega. $$ Set $Y_N=\Omega^{-1/2}\sqrt{N}D$ and $B_N=\Omega^{-1/2}(N\widehat V)\Omega^{-1/2}$. Then $Y_N\xrightarrow{d}N_p(0,I_p)$, $B_N\xrightarrow{p}I_p$, and $B_N^{-1}\xrightarrow{p}I_p$. Rewriting the statistic gives $W=Y_N^\mathsf{T}B_N^{-1}Y_N$. Slutsky’s theorem and continuity of the quadratic form imply convergence to $Y^\mathsf{T}Y$, where $Y\sim N_p(0,I_p)$. That sum of $p$ independent squared standard normal variables has a $\chi^2_p$ distribution. $\square$

This theorem gives the reference distribution when both group sizes grow with a nonvanishing proportion of the total. It does not guarantee a good approximation with small samples, and it does not make the finite-sample distribution independent of the unknown covariance matrices.

Worked Example: A Nondiagonal Covariance Estimate

Let $p=2$, $n_1=12$, $n_2=18$, $\bar X_1-\bar X_2=(0.5,1)^\mathsf{T}$, and $\delta_0=0$. Suppose $$ S_1=\begin{pmatrix}2&1\\1&2\end{pmatrix}, \qquad S_2=\begin{pmatrix}3&0\\0&1\end{pmatrix}. $$ The off-diagonal entry in $S_1$ means the first and second coordinates contribute jointly to the adjusted distance.

The covariance estimate is $$ \widehat V=\frac{S_1}{12}+\frac{S_2}{18} =\begin{pmatrix}1/3&1/12\\1/12&2/9\end{pmatrix}. $$ Its determinant is $(1/3)(2/9)-(1/12)^2=2/27-1/144=29/432>0$, so it is invertible. The inverse is $$ \widehat V^{-1} =\frac{432}{29}\begin{pmatrix}2/9&-1/12\\-1/12&1/3\end{pmatrix} =\frac{1}{29}\begin{pmatrix}96&-36\\-36&144\end{pmatrix}. $$ Therefore $$ W=\begin{pmatrix}0.5&1\end{pmatrix} \widehat V^{-1} \begin{pmatrix}0.5\\1\end{pmatrix} =\frac{24-36+144}{29} =\frac{132}{29}\approx4.552. $$ Using the large-sample $\chi^2_2$ reference, the upper-tail probability is $\exp(-W/2)\approx0.103$. The negative cross-term is not an error: it follows from the covariance-adjusted geometry of the quadratic form.

Why the Exact Hotelling Law Does Not Generally Apply

Under multivariate normal sampling with a common covariance matrix, the pooled two-sample Hotelling test has an exact finite-sample $F$ calibration. That result relies on the pooled covariance estimate having a particular Wishart distribution and on its independence from the difference of sample means. The separate-covariance Wald statistic uses a different covariance estimate, $S_1/n_1+S_2/n_2$. Under unequal covariance matrices, its terms have different covariance scales, and their sum does not generally have the Wishart form needed for the exact Hotelling result.

Result: Even for independent multivariate normal samples, the separate-covariance statistic $W$ does not generally have an exact Hotelling $F$ distribution under the null. Its finite-sample null distribution can depend on both covariance matrices, whereas its fixed-$p$, large-sample calibration is $\chi^2_p$.

A one-dimensional normal example makes the issue visible. Suppose $p=1$, both group sizes equal $m$, the common null mean is $\mu$, and the group variances are $\sigma_1^2$ and $\sigma_2^2$. Put $\nu=m-1$. Under the null, independent normal-sampling results give $$ \bar X_1-\bar X_2 =\sqrt{\frac{\sigma_1^2+\sigma_2^2}{m}}Z, \qquad \frac{\nu S_i}{\sigma_i^2}=U_i, $$ where $Z\sim N(0,1)$, $U_1,U_2\sim\chi^2_\nu$, and these three variables are independent. Substitution into the separate-variance statistic yields $$ W=\frac{\nu(\sigma_1^2+\sigma_2^2)Z^2} {\sigma_1^2U_1+\sigma_2^2U_2}. $$ When $\sigma_1^2=\sigma_2^2$, $U_1+U_2\sim\chi^2_{2\nu}$, and this reduces to the familiar $F_{1,2\nu}$ form. With unequal variances, the denominator is a weighted sum of independent chi-square variables, not the same scaled chi-square denominator. The expression also involves the relative sizes of the two variances. Thus the usual exact pooled Hotelling calibration cannot simply be assigned to $W$.

Worked Example: Unequal Variances in One Dimension

Continue with $p=1$ and equal group sizes, now taking $m=5$, so $\nu=4$. Let $\sigma_1^2=1$ and $\sigma_2^2=4$. Under the null, the formula above becomes $$ W=\frac{4(1+4)Z^2}{U_1+4U_2} =\frac{20Z^2}{U_1+4U_2}, \qquad U_1,U_2\sim\chi^2_4. $$ The denominator is not a common scale times $U_1+U_2$. In contrast, if both variances were 1, it would be $U_1+U_2\sim\chi^2_8$, and $W=8Z^2/\chi^2_8\sim F_{1,8}$. The unequal-variance statistic therefore cannot be treated as that exact $F_{1,8}$ statistic. This calculation also shows why knowing only the sample sizes and the dimension does not, in general, specify its finite-sample null distribution.

Using and Interpreting the Test

In practice, calculate $W$ from the separate covariance estimates and use the $\chi^2_p$ approximation only when its large-sample justification is reasonable. The statistic summarizes evidence against the specified vector null hypothesis as a single nonnegative number; it does not identify which coordinates account for the discrepancy. Interpretation of individual coordinates requires additional inferential work and should not be inferred from the quadratic form alone.

The distinction between the separate-covariance Wald test and the pooled test is substantive, not merely computational. Pooling estimates a common covariance structure; using $S_1/n_1+S_2/n_2$ allows the two groups to have different covariance matrices. If the common-covariance assumption is credible, the pooled procedure supplies a different test with its own exact calibration under multivariate normality. If that assumption is not credible, applying its exact $F$ calibration to the separate-covariance statistic is not justified.

Common Mistakes

Using the pooled exact $F$ calibration automatically. The exact pooled result does not transfer to a quadratic form built from separate covariance estimates. Use the appropriate large-sample calibration, or an appropriate finite-sample method.
Leaving out a sample-size divisor. The covariance estimate for the mean difference is $S_1/n_1+S_2/n_2$, not $S_1+S_2$. The sample sizes determine the uncertainty of each sample mean.
Ignoring covariance between coordinates. If either $S_i$ has nonzero off-diagonal entries, those entries affect the inverse matrix and hence the Wald distance.
Treating the chi-square result as exact. The $\chi^2_p$ statement is a large-sample limit under the conditions in the theorem, not an exact small-sample law.

Key takeaway: For two independent mean vectors with possibly unequal covariance matrices, estimate the covariance of their difference as $S_1/n_1+S_2/n_2$ and form the corresponding Wald quadratic. Its standard justification is asymptotic $\chi^2_p$ calibration; an exact pooled Hotelling calibration does not generally apply.

Check Your Understanding

Use the definitions and results above to answer the following questions.

Why does the covariance estimate of the difference of two independent sample means contain $S_1/n_1$ and $S_2/n_2$?
State the sample-size condition and the limiting reference distribution in the asymptotic theorem.
For $p=1$, equal group sizes, and normal sampling, what feature of the denominator changes when the group variances are unequal?
Why is the exact pooled Hotelling $F$ calibration not automatically valid for the separate-covariance statistic?
What role do the off-diagonal entries of the covariance estimate play in a multivariate Wald statistic?