Assessing the Common-Covariance Assumption

Why Assess a Common Covariance?

The pooled two-sample Hotelling test uses one covariance estimate for both groups. Its exact calibration requires that the groups share a covariance matrix, in addition to the relevant sampling and distributional conditions. In practice, the covariance matrices are unknown, so the assumption needs to be assessed rather than simply treated as a fact. The goal is not to replace a formal test of covariance equality with a plot; it is to use complementary evidence to judge whether pooling is plausible and whether the pooled inference may be sensitive to heterogeneity.

Covariance heterogeneity can involve different marginal variances, different correlations, or both. Two groups can even have the same covariance eigenvalues and determinant while having different covariance matrices because their principal directions differ. Comparing only one summary, such as total generalized variance, can therefore miss an important discrepancy.

Group-Specific Covariance and Scatter

Let $X_{gj}$ be the $p$-dimensional observation for individual $j$ in group $g$, where $g=1,2$, and let the independent observations in group $g$ have mean vector $\mu_g$ and covariance matrix $\Sigma_g$. Write $n_g$ for the group sample size, $\bar X_g$ for its sample mean, and $N=n_1+n_2$. The common-covariance assumption is $\Sigma_1=\Sigma_2$. Its sample counterpart compares the separate within-group covariance estimates.

Definition: The group-specific sample covariance matrix and scatter matrix are $$ S_g=\frac{1}{n_g-1}\sum_{j=1}^{n_g}(X_{gj}-\bar X_g)(X_{gj}-\bar X_g)^\top,\qquad W_g=(n_g-1)S_g. $$ When $n_g\geq 2$, $S_g$ estimates $\Sigma_g$ without bias. The pooled covariance estimate is $S_p=(W_1+W_2)/(N-2)$; it combines the two groups’ scatter matrices using their degrees of freedom.

The matrices $S_1$ and $S_2$ are the main diagnostic objects. Inspect their diagonal entries, which are sample variances, and their off-diagonal entries, which are sample covariances. It is usually useful to inspect the corresponding correlations too: differences in covariance can reflect different marginal scales, different association patterns, or both. A correlation comparison does not establish covariance equality, since equal correlations do not imply equal variances.

Worked Example: Equal Determinants, Different Covariance Directions

Suppose the observed group covariance estimates are $$ S_1=\begin{pmatrix}2&1\\1&2\end{pmatrix}, \qquad S_2=\begin{pmatrix}2&-1\\-1&2\end{pmatrix}. $$ Both matrices are positive definite because each has eigenvalues $3$ and $1$, and each has determinant $3$. A comparison based only on determinants or on the separate eigenvalue lists would suggest no difference.

The off-diagonal entries, however, have opposite signs. Group 1 has positive sample covariance, while group 2 has negative sample covariance. Their principal directions differ, so the covariance matrices are not equal. To compare their relative scales across directions, compute the eigenvalues of $S_1^{-1}S_2$. Since $$ S_1^{-1}S_2=\frac{1}{3}\begin{pmatrix}5&-4\\-4&5\end{pmatrix}, $$ its eigenvalues are $3$ and $1/3$: the sum is $10/3$ and the product is $1$, so these are the roots of $\lambda^2-(10/3)\lambda+1=0$. The discrepancy is substantial despite equal determinants and equal separate eigenvalue lists.

Residual Plots as a Diagnostic

For each group, examine the centered residual vectors $R_{gj}=X_{gj}-\bar X_g$. Pairwise scatterplots of their coordinates can reveal differences in spread, orientation, curvature, or unusual observations. Plotting the groups separately, or distinguishing them clearly in a common plot, helps avoid mistaking one group’s structure for the other’s. Residual plots are exploratory: sample variation can make genuinely equal populations look different, and a small sample can conceal real differences.

A practical inspection begins with marginal residual plots and pairwise scatterplots, then checks whether the patterns look reasonably consistent between groups. Look for a group with a much wider spread, a changed direction of association, or a few points that dominate the sample covariance. If a residual plot shows a striking outlier, check the observation and the data collection before interpreting the covariance comparison. A single extreme point can have a large effect on a sample covariance matrix.

Plots depend on the coordinate system. Changing measurement units can make one axis appear much wider than another, and a rotation of the variables changes the appearance of pairwise plots. This is why visual inspection, covariance summaries, and eigenvalue comparisons complement one another rather than serving as interchangeable checks.

Worked Example: Reading a Residual Pattern

Consider four centered residual vectors in group 1: $(2,0),(-2,0),(0,1),(0,-1)$. Their coordinate sums are zero, so they are already centered. The first coordinate has sum of squares $2^2+(-2)^2=8$, the second has sum of squares $1^2+(-1)^2=2$, and the cross-product sum is zero. With $n_1-1=3$, $$ S_1=\begin{pmatrix}8/3&0\\0&2/3\end{pmatrix}. $$

In group 2, take residuals $(1,0),(-1,0),(0,2),(0,-2)$. The coordinate sums of squares are $2$ and $8$, and the cross-product sum is again zero. Thus $$ S_2=\begin{pmatrix}2/3&0\\0&8/3\end{pmatrix}. $$

Separate residual plots would show that group 1 varies more along the first coordinate, whereas group 2 varies more along the second. The generalized eigenvalues are the diagonal ratios $ (2/3)/(8/3)=1/4$ and $(8/3)/(2/3)=4$. Both the plots and the matrices indicate a reversal in the relative spreads; neither alone proves that the population covariance matrices differ.

Comparing Covariance Matrices with Generalized Eigenvalues

When $S_1$ and $S_2$ are positive definite, define the generalized eigenvalues as the ordinary eigenvalues of the symmetric matrix $S_1^{-1/2}S_2S_1^{-1/2}$, where $S_1^{-1/2}$ is the symmetric inverse square root of $S_1$. They are positive. Equivalently, they are the roots of $\det(S_2-\lambda S_1)=0$, or the eigenvalues of $S_1^{-1}S_2$. This comparison measures the relative scale of $S_2$ to $S_1$ in directions after accounting for the scale and orientation in $S_1$.

If the matrices are equal, every generalized eigenvalue is $1$. Values well above and below $1$ indicate directions in which the second group has greater and smaller relative variance, respectively. A collection near $1$ is compatible with similar covariance structure, but there is no universal cutoff that makes it a formal decision: sample sizes, dimension, and sampling variability matter. If a covariance estimate is singular, for example because the sample size is too small relative to $p$, this positive-definite comparison cannot be applied as stated.

Worked Example: Interpreting Relative Eigenvalues

Suppose $S_1=\operatorname{diag}(1,4)$ and $S_2=\operatorname{diag}(2,1)$. Both are positive definite, and $$ S_1^{-1/2}S_2S_1^{-1/2} =\begin{pmatrix}1&0\\0&1/2\end{pmatrix} \begin{pmatrix}2&0\\0&1\end{pmatrix} \begin{pmatrix}1&0\\0&1/2\end{pmatrix} =\begin{pmatrix}2&0\\0&1/4\end{pmatrix}. $$

The generalized eigenvalues are $2$ and $1/4$. In the first coordinate, group 2 has twice the estimated variance of group 1; in the second, it has one quarter as much. The result is not a single overall “larger covariance” conclusion: the relative spread changes with direction. In this diagonal example the coordinates make the directions explicit; for non-diagonal matrices, the generalized eigenvectors describe the corresponding directions after the first covariance has been used to standardize them.

What Heterogeneity Means for Pooled Inference

The pooled covariance estimate has a clear interpretation under the common-covariance assumption. When the true matrices differ, it still averages the observed within-group scatter, but it no longer estimates a shared population covariance. The following expectation makes that distinction precise.

Result: If each group consists of independent observations with a common mean $\mu_g$ and covariance $\Sigma_g$, and $n_g\geq2$, then $$ E[S_p]=\frac{(n_1-1)\Sigma_1+(n_2-1)\Sigma_2}{N-2}. $$ In particular, this is a degrees-of-freedom-weighted average when the group covariances differ, not either group’s covariance in general.

Derivation. For one group, expand the scatter as $\sum_j X_{gj}X_{gj}^{\top}-n_g\bar X_g\bar X_g^\top$. Independence and the common within-group mean give $E[\sum_j X_{gj}X_{gj}^{\top}]=n_g(\Sigma_g+\mu_g\mu_g^\top)$. Also, $E[\bar X_g\bar X_g^\top]=\Sigma_g/n_g+\mu_g\mu_g^\top$. Subtracting $n_g$ times the second expression from the first yields $E[W_g]=(n_g-1)\Sigma_g$. Taking expectations in $S_p=(W_1+W_2)/(N-2)$ gives the result. $\square$

There is a second issue: the covariance of the observed difference in sample means is determined by both group covariances, with weights set by the sample sizes.

Result: For independent groups with finite covariance matrices $\Sigma_1$ and $\Sigma_2$, $$ \operatorname{Cov}(\bar X_1-\bar X_2)=\frac{\Sigma_1}{n_1}+\frac{\Sigma_2}{n_2}. $$ The pooled Hotelling procedure instead uses $S_p(1/n_1+1/n_2)$ as its estimated covariance matrix for the difference.

Derivation. Independence of the groups implies $\operatorname{Cov}(\bar X_1,\bar X_2)=0$. Within group $g$, independence gives $\operatorname{Cov}(\bar X_g)=\Sigma_g/n_g$, since the covariance of a sum of $n_g$ independent observations is $n_g\Sigma_g$, and averaging divides this by $n_g^2$. Applying these facts to the difference yields the displayed formula. $\square$

These expressions show why group sizes matter. If the covariances differ and sample sizes are unequal, the expectation of the pooled covariance multiplied by $1/n_1+1/n_2$ will generally not equal the true covariance of the mean difference. Even when sample sizes are equal, so that these two covariance expressions agree in expectation, the usual exact pooled $F$ calibration does not follow under unequal covariances. The exact pooled two-sample Hotelling calibration requires independent multivariate normal samples with a common positive-definite covariance matrix (and adequate sample sizes for the statistic and its reference distribution). Heterogeneity undermines the distributional justification for that calibration, not just the interpretation of a pooled estimate.

Worked Example: Unequal Sizes and the Pooled Variance Estimate

Suppose $p=2$, $n_1=20$, $n_2=5$, and the group covariance estimates are $S_1=\operatorname{diag}(1,9)$ and $S_2=\operatorname{diag}(4,1)$. The pooled estimate is $$ S_p=\frac{19S_1+4S_2}{23} =\begin{pmatrix}35/23&0\\0&175/23\end{pmatrix} \approx\begin{pmatrix}1.522&0\\0&7.609\end{pmatrix}. $$

For illustration, treat these observed matrices as the corresponding population covariances. The covariance of the difference in sample means is then $$ \frac{\Sigma_1}{20}+\frac{\Sigma_2}{5} =\begin{pmatrix}1/20+4/5&0\\0&9/20+1/5\end{pmatrix} =\begin{pmatrix}0.85&0\\0&0.65\end{pmatrix}. $$ The pooled procedure’s covariance estimate has expectation, under these matrices, $$ \left(\frac{1}{20}+\frac{1}{5}\right)S_p =0.25S_p \approx\begin{pmatrix}0.380&0\\0&1.902\end{pmatrix}. $$ These matrices differ in both coordinates. Here the smaller group has the larger first-coordinate variance, so its covariance receives substantial weight in the true mean-difference variance, while the pooled estimate weights scatter by degrees of freedom. This illustrates why covariance heterogeneity combined with unequal group sizes can materially affect pooled inference.

Common Mistake and Practical Interpretation

A common mistake is to conclude that pooling is justified because the determinants, traces, or separate eigenvalue lists of $S_1$ and $S_2$ are similar. Those summaries do not generally encode the matrices’ orientations. In the worked example with opposite correlations, the determinants and eigenvalue lists agreed exactly, yet the covariance matrices differed. The correction is to inspect the full matrices, examine group-specific residual plots, and, when the estimates are positive definite, compare generalized eigenvalues as a directional diagnostic.

These checks are evidence, not a guarantee. A formal covariance test can provide a complementary assessment, but its calibration and sensitivity to departures from normality also matter. If the diagnostics suggest meaningful heterogeneity, do not treat the standard pooled test’s exact calibration as automatically valid. Unequal-covariance approaches are the relevant alternative; choosing and applying one is a separate inferential task. The present assessment helps identify when that issue deserves attention.

Check Your Understanding

Use the definitions and reasoning in this tutorial to answer the following.

What information is in the off-diagonal entries of a group covariance matrix, and why can comparing only marginal variances miss heterogeneity?
Two positive-definite covariance estimates have equal determinants and equal separate eigenvalue lists. Does that establish equality of the matrices? Explain.
Why does a generalized eigenvalue substantially above $1$ indicate a direction of greater relative variance for the second group?
Under what condition does the pooled covariance estimate have the interpretation of an estimate of one shared covariance matrix?
Why can equal group sizes make the pooled estimate’s covariance for the mean difference correct in expectation under heterogeneity without restoring the exact pooled $F$ calibration?