Pairwise Hotelling Comparisons Across Several Groups

Why Pairwise Comparisons Need Their Own Calibration

With several groups, an overall test can address whether the mean vectors are all equal, but it does not identify which pairs differ. A pairwise comparison focuses on one difference, such as $\mu_i-\mu_j$, while estimating covariance from the within-group variation across all groups. Under multivariate normal sampling and a common covariance matrix, each such comparison has an exact Hotelling $T^2$ calibration.

The word each matters. The test and confidence ellipsoid below give exact inference for one specified pair at a time. Although they can be calculated for every pair, their individual coverage statements do not automatically provide a joint guarantee for the entire collection.

Definition: Suppose there are $k$ independent groups, each with $p$-dimensional observations. In group $i$, let $n_i$ be the sample size, $\bar X_i$ the sample mean vector, and $S_i$ the sample covariance matrix. Write $N=\sum_{i=1}^k n_i$ and $\nu=N-k$. The pooled within-group covariance estimate is $$ S_p=\frac{\sum_{i=1}^k (n_i-1)S_i}{\nu}. $$ For groups $i$ and $j$, the estimated mean difference is $d_{ij}=\bar X_i-\bar X_j$, and its estimated covariance is $(1/n_i+1/n_j)S_p$.

The model assumptions are that observations are independent across groups and within groups, $X_{ir}\sim N_p(\mu_i,\Sigma)$ for $r=1,\ldots,n_i$, and the common covariance matrix $\Sigma$ is positive definite. Assume $n_i\geq 2$ and $\nu\geq p$, so the pooled estimate is defined and is nonsingular with probability one. The common-covariance assumption is essential to this exact pooled calibration; if group covariances differ, this test is not the stated exact procedure.

The Exact Pairwise Hotelling Test

For a specified pair, the null hypothesis is $H_0:\mu_i-\mu_j=0$. The natural squared distance measures the estimated difference relative to its estimated covariance:

T_{ij}^{\,2} = d_{ij}^{\mathsf T} \left[\left(\frac{1}{n_i}+\frac{1}{n_j}\right)S_p\right]^{-1} d_{ij} = \frac{n_i n_j}{n_i+n_j}\, d_{ij}^{\mathsf T}S_p^{-1}d_{ij}.

The second expression follows because $(1/n_i+1/n_j)^{-1}=n_i n_j/(n_i+n_j)$. Thus, a large $T_{ij}^{\,2}$ means the observed difference is large relative to its estimated within-group variability, accounting for correlation among the $p$ outcomes.

Theorem: Under the common-covariance multivariate normal model and $H_0:\mu_i=\mu_j$, for any specified pair $i,j$, $$ \frac{\nu-p+1}{p\nu}\,T_{ij}^{\,2} \sim F_{p,\,\nu-p+1}. $$ At level $\alpha$, reject this pairwise null hypothesis when $T_{ij}^{\,2}>\frac{p\nu}{\nu-p+1}F_{p,\,\nu-p+1;1-\alpha}$, where $F_{p,\,\nu-p+1;1-\alpha}$ is the $1-\alpha$ quantile of the indicated $F$ distribution.

The degrees of freedom use all the within-group residual information: $\nu=N-k$, not just the residual degrees of freedom in the two groups being compared. The reason is that $S_p$ uses the within-group scatter from every group. The result is an exact, not approximate, calibration under the stated model.

Why the Pairwise Calibration Is Exact

For each group, centering removes one degree of freedom from its $n_i$ observations. A useful way to see what remains is to apply an orthogonal transformation to the $n_i$ observation indices, with its first row proportional to $(1,\ldots,1)$. The first transformed vector is $\sqrt{n_i}\bar X_i$; the other $n_i-1$ transformed vectors are residual coordinates.

Result: Under the model assumptions, $\bar X_i$ is independent of the within-group residual scatter, and the pooled scatter $\nu S_p$ has a Wishart distribution with $\nu$ degrees of freedom and scale matrix $\Sigma$. It is also independent of all group sample means.

To justify this result, the orthogonal transformation preserves the joint multivariate normal distribution within each group. Its transformed coordinates are jointly normal, and their cross-covariances are zero because distinct rows of the transformation are orthogonal. Jointly normal vectors with zero cross-covariance are independent. The first coordinate therefore gives the sample mean independently of the remaining residual coordinates. Across groups, independence is part of the sampling model. Pooling the residual coordinates gives $\sum_i(n_i-1)=\nu$ independent $N_p(0,\Sigma)$ vectors, whose sum of outer products is $\nu S_p$. This is the defining Wishart construction.

For the chosen pair, independence of group samples gives $$ d_{ij}\sim N_p\left(\mu_i-\mu_j,\left(\frac{1}{n_i}+\frac{1}{n_j}\right)\Sigma\right), $$ independently of $S_p$. Under the null, the difference has mean zero. The exact two-sample Hotelling $T^2$-to-$F$ result, applied to this normal difference and the independent pooled Wishart estimate, gives the stated $F_{p,\nu-p+1}$ calibration. The other groups affect the test through $S_p$ and $\nu$, not through the pair's mean difference. $\square$

Worked Example: Testing One Pair With Three Groups

Suppose three groups have sample sizes $5,6,7$, and the pooled within-group covariance estimate for two outcomes is $$ S_p=\begin{pmatrix}4&0\\0&1\end{pmatrix}. $$ For groups 1 and 2, suppose $d_{12}=(2,1)^{\mathsf T}$. Calculate the pairwise statistic and its exact null calibration.

Here $N=18$, $k=3$, and $\nu=N-k=15$. The pair's sample-size multiplier is $$ \frac{n_1n_2}{n_1+n_2}=\frac{5\cdot6}{5+6}=\frac{30}{11}. $$ Also $S_p^{-1}=\begin{pmatrix}1/4&0\\0&1\end{pmatrix}$, so $$ d_{12}^{\mathsf T}S_p^{-1}d_{12} = (2,1)\begin{pmatrix}1/4&0\\0&1\end{pmatrix} \begin{pmatrix}2\\1\end{pmatrix} =1+1=2. $$ Therefore $T_{12}^{\,2}=(30/11)(2)=60/11$, approximately $5.455$. With $p=2$, the null-calibrated statistic is $$ \frac{15-2+1}{2(15)}\frac{60}{11} =\frac{14}{30}\frac{60}{11} =\frac{28}{11} \approx 2.545, $$ which is compared with an $F_{2,14}$ distribution. The degrees of freedom come from all three groups, not only groups 1 and 2.

Individual Confidence Ellipsoids

The same statistic can be inverted to obtain a confidence region for the unknown difference $\delta_{ij}=\mu_i-\mu_j$. Replace the observed difference in the quadratic form by its discrepancy from a candidate value $\delta$. The resulting region is an ellipsoid centered at the observed difference.

Theorem: Under the common-covariance multivariate normal model, an exact $100(1-\alpha)\%$ confidence ellipsoid for $\mu_i-\mu_j$, for one specified pair, is $$ \left\{\delta\in\mathbb R^p: (d_{ij}-\delta)^{\mathsf T} \left[\left(\frac{1}{n_i}+\frac{1}{n_j}\right)S_p\right]^{-1} (d_{ij}-\delta) \leq \frac{p\nu}{\nu-p+1}F_{p,\,\nu-p+1;1-\alpha} \right\}. $$

For the true value $\delta_{ij}$, the quadratic form on the left has the same exact $F$ calibration as the pairwise test statistic. Thus its probability of being no larger than the displayed cutoff is $1-\alpha$. The event that the true difference lies in the ellipsoid is exactly that inequality, proving the coverage statement. $\square$

The ellipsoid's orientation and shape reflect the estimated covariance: directions with greater estimated variability have greater extent. It is a region for the whole mean-difference vector, not a collection of separate one-coordinate intervals.

Worked Example: Writing a Pairwise Confidence Ellipsoid

Suppose $k=3$ groups have sample sizes $8,10,9$, so $N=27$ and $\nu=24$. For groups 1 and 2, let $d_{12}=(1,2)^{\mathsf T}$ and $$ S_p=\begin{pmatrix}4&1\\1&2\end{pmatrix}. $$ Write the 95% confidence ellipsoid explicitly.

The covariance multiplier for the difference is $1/8+1/10=9/40$, so its estimated covariance is $(9/40)S_p$. The determinant of $S_p$ is $4(2)-1(1)=7$, hence $$ S_p^{-1}=\frac{1}{7}\begin{pmatrix}2&-1\\-1&4\end{pmatrix}, \qquad \left[\frac{9}{40}S_p\right]^{-1} =\frac{40}{63}\begin{pmatrix}2&-1\\-1&4\end{pmatrix}. $$ For $p=2$, the cutoff is $\{2(24)/(24-2+1)\}F_{2,23;0.95}=(48/23)F_{2,23;0.95}$. Therefore the region consists of all $\delta=(\delta_1,\delta_2)^{\mathsf T}$ satisfying $$ \frac{40}{63} \left\{2(1-\delta_1)^2-2(1-\delta_1)(2-\delta_2)+4(2-\delta_2)^2\right\} \leq \frac{48}{23}F_{2,23;0.95}. $$ The center is the estimated difference $(1,2)^{\mathsf T}$; the off-diagonal entries in $S_p$ determine the ellipse's orientation.

Calculating Every Pair and Interpreting the Family

There are $k(k-1)/2$ distinct unordered pairs. Each pair uses its own difference and sample-size multiplier, but all use the same $S_p$, $\nu$, and null $F$ degrees of freedom. The estimates are related: for three groups, $d_{12}+d_{23}=d_{13}$. Consequently, the comparisons are not generally independent, and listing individual $p$-values or confidence ellipsoids should not be mistaken for a simultaneous analysis.

Worked Example: Computing Three Pairwise Statistics

Consider three groups with $n_1=n_2=n_3=4$, $p=2$, $S_p=I_2$, and sample means $\bar X_1=(0,0)^{\mathsf T}$, $\bar X_2=(2,0)^{\mathsf T}$, and $\bar X_3=(0,1)^{\mathsf T}$. Here $N=12$, $k=3$, and $\nu=9$. The multiplier for every pair is $4\cdot4/(4+4)=2$.

For pair 1 and 2, $d_{12}=(-2,0)^{\mathsf T}$, so $T_{12}^{\,2}=2\{(-2)^2+0^2\}=8$. For pair 1 and 3, $d_{13}=(0,-1)^{\mathsf T}$, so $T_{13}^{\,2}=2\{0^2+(-1)^2\}=2$. For pair 2 and 3, $d_{23}=(2,-1)^{\mathsf T}$, so $T_{23}^{\,2}=2\{2^2+(-1)^2\}=10$.

Since $p=2$ and $\nu=9$, each null-calibrated $F$ statistic is $8/(2\cdot9)$ times its $T^2$: the three values are $32/9$, $8/9$, and $40/9$, each compared with $F_{2,8}$. These are individual pairwise calculations. The differences also satisfy $d_{12}+d_{23}=d_{13}$, illustrating why the three comparisons cannot be treated as unrelated experiments.

Common Pitfall: Individual Is Not Simultaneous

A common error is to compute every pairwise ellipsoid at confidence level $1-\alpha$ and then claim that all of them contain their true differences with probability $1-\alpha$. The derivation proves $1-\alpha$ coverage for one specified pair. It does not prove that the entire collection has that coverage: several individual coverage events can fail together or separately, and their dependence does not make their joint probability equal to any one marginal probability.

A second error is to pool only the two groups in each comparison while still using the degrees of freedom from all $k$ groups. The covariance estimator and its degrees of freedom must match. In the procedure here, every pair uses the all-group $S_p$ and $\nu=N-k$; changing the covariance estimate changes the calibration argument.

The main practical value of this method is that it translates a multivariate pairwise difference into an interpretable covariance-adjusted distance and an exact finite-sample reference distribution. Before interpreting a collection of results, state whether the conclusions are individual or intended to cover the full family.

Check Your Understanding

Answer each question using the assumptions and notation stated above.

For $k$ groups with total sample size $N$, what residual degrees of freedom enter the pooled pairwise test, and why?
How does the pairwise statistic change when the estimated covariance of the mean difference is written directly as $(1/n_i+1/n_j)S_p$?
What $F$ degrees of freedom calibrate a pairwise $T^2$ statistic when there are $p$ outcomes and $\nu$ pooled residual degrees of freedom?
Why does an individual $100(1-\alpha)\%$ ellipsoid not establish $100(1-\alpha)\%$ joint coverage for all pairs?
If the covariance matrices differ among groups, which assumption needed for the exact pooled pairwise calibration fails?