High-Dimensional Mean Testing · Tutorial 820 of 975

Two-Sample Maximum Tests for Sparse Differences

Learn how to standardize two-sample mean differences when group variances may differ, and how to calibrate their maximum under fixed- and growing-dimension null models.

Advanced 12 min read

What You'll Learn

Define a two-sample maximum-coordinate statistic for testing equality of mean vectors.
Derive its pooled standard error under equal covariance matrices.
Construct coordinatewise Welch standard errors when the covariance matrices differ.
Establish the fixed-dimension null limit and its limiting correlation matrix.
Derive extreme-value calibration for independent Gaussian coordinates.
Recognize when marginal studentization does not justify independent-coordinate calibration.

Why Standardize a Two-Sample Maximum?

A maximum-coordinate test targets alternatives in which only a small number of coordinates shift. For two independent groups, however, a raw difference in sample means is hard to interpret: its variability depends both on the group sample sizes and on the variance of that particular coordinate. A maximum of unstandardized differences can therefore select a noisy coordinate rather than a coordinate with strong evidence of a mean shift.

The remedy is to divide each coordinate difference by an estimate of its own standard error, then take the largest absolute value. The key choice is how to estimate that standard error. If the groups share a covariance matrix, pooling their coordinate variances is appropriate. If their covariance matrices differ, the two variance contributions must remain separate.

Definition: Let $X_1,\ldots,X_{n_1}$ and $Y_1,\ldots,Y_{n_2}$ be independent samples of $p$-dimensional vectors, independent within each group, with mean vectors $\mu_1,\mu_2$ and covariance matrices $\Sigma_1,\Sigma_2$. Write $\bar X_j,\bar Y_j$ for the sample means of coordinate $j$, and $S_{1,jj},S_{2,jj}$ for the corresponding sample variances. The two-sample studentized maximum statistic is $$ T_{\max}=\max_{1\leq j\leq p}\left|\frac{\bar X_j-\bar Y_j}{\sqrt{S_{1,jj}/n_1+S_{2,jj}/n_2}}\right|. $$ It tests $H_0:\mu_1=\mu_2$ against alternatives in which at least one coordinate mean differs. The denominator is the Welch standard error, which does not require equal covariance matrices.

Equal and Unequal Covariance Scaling

For a fixed coordinate, independence between the groups gives $\operatorname{Var}(\bar X_j-\bar Y_j)=\Sigma_{1,jj}/n_1+\Sigma_{2,jj}/n_2$. This variance identity motivates the Welch denominator. If $\Sigma_1=\Sigma_2=\Sigma$, the common coordinate variance can instead be estimated by pooling the within-group sample variances: $$ S_{p,jj}=\frac{(n_1-1)S_{1,jj}+(n_2-1)S_{2,jj}}{n_1+n_2-2}, \qquad \widehat{\operatorname{se}}_{j,\mathrm{pool}}=\sqrt{S_{p,jj}\left(\frac{1}{n_1}+\frac{1}{n_2}\right)}. $$

This pooled expression estimates the same standard error under equal covariance matrices; it does not assume that different coordinates are uncorrelated. In contrast, when the groups have unequal covariance matrices, the appropriate coordinatewise plug-in variance is $S_{1,jj}/n_1+S_{2,jj}/n_2$. The latter retains separate estimates of each group’s variability.

Worked Example: Coordinatewise Welch Standard Errors

Suppose $n_1=10$ and $n_2=15$. For three coordinates, let the group sample variances be equal between groups within each coordinate, with values $0.44a_j^2$, where $a_1=0.50$, $a_2=1.20$, and $a_3=0.80$. Find the Welch standard errors.

For any coordinate $j$, the estimated variance of the difference is $$ \frac{0.44a_j^2}{10}+\frac{0.44a_j^2}{15} =0.44a_j^2\left(\frac{1}{6}\right) =\frac{11a_j^2}{150}. $$ Thus the standard error is $a_j\sqrt{11/150}$. Since $\sqrt{11/150}=0.270801280\ldots$, the standard errors, rounded to six decimal places, are $0.135401$, $0.324962$, and $0.216641$, respectively.

For example, if the observed differences are $0.30,0.60,-0.20$, their studentized values are approximately $2.216$, $1.846$, and $-0.923$. The maximum absolute statistic is therefore approximately $2.216$. The units and coordinate-specific scales have been removed, so this statistic can be compared with a null calibration for the maximum rather than with a cutoff for one isolated coordinate.

Worked Example: Equal-Covariance Pooled Scaling

Suppose $n_1=12,n_2=10$, equal covariance matrices are assumed, and the three observed mean differences are $1,2,-1$. The pooled sample variances for these coordinates are $0.5,1,2$. Compute the pooled studentized values.

Here $1/n_1+1/n_2=1/12+1/10=11/60$. The three estimated standard errors are therefore $$ \sqrt{0.5(11/60)}=\sqrt{11/120}\approx0.3028,\quad \sqrt{1(11/60)}\approx0.4282,\quad \sqrt{2(11/60)}=\sqrt{11/30}\approx0.6055. $$ Dividing the differences by these standard errors gives approximately $3.303$, $4.671$, and $-1.651$. Hence $T_{\max}\approx4.671$. This calculation produces the statistic, not its cutoff: a maximum over three coordinates must be calibrated as a maximum, not as a single $t$-statistic.

Fixed-Dimension Null Calibration

The next result specifies the joint null limit when $p$ is fixed. Its assumptions ensure a central limit theorem for the group sample means as well as consistency of the sample variances.

Theorem: Suppose $p$ is fixed; observations are independent between groups and independent and identically distributed within each group; both groups have finite coordinate variances; and $n_1/(n_1+n_2)\to\lambda\in(0,1)$ as $n_1,n_2\to\infty$. Under $H_0:\mu_1=\mu_2$, suppose $\Sigma_{1,jj}/\lambda+\Sigma_{2,jj}/(1-\lambda)>0$ for every coordinate. Define $$ A=\frac{\Sigma_1}{\lambda}+\frac{\Sigma_2}{1-\lambda}, \qquad R_{\infty,jk}=\frac{A_{jk}}{\sqrt{A_{jj}A_{kk}}}. $$ Then the vector of Welch-studentized coordinate differences converges in distribution to $N(0,R_\infty)$. Consequently, $T_{\max}$ converges in distribution to $\max_{1\leq j\leq p}|Z_j|$, where $Z\sim N(0,R_\infty)$.

Proof. Set $N=n_1+n_2$. Under the null, the mean difference has mean zero and covariance $\Sigma_1/n_1+\Sigma_2/n_2$. The multivariate central limit theorem applied to the independent group sample means gives $$ \sqrt{N}(\bar X-\bar Y)\xrightarrow{d}N(0,A), $$ because $N/n_1\to1/\lambda$ and $N/n_2\to1/(1-\lambda)$. The finite coordinate variances imply that each within-group sample variance is consistent for its population variance. Hence $N(S_{1,jj}/n_1+S_{2,jj}/n_2)\xrightarrow{p}A_{jj}$ for every $j$. Since $p$ is fixed and each $A_{jj}>0$, applying Slutsky’s theorem coordinatewise to the joint vector gives the stated standardized normal limit, with correlations $R_{\infty,jk}$. The map taking a vector to its maximum absolute coordinate is continuous, so the continuous mapping theorem gives the limit for $T_{\max}$. $\square$

The limiting correlation is essential: coordinates need not be independent, even when the two groups have equal covariance matrices. For fixed $p$, a level-$\alpha$ asymptotic test rejects when $T_{\max}>c_\alpha$, where $c_\alpha$ satisfies $P(\max_j|Z_j|\leq c_\alpha)=1-\alpha$ for $Z\sim N(0,R_\infty)$. In practice, the covariance structure must be known or estimated well enough to obtain this joint cutoff.

Worked Example: Limiting Correlation Under Unequal Covariances

Let $n_1/N\to1/3$ and suppose the two covariance matrices for two coordinates are $$ \Sigma_1=\begin{pmatrix}4&1\\1&1\end{pmatrix}, \qquad \Sigma_2=\begin{pmatrix}1&0\\0&9\end{pmatrix}. $$ The limiting covariance matrix for the scaled mean difference is $$ A=\frac{\Sigma_1}{1/3}+\frac{\Sigma_2}{2/3} =3\Sigma_1+\frac{3}{2}\Sigma_2 =\begin{pmatrix}13.5&3\\3&16.5\end{pmatrix}. $$ Therefore the limiting correlation is $R_{\infty,12}=3/\sqrt{13.5(16.5)}=3/\sqrt{222.75}\approx0.2010$. The maximum’s null calibration must account for this correlation; it is not the calibration for two independent coordinates.

Growing Dimension and Extreme-Value Calibration

When $p$ grows, a fixed-dimensional normal limit alone does not justify a growing-dimensional cutoff. A useful reference case is a vector of $p$ independent standard normal coordinates. Let $M_p=\max_{1\leq j\leq p}|Z_j|$, where the $Z_j$ are independent $N(0,1)$ variables, and put $m=2p$. Define $$ b_m=\sqrt{2\log m}-\frac{\log\log m+\log(4\pi)}{2\sqrt{2\log m}}, \qquad a_m=\sqrt{2\log m}. $$

Theorem: For independent standard normal coordinates, for every real $x$, $$ P\left(a_m(M_p-b_m)\leq x\right)\longrightarrow\exp(-e^{-x}) \quad\text{as }p\to\infty,\qquad m=2p. $$ Thus an asymptotic level-$\alpha$ cutoff for the Gaussian maximum is $b_m+x_\alpha/a_m$, where $x_\alpha=-\log[-\log(1-\alpha)]$.

Derivation. For $u=b_m+x/a_m$, the standard normal tail satisfies $\bar\Phi(u)\sim\phi(u)/u$ as $u\to\infty$, where $\phi$ and $\bar\Phi$ are the standard normal density and upper-tail probability. Substitution of the displayed $b_m,a_m$ gives $m\bar\Phi(b_m+x/a_m)\to e^{-x}$. Independence and symmetry then yield $$ P(M_p\leq u)=\{1-2\bar\Phi(u)\}^{p} =\{1-(m/p)\bar\Phi(u)\}^{p}\longrightarrow\exp(-e^{-x}), $$ using $p=m/2$ and $(1-c_p)^p\to e^{-c}$ when $pc_p\to c$. This proves the stated limit. $\square$

For a two-sample statistic with estimated standard errors, this Gaussian cutoff is justified only when the standardized maximum is sufficiently well approximated by the corresponding Gaussian maximum. In particular, if $p$ grows, pointwise consistency of each sample variance is not by itself enough: the approximation must hold uniformly across coordinates at the scale of the maximum. Independence across coordinates is also a substantive condition, not a consequence of studentization. Under dependence, calibration must instead use the joint dependence structure or an appropriate dependence correction.

Why the Distinction Matters

The maximum statistic is designed to respond to a few unusually large standardized differences. It is therefore sensitive to both the scale in each coordinate and the multiplicity of searching across coordinates. Correct group-specific standard errors address the first issue; a joint maximum cutoff addresses the second. Neither step can replace the other.

Common mistake: pool under unequal variances. Using one pooled coordinate variance when the groups have different coordinate variances can misstate the null variability. Keep the two terms $S_{1,jj}/n_1$ and $S_{2,jj}/n_2$ separate unless equal covariance matrices are justified.
Common mistake: use a one-coordinate cutoff. Comparing $T_{\max}$ with a marginal normal or $t$ cutoff ignores that the statistic selects the largest result among $p$ coordinates. Use a maximum calibration.
Common mistake: infer independence from standardization. Dividing by coordinatewise standard errors gives unit marginal variances under the null, but it does not remove cross-coordinate correlations. The limiting correlation matrix, or a valid high-dimensional dependence calibration, still matters.

Key takeaway: Form the two-sample maximum from coordinatewise studentized mean differences. Pool variances only under a common-covariance assumption; otherwise use the Welch denominator. Calibrate the maximum using its joint null distribution, with the dimension and coordinate dependence explicitly accounted for.

Check Your Understanding

Work through these questions to check the scaling and calibration logic.

Write the estimated variance of $\bar X_j-\bar Y_j$ when the two groups have unequal coordinate variances.
Under equal covariance matrices, why can the within-group sample variances be pooled for a given coordinate?
In the fixed-dimension theorem, what is the limiting correlation between standardized coordinates $j$ and $k$?
Why does a marginal standard normal cutoff generally fail to calibrate a maximum over many coordinates?
What additional issue arises when using the independent-coordinate extreme-value cutoff for a dependent vector of mean differences?

Back to Tutorials