Scheffé Inference for All Group Mean Contrasts

Why Inference for All Contrasts Requires a Larger Critical Value

With several groups, a researcher may compare a treatment with a control, compare two treatments, or examine a weighted average of groups against another weighted average. If the contrast is chosen only after looking at the data, an ordinary confidence interval for that one contrast does not account for the selection. Scheffé inference addresses this by constructing regions that cover every linear contrast at once. A contrast selected later from the same data is then still covered.

We work with $g$ independent groups. Group $i$ contains $n_i$ observations from a $p$-variate normal distribution with mean vector $\mu_i$ and common positive-definite covariance matrix $\Sigma$. Write $N=\sum_{i=1}^g n_i$, $\nu=N-g$, and let $\bar X_i$ be the sample mean vector for group $i$. The pooled within-group covariance estimator is $S$, formed by dividing the pooled within-group sum-of-squares-and-cross-products matrix by $\nu$. We assume $\nu\ge p$, so $S$ is positive definite with probability one under this model.

Definition: A group mean contrast is $\theta_c=\sum_{i=1}^g c_i\mu_i$, where the fixed coefficients $c_1,\ldots,c_g$ satisfy $\sum_{i=1}^g c_i=0$. Its estimator is $\hat\theta_c=\sum_{i=1}^g c_i\bar X_i$, and its variance multiplier is $a_c=\sum_{i=1}^g c_i^2/n_i$. Thus $\operatorname{Cov}(\hat\theta_c)=a_c\Sigma$. We exclude the all-zero coefficient vector, for which $a_c=0$.

The zero-sum condition makes the contrast compare group means rather than their common overall level. The covariance formula follows from independence across groups and $\operatorname{Cov}(\bar X_i)=\Sigma/n_i$. The estimated covariance of $\hat\theta_c$ is therefore $a_cS$.

The Scalar Scheffé Bound for One Response Coordinate

First fix a response coordinate $r$, where $r\in\{1,\ldots,p\}$. Let $\mu_{ir}$ and $\bar X_{ir}$ denote the $r$th entries of $\mu_i$ and $\bar X_i$, and let $S_{rr}$ estimate the variance $\sigma_{rr}$ of that response. The scalar contrast error is $\sum_i c_i(\bar X_{ir}-\mu_{ir})$. The key step is to maximize its standardized squared error over all zero-sum coefficient vectors.

Theorem: Under the pooled Gaussian model, for a fixed coordinate $r$, $$ \sup_{\substack{c\ne 0\\ \sum_i c_i=0}} \frac{\left\{\sum_{i=1}^g c_i(\bar X_{ir}-\mu_{ir})\right\}^2}{a_c S_{rr}} \ \sim\ (g-1)F_{g-1,\nu}. $$ Here $F_{q,\nu}$ denotes an $F$ random variable with $q=g-1$ and $\nu$ degrees of freedom.

To prove the distributional statement, let $D=\operatorname{diag}(1/n_1,\ldots,1/n_g)$ and $e=(\bar X_{1r}-\mu_{1r},\ldots,\bar X_{gr}-\mu_{gr})'$. Independence and normality give $e\sim N(0,\sigma_{rr}D)$. Define $y=D^{-1/2}e$, so $y\sim N(0,\sigma_{rr}I_g)$. For any admissible $c$, put $u=D^{1/2}c/\sqrt{a_c}$. Then $u'u=1$, and $$ \frac{c'e}{\sqrt{a_c}}=u'y,\qquad 1'c=0\ \Longleftrightarrow\ 1'D^{-1/2}u=0. $$ As $c$ ranges over nonzero zero-sum vectors, $u$ ranges over the unit sphere in the $q=g-1$ dimensional subspace orthogonal to $D^{-1/2}1$. Therefore the supremum of $(c'e)^2/a_c$ is the squared length of the orthogonal projection of $y$ onto that subspace. In an orthonormal basis for the subspace, the $q$ projected coordinates are independent $N(0,\sigma_{rr})$ variables. Consequently, $$ \sup_{\substack{c\ne0\\1'c=0}}\frac{(c'e)^2}{a_c\sigma_{rr}} \sim\chi^2_q. $$ This whitening is essential: the untransformed group-mean error vector has covariance $\sigma_{rr}D$, not $\sigma_{rr}I_g$, so an ordinary orthonormal basis of the zero-sum subspace would not generally yield equal-variance independent coordinates.

Under the pooled Gaussian model, the within-group residual sum of squares for coordinate $r$, divided by $\sigma_{rr}$, has a $\chi^2_\nu$ distribution and is independent of the group sample means. Thus $\nu S_{rr}/\sigma_{rr}\sim\chi^2_\nu$, independently of the numerator supremum. Dividing the independent chi-square variables by their degrees of freedom gives the stated $qF_{q,\nu}$ distribution. This completes the proof. $\square$

Simultaneous Regions for Vector Contrasts

The scalar theorem applies separately to each of the $p$ response coordinates. To obtain a joint guarantee across all coordinates, use the theorem at level $1-\alpha/p$ for each coordinate and apply the Bonferroni inequality. Independence between response coordinates is not required; they may be correlated through $\Sigma$.

Theorem: For $0<\alpha<1$, define $$ K^2=(g-1)F_{g-1,\nu;\,1-\alpha/p}, $$ where $F_{q,\nu;\,1-\alpha/p}$ is the $1-\alpha/p$ quantile of the $F_{q,\nu}$ distribution. Then, with probability at least $1-\alpha$, simultaneously for every nonzero $c$ satisfying $\sum_i c_i=0$ and every coordinate $r$, $$ \left|\hat\theta_{c,r}-\theta_{c,r}\right| \le K\sqrt{a_cS_{rr}}. $$ Hence a simultaneous confidence region for every vector contrast $\theta_c$ is the coordinate-wise box $$ \prod_{r=1}^p \left[\hat\theta_{c,r}-K\sqrt{a_cS_{rr}},\ \hat\theta_{c,r}+K\sqrt{a_cS_{rr}}\right]. $$

For each fixed coordinate $r$, the scalar theorem says the probability that any contrast violates its bound with the corresponding scalar critical value is $\alpha/p$. The selected critical value $K$ is exactly that scalar critical value, since its square is $qF_{q,\nu;1-\alpha/p}$. Let $A_r$ be the event that at least one contrast violates the bound in coordinate $r$. Then $P(A_r)\le\alpha/p$. The probability that any coordinate fails is bounded by $$ P\left(\bigcup_{r=1}^p A_r\right) \le \sum_{r=1}^pP(A_r) \le p\frac{\alpha}{p} =\alpha. $$ On the complementary event, all coordinate inequalities hold for every contrast simultaneously, which is exactly the asserted box coverage. $\square$

These are simultaneous coordinate-wise regions, not generally ellipsoids. Their construction is conservative because Bonferroni does not use the dependence among response coordinates. But their coverage claim is direct: the same random event covers every zero-sum contrast, including one chosen after inspecting the results. For any fixed response-direction vector $b$, the same event also gives $$ |b'(\hat\theta_c-\theta_c)| \le K\sqrt{a_c}\sum_{r=1}^p |b_r|\sqrt{S_{rr}}, $$ by the triangle inequality. Thus arbitrary linear combinations of response coordinates inherit a valid simultaneous bound.

Worked Examples

Worked Example: A Two-Group Contrast With Two Responses

Suppose two independent groups have sample sizes $n_1=10$ and $n_2=12$, and each observation has $p=2$ responses. Let $N=22$, so $g=2$, $\nu=N-g=20$. Consider the difference $\theta_c=\mu_1-\mu_2$, with $c=(1,-1)'$.

The coefficients sum to zero, and the variance multiplier is $a_c=1^2/10+(-1)^2/12=1/10+1/12=11/60$. The estimated difference is $\hat\theta_c=\bar X_1-\bar X_2$. At familywise level $1-\alpha=0.95$, the coordinate-wise critical constant is $K=\sqrt{F_{1,20;\,0.975}}$, because $g-1=1$ and $1-\alpha/p=0.975$.

The simultaneous region for this difference has coordinate $r$ half-width $\sqrt{(11/60)S_{rr}F_{1,20;\,0.975}}$. For example, if the observed pooled variances are $S_{11}=4$ and $S_{22}=9$, the half-widths are respectively $\sqrt{(11/60)(4)F_{1,20;\,0.975}}$ and $\sqrt{(11/60)(9)F_{1,20;\,0.975}}$. The critical value is larger than the one-coordinate 95% critical value because the target is simultaneous coverage for both responses, not just one.

Worked Example: A Weighted Contrast Across Three Groups

Consider three groups with $n_1=6,n_2=6,n_3=12$, each measured on two responses. For the contrast with $c=(1,1,-2)'$, the coefficients sum to $1+1-2=0$. Its variance multiplier is $a_c=1/6+1/6+4/12=1/3+1/3=2/3$.

Suppose the observed mean vectors are $\bar X_1=(10,4)'$, $\bar X_2=(13,5)'$, and $\bar X_3=(12,8)'$. Then $\hat\theta_c=\bar X_1+\bar X_2-2\bar X_3=(10+13-24,\ 4+5-16)'=(-1,-7)'$. Here $g=3$, $N=24$, and $\nu=21$. With $\alpha=0.05$, the critical constant is $K=\sqrt{2F_{2,21;\,0.975}}$.

If the pooled covariance has diagonal entries $S_{11}=4$ and $S_{22}=9$, the two half-widths are $K\sqrt{(2/3)4}=K\sqrt{8/3}$ and $K\sqrt{(2/3)9}=K\sqrt{6}$. The resulting box is centered at $(-1,-7)'$. Its stated coverage belongs to the simultaneous family over all zero-sum contrasts, not just this particular weighted comparison.

Worked Example: The One-Response Case and a Numerical Critical Value

Take $p=1$, three groups, and $\nu=18$. At familywise level $0.95$, the Scheffé squared critical constant is $2F_{2,18;\,0.95}$. For an $F_{2,\nu}$ variable, the upper-tail probability is $P(F_{2,\nu}>x)=(\nu/(\nu+2x))^{\nu/2}$. Solving $(18/(18+2x))^9=0.05$ gives $x=9(0.05^{-1/9}-1)\approx3.555$. Thus $K=\sqrt{2(3.555)}\approx2.666$.

For group sizes $n=(8,10,12)$, compare groups 1 and 3 using $c=(1,0,-1)'$. Then $a_c=1/8+1/12=5/24$. If $S_{11}=4$, the standard-error estimate is $\sqrt{a_cS_{11}}=\sqrt{(5/24)4}=\sqrt{5/6}\approx0.913$. The simultaneous interval half-width is $2.666(0.913)\approx2.434$. This interval is protected even if the contrast is chosen from the full family of group comparisons after inspecting the data.

Interpretation, Rescaling, and Common Pitfalls

The protection comes from maximizing over the entire $g-1$ dimensional space of group contrasts before setting the critical value. It is therefore appropriate when the analyst may explore many weighted comparisons and report a contrast selected for its apparent size. A procedure calibrated only for a pre-specified contrast does not automatically have this protection.

Contrast coefficients can be rescaled without changing the underlying comparison, but the sign and scale must be interpreted correctly. Replacing $c$ by $\lambda c$ changes $\theta_c$ and $\hat\theta_c$ by the factor $\lambda$, changes $a_c$ by $\lambda^2$, and changes the standard-error width by $|\lambda|$. Positive rescaling preserves direction; negative rescaling reverses it. For example, $c=(1,-1)$ estimates $\mu_1-\mu_2$, whereas $-c=(-1,1)$ estimates $\mu_2-\mu_1$. Their confidence regions have the same width with centers reflected through zero.

Using a single-contrast critical value after selection. An interval valid for a pre-specified contrast need not cover a contrast chosen after examining the data. Scheffé’s supremum explicitly protects all contrasts.
Ignoring the response dimension. Applying the scalar result to each of $p$ coordinates at level $1-\alpha$ does not guarantee joint coverage $1-\alpha$. The stated construction uses $1-\alpha/p$ per coordinate.
Whitening incorrectly. Group mean errors have covariance proportional to $D$, not the identity. The transformation $y=D^{-1/2}e$ is what makes the projection argument valid.
Treating the box as an ellipsoid. The theorem gives a coordinate-wise box. It does not claim that this box is an exact multivariate ellipsoid or that its boundary has an $F$ distribution.
Rescaling with the wrong sign interpretation. A negative multiplier reverses the contrast direction; only the width uses the absolute multiplier.

Scheffé regions can be wider than regions designed for a short, pre-planned list of comparisons. That is the cost of allowing unrestricted data-driven selection. When the scientific question genuinely involves all possible contrasts, the broader guarantee is the relevant one; when only a small family was fixed in advance, a method tailored to that family may be more efficient.

Check Your Understanding

Use the pooled Gaussian assumptions and distinguish the all-contrast guarantee from inference for a single pre-specified comparison.

Why must the coefficients of a group mean contrast sum to zero?
For $c=(1,-1,0)'$ and group sizes $(8,10,12)$, calculate $a_c$.
In the projection proof, why is $D^{-1/2}e$ used instead of $e$ directly?
Why does the simultaneous coordinate-wise construction use a quantile at $1-\alpha/p$?
If a contrast is multiplied by $-3$, how do its estimate, variance multiplier, and confidence-region width change?