Scheffé's Bound for All Linear Contrasts

Why a Bound for Every Contrast Is Possible

A contrast may be chosen after examining the design or considering the scientific question. If inference is intended to cover not just a short list of contrasts but every estimable linear contrast, a separate pointwise critical value is not enough: the choice among contrasts itself must be accounted for. Scheffé’s bound handles this broad target by measuring the largest standardized estimation error over the entire contrast space.

The key geometric fact is that, although there are infinitely many contrasts, the maximum standardized error is a single quadratic form in the coefficient estimation error. Under the normal linear model, that quadratic form, divided by the estimated error variance, has an exact F distribution.

Model and notation: Let $Y=X\beta+\varepsilon$, where $X$ is a fixed $n\times p$ matrix of full column rank $p$, $n>p$, and $\varepsilon\sim N(0,\sigma^2 I_n)$ with $\sigma^2>0$. Thus $\hat{\beta}=(X'X)^{-1}X'Y$, $\nu=n-p$, and $s^2=\|Y-X\hat{\beta}\|^2/\nu$. A linear contrast is $c'\beta$, where $c\in\mathbb{R}^p$; full column rank makes every such contrast estimable.

Write $A=X'X$, which is positive definite under the full-rank assumption. The estimated contrast is $c'\hat{\beta}$, and its estimated standard error is $s\sqrt{c'A^{-1}c}$. For a nonzero $c$, this standard error is well-defined when $s>0$. Under the model, $s>0$ with probability one when $\nu>0$.

Maximizing over All Contrast Directions

Consider the standardized error for one contrast, $|c'(\hat{\beta}-\beta)|/[s\sqrt{c'A^{-1}c}]$. Multiplying $c$ by a nonzero scalar does not change this quantity. Therefore, maximizing over all nonzero $c$ is a search over directions in the contrast space, not over their arbitrary scales.

Theorem: For any positive-definite $A$, vector $d\in\mathbb{R}^p$, and $s>0$, $$ \sup_{c\ne 0}\frac{|c'd|}{s\sqrt{c'A^{-1}c}} = \frac{\sqrt{d'Ad}}{s}. $$ If $d\ne 0$, the maximum is attained by a contrast vector proportional to $Ad$. If $d=0$, the supremum is zero.

To prove the identity, let $u=A^{-1/2}c$ and $v=A^{1/2}d$, where $A^{1/2}$ is the positive-definite square root of $A$. Then $c=A^{1/2}u$, so $c'd=u'v$, and $c'A^{-1}c=u'u$. The Cauchy–Schwarz inequality gives $|u'v|\leq \|u\|\|v\|$, hence the displayed ratio is at most $\|v\|/s=\sqrt{d'Ad}/s$. If $d\ne0$, choose $u$ proportional to $v$, which is permitted because $c=A^{1/2}u$ can be any nonzero vector. Equality then holds. When $d=0$, every numerator is zero. This proves the identity. $\square$

Worked Example: Finding the Contrast Direction That Maximizes Error

Suppose $X'X=\begin{pmatrix}4&0\\0&9\end{pmatrix}$, the coefficient estimation error is $d=\hat{\beta}-\beta=(1/2,-1/3)'$, and $s=1$. Find the largest standardized error over all nonzero two-dimensional contrasts and verify a maximizing direction.

The theorem says its square is $d'X'Xd/s^2$. Here $$ d'X'Xd = 4\left(\frac12\right)^2+9\left(-\frac13\right)^2 =1+1=2. $$ Thus the maximum standardized error is $\sqrt{2}$.

A maximizing direction is $c=X'Xd=(2,-3)'$. For this contrast, $c'd=2(1/2)+(-3)(-1/3)=2$, while $$ c'(X'X)^{-1}c = \frac{2^2}{4}+\frac{(-3)^2}{9} =1+1=2. $$ Its standardized error is therefore $2/\sqrt{2}=\sqrt{2}$, agreeing with the maximum. The example illustrates why considering only a few preselected contrasts could miss the direction with the greatest standardized error.

Connecting the Maximum to an F Statistic

For the full contrast space, substitute $d=\hat{\beta}-\beta$ into the maximization identity. The largest squared standardized error is $(\hat{\beta}-\beta)'X'X(\hat{\beta}-\beta)/s^2$. Under the normal model, the numerator after division by $\sigma^2$ is chi-square with $p$ degrees of freedom. The residual sum of squares, also divided by $\sigma^2$, is an independent chi-square variable with $\nu$ degrees of freedom. These two facts yield the required F distribution.

Theorem (Scheffé’s simultaneous bound): Under the stated normal linear-model assumptions, for $0<\alpha<1$, $$ P\left\{ \frac{|c'(\hat{\beta}-\beta)|}{s\sqrt{c'(X'X)^{-1}c}} \leq \sqrt{pF_{p,\nu;1-\alpha}} \ \text{for every }c\ne0 \right\} =1-\alpha, $$ where $F_{p,\nu;1-\alpha}$ is the $1-\alpha$ quantile of the central $F$ distribution with $p$ and $\nu$ degrees of freedom.

Here is the distributional derivation. Put $d=\hat{\beta}-\beta$. From the model, $$ d=(X'X)^{-1}X'\varepsilon \quad\text{and}\quad d\sim N_p\bigl(0,\sigma^2(X'X)^{-1}\bigr). $$ Consequently $Z=(X'X)^{1/2}d/\sigma\sim N_p(0,I_p)$, and $$ Q=\frac{d'X'Xd}{\sigma^2}=Z'Z\sim\chi^2_p. $$ For the residuals, define $H=X(X'X)^{-1}X'$ and $M=I_n-H$. The matrix $H$ is the orthogonal projection onto the column space of $X$, while $M$ projects onto its orthogonal complement. Since $\hat{\varepsilon}=Y-X\hat{\beta}=M\varepsilon$, the residual sum of squares satisfies $$ \frac{\|Y-X\hat{\beta}\|^2}{\sigma^2} = \frac{\varepsilon'M\varepsilon}{\sigma^2} \sim\chi^2_\nu, $$ because $M$ is an orthogonal projection of rank $\nu=n-p$.

The coefficient estimation error depends only on $H\varepsilon$: indeed, $X'\varepsilon=X'H\varepsilon$, since $X'M=0$. The residual vector depends only on $M\varepsilon$. These are independent: they are jointly normal linear transformations of $\varepsilon$, and their cross-covariance is $\sigma^2 HM=0$. Thus $Q$ is independent of the residual sum of squares. With $s^2=\|Y-X\hat{\beta}\|^2/\nu$, it follows that $$ \frac{d'X'Xd}{p s^2} = \frac{Q/p}{\{\|Y-X\hat{\beta}\|^2/\sigma^2\}/\nu} \sim F_{p,\nu}. $$ By the maximization theorem, the supremum of the squared standardized errors is $d'X'Xd/s^2$, or $p$ times this F statistic. Therefore the event that every standardized error is at most $\sqrt{pF_{p,\nu;1-\alpha}}$ is exactly the event that this F statistic is at most $F_{p,\nu;1-\alpha}$, which has probability $1-\alpha$. This proves the simultaneous bound. $\square$

Worked Example: Calculating the Bound in a Three-Parameter Model

Suppose a full-rank normal linear model has $p=3$, residual degrees of freedom $\nu=12$, $X'X=\operatorname{diag}(2,4,8)$, and $s^2=4$. At the 95% level, use $F_{3,12;0.95}\approx3.49$. What is the Scheffé bound on the standardized error of every contrast, and what does it imply for $c=(1,-1,0)'$?

The common critical value is $$ \sqrt{pF_{p,\nu;0.95}} = \sqrt{3(3.49)} = \sqrt{10.47} \approx3.236. $$ Thus, with probability $0.95$, every contrast simultaneously has absolute estimation error no greater than $3.236$ times its estimated standard error.

For $c=(1,-1,0)'$, the design factor is $$ c'(X'X)^{-1}c = \frac12+\frac14 = \frac34. $$ Its estimated standard error is $s\sqrt{3/4}=2\sqrt{3/4}=\sqrt{3}\approx1.732$. The corresponding error bound is approximately $3.236(1.732)=5.60$. The same 95% guarantee applies to this contrast and, simultaneously, to all other nonzero contrasts in the model.

Worked Example: The One-Dimensional Special Case

Consider a normal linear model with $p=1$ and $\nu=10$. There is only one contrast direction (multiplying its coefficient by a nonzero number changes neither its standardized absolute error nor the direction). At the 95% level, $F_{1,10;0.95}\approx4.965$, so the Scheffé critical value is $$ \sqrt{1(4.965)}\approx2.228. $$

This agrees with the two-sided $t$ critical value $t_{10;0.975}\approx2.228$. Indeed, if $Z\sim N(0,1)$ and $V\sim\chi^2_\nu$ independently, then $T=Z/\sqrt{V/\nu}\sim t_\nu$, and $T^2=Z^2/(V/\nu)\sim F_{1,\nu}$. Thus $F_{1,\nu}$ is the square of $t_\nu$, and in one dimension the Scheffé adjustment introduces no extra penalty for searching across contrast directions.

Scope and a Common Mistake

The number $p$ in the critical value counts the dimension of the coefficient space in a full-rank parameterization. It is not the number of contrasts someone happens to list. That distinction is why the bound can protect against choosing any contrast after seeing the data: it controls the maximum over the whole $p$-dimensional space, not just over a specified finite family.

A common mistake is to use a pointwise $t$ critical value for each contrast and conclude that all contrasts are covered simultaneously. A pointwise statement controls one contrast at a time; it does not control the maximum over all directions. The correction is to use the Scheffé critical value $\sqrt{pF_{p,\nu;1-\alpha}}$, whose derivation explicitly accounts for that maximum.

The guarantee concerns linear contrasts that are estimable in the model and depends on the stated normal, independent, constant-variance error assumptions. It is not a guarantee for arbitrary nonlinear functions of $\beta$. For a rank-deficient design, the same argument can be applied after expressing the model in $r=\operatorname{rank}(X)$ independent coefficient coordinates; the relevant dimension is then $r$, and only estimable contrasts are included.

Scheffé’s bound is especially useful when the scientific contrasts are not fixed in advance or when many possible comparisons may be explored. Its price is a critical value based on the full contrast-space dimension, even if only a few contrasts are ultimately reported. The guarantee is broad precisely because it pays for that broad scope.

Check Your Understanding

Answer each question using the model assumptions and results above.

Why does multiplying a contrast vector $c$ by a nonzero scalar leave its standardized absolute error unchanged?
In the maximization identity, which contrast direction attains the maximum when $d\ne0$, and why?
What are the degrees of freedom of the two independent chi-square quantities that form the F statistic?
Why is the Scheffé critical value based on $p$, rather than on the number of contrasts reported?
What assumption allows the coefficient estimation error and residual sum of squares to be treated as independent in the proof?

Scheffé's Bound for All Linear Contrasts

What You'll Learn