Bonferroni Versus Scheffé by Contrast Geometry

Two Ways to Protect a Family of Contrasts

Suppose a normal linear model has several contrast estimates to report. The inferential target is not just whether each individual interval has high coverage; it is whether all intervals cover their targets together. Bonferroni protects a specified finite family by dividing the error budget among its members. Scheffé protects every contrast in a parameter space at once. The trade-off is between a critical value that responds to the family’s size and one that responds to the dimension of the full contrast space.

To compare widths fairly, consider the same contrast under both methods. Each interval’s width contains a standard-error factor determined by that contrast. The methods differ in the multiplier applied to that factor. The geometry of the family matters in a subtler way: it does not change Bonferroni’s multiplier, but it helps determine how much protection a finite family actually needs compared with a guarantee covering every direction.

Model, Contrast Family, and Interval Formulas

Definition: Assume the full-rank normal linear model \(Y\sim N_n(X\beta,\sigma^2 I_n)\), where \(X\) has \(p\) columns and \(n>p\). Write \(\hat{\beta}=(X'X)^{-1}X'Y\), let \(s^2\) be the residual mean square, and let \(\nu=n-p\) be its degrees of freedom. A contrast is \(c'\beta\), where \(c\in\mathbb{R}^p\) is nonzero. Its estimated standard error is \(s\sqrt{q(c)}\), where \(q(c)=c'(X'X)^{-1}c\). A specified finite family consists of \(m\) contrasts \(c_1'\beta,\ldots,c_m'\beta\).

Let \(t_{\nu,\gamma}\) denote the \(\gamma\)-quantile of the central \(t\) distribution with \(\nu\) degrees of freedom, and let \(F_{p,\nu;\gamma}\) denote the \(\gamma\)-quantile of the central \(F\) distribution with degrees of freedom \(p,\nu\). For two-sided Bonferroni intervals with familywise error target \(\alpha\), each member receives error allowance \(\alpha/m\), split equally between its two tails:

c_j'\hat{\beta}\ \pm\ t_{\nu,\,1-\alpha/(2m)}\,s\sqrt{q(c_j)},\qquad j=1,\ldots,m.

Scheffé’s simultaneous intervals for all contrasts in the \(p\)-dimensional coefficient space are

c'\hat{\beta}\ \pm\ \sqrt{pF_{p,\nu;\,1-\alpha}}\,s\sqrt{q(c)},\qquad c\ne 0.

The Bonferroni multiplier depends on \(m\) and \(\nu\); the Scheffé multiplier depends on \(p,\nu\), and \(\alpha\), but not on the number of contrasts selected for display. Since every member of a finite family is among all contrasts, the Scheffé intervals also cover that family simultaneously.

Why Both Procedures Control Simultaneous Coverage

For an individual contrast, the statistic \((c'\hat{\beta}-c'\beta)/(s\sqrt{q(c)})\) has a central \(t_\nu\) distribution under the stated model. Thus the Bonferroni interval for that contrast has noncoverage probability \(\alpha/m\). The guarantee for all \(m\) intervals follows without requiring their errors to be independent.

Theorem: The Bonferroni intervals displayed above have simultaneous coverage at least \(1-\alpha\). Scheffé’s intervals have simultaneous coverage at least \(1-\alpha\) for every contrast in \(\mathbb{R}^p\), and hence for every specified finite family.

Proof of the Bonferroni guarantee. Let \(A_j\) be the event that interval \(j\) fails to cover its contrast. By the individual \(t\) result and the two-sided critical value, \(P(A_j)=\alpha/m\). The event of at least one failure is the union of these events. The union bound therefore gives

P\left(\bigcup_{j=1}^m A_j\right) \le \sum_{j=1}^m P(A_j) = m\frac{\alpha}{m} =\alpha.

Taking complements proves that the probability all intervals cover is at least \(1-\alpha\). No independence assumption about the contrast estimates was used. \(\square\)

Why the Scheffé multiplier works. Under the model, define \(Z=(X'X)^{1/2}(\hat{\beta}-\beta)/\sigma\) and \(W=\nu s^2/\sigma^2\). Then \(Z\sim N_p(0,I_p)\), \(W\sim\chi^2_\nu\), and \(Z\) and \(W\) are independent. For any nonzero contrast \(c\), set \(u=(X'X)^{-1/2}c/\sqrt{q(c)}\), which is a unit vector. Its studentized error is \(u'Z/\sqrt{W/\nu}\). As \(u\) ranges over all unit vectors, the largest absolute numerator is \(\|Z\|\), by the Cauchy–Schwarz inequality, with equality in the direction of \(Z\) when \(Z\ne0\). Consequently,

\sup_{\|u\|=1}\frac{(u'Z)^2}{W/\nu} =\frac{\|Z\|^2}{W/\nu} =p\frac{\|Z\|^2/p}{W/\nu}.

The final ratio divided by \(p\) has an \(F_{p,\nu}\) distribution. Hence the probability that the supremum is at most \(pF_{p,\nu;\,1-\alpha}\) is \(1-\alpha\). This gives simultaneous coverage over all contrasts. Restricting the supremum to a finite family cannot make it larger, which proves the stated familywise guarantee. \(\square\)

Width Comparison and Contrast Geometry

For any given contrast \(c\), both interval widths are its estimated standard error multiplied by a method-specific critical value and by 2. Therefore their direct comparison is especially simple: Bonferroni is narrower precisely when its critical value is smaller than Scheffé’s. The standard error affects the absolute widths, but it cancels from their ratio.

\frac{\text{Bonferroni width for }c}{\text{Scheffé width for }c} = \frac{t_{\nu,\,1-\alpha/(2m)}}{\sqrt{pF_{p,\nu;\,1-\alpha}}}.

To describe geometry, associate to contrast \(c_j\) the unit vector \(u_j=(X'X)^{-1/2}c_j/\sqrt{q(c_j)}\). The correlation between the two contrast estimates is

\operatorname{Corr}(c_i'\hat{\beta},c_j'\hat{\beta}) = \frac{c_i'(X'X)^{-1}c_j}{\sqrt{q(c_i)q(c_j)}} =u_i'u_j.

Thus angles between these unit vectors encode the dependence among standardized contrast estimates. Bonferroni uses none of these angles: for fixed \(m,\nu,\alpha\), it assigns the same multiplier whether the directions are nearly identical or widely spread. Scheffé instead protects every direction in the \(p\)-dimensional space. The more a finite family occupies only a restricted portion of that space, the more potentially conservative an all-directions guarantee can be. This geometric observation does not change either method’s promised coverage; it explains why the guarantees can differ in sharpness.

Worked Example: Two Orthogonal Contrasts

Suppose \(p=2\), \(\nu=20\), \(\alpha=0.05\), and \((X'X)^{-1}=I_2\). Let \(s=0.5\), and consider \(c_1=(1,0)'\) and \(c_2=(0,1)'\). Both have \(q(c_j)=1\), so each estimated standard error is \(0.5\). The two directions are orthogonal.

Here \(m=2\). The Bonferroni critical value is \(t_{20,\,1-0.05/4}=t_{20,0.9875}\approx2.423\). Its half-width for either contrast is \(2.423(0.5)=1.2115\), so the full width is \(2.423\).

For Scheffé, \(\sqrt{2F_{2,20;\,0.95}}\approx\sqrt{2(3.493)}\approx2.643\). The half-width is \(2.643(0.5)=1.3215\), giving full width \(2.643\). Bonferroni is narrower for this family. The comparison concerns the multipliers; the orthogonality has not been used to calculate the Bonferroni value.

Worked Example: Adding a Third Direction

Keep \(p=2\), \(\nu=20\), \(\alpha=0.05\), \((X'X)^{-1}=I_2\), and \(s=0.5\), but now use \(c_1=(1,0)'\), \(c_2=(0,1)'\), and \(c_3=(1,1)'/\sqrt{2}\). Each has \(q(c_j)=1\), so all three standard errors are \(0.5\). The third direction is at a 45-degree angle to each of the first two.

Now \(m=3\), and the Bonferroni critical value is \(t_{20,\,1-0.05/6}=t_{20,0.991666\ldots}\approx2.613\). Each Bonferroni interval has full width \(2(2.613)(0.5)=2.613\). The Scheffé multiplier remains approximately \(2.643\), so its width is \(2(2.643)(0.5)=2.643\). Bonferroni remains slightly narrower, but adding a contrast has increased its width from \(2.423\) to \(2.613\). Scheffé’s width is unchanged because the parameter dimension and confidence level have not changed.

Worked Example: A Larger Family Across the Same Space

Continue with \(p=2\), \(\nu=20\), \(\alpha=0.05\), \((X'X)^{-1}=I_2\), and \(s=0.5\). Consider \(m=20\) distinct unit contrast directions, with angles \(\theta_j=(j-1)\pi/20\) for \(j=1,\ldots,20\), so \(c_j=(\cos\theta_j,\sin\theta_j)'\). Each has \(q(c_j)=1\); the directions spread around the plane, treating opposite vectors as the same unoriented contrast direction.

The Bonferroni critical value is \(t_{20,\,1-0.05/40}=t_{20,0.99875}\approx3.455\). Thus each interval has width \(2(3.455)(0.5)=3.455\). The Scheffé width remains \(2(2.643)(0.5)=2.643\). In this family, Scheffé is narrower. It does not pay an increasing penalty for each listed direction, while Bonferroni’s critical value rises with \(m\). The family also spreads through the full two-dimensional contrast space, making the distinction between a finite-family guarantee and an all-directions guarantee especially relevant.

What the Geometry Does—and Does Not—Change

The examples show that neither method is always narrower. For a fixed model dimension, residual degrees of freedom, and confidence level, Scheffé has a fixed multiplier; increasing \(m\) eventually makes Bonferroni less attractive. For a fixed family size, changing the directions does not change Bonferroni’s multiplier. It does change the correlations among estimates, and therefore how much overlap or redundancy the family contains.

A family of nearly parallel directions has strongly related standardized estimates. Counting them as \(m\) separate Bonferroni members still uses the same \(\alpha/m\) allocation. A dispersed family may probe more distinct directions, while remaining subject to exactly the same Bonferroni rule when \(m\) is unchanged. Scheffé takes the opposite stance: it ignores the selected family’s arrangement and protects the whole sphere of directions. That broad guarantee can be valuable when contrasts are not fixed in advance or when an all-contrasts statement is required, but it may be more protection than a small, prespecified family needs.

The direct width comparison should not be mistaken for a comparison of actual joint coverage. Both procedures guarantee at least the chosen familywise level; either may exceed it. Their intervals can have different widths while both meet the guarantee. The ratio of critical values answers which interval is wider for a particular contrast, not how much coverage either procedure actually achieves for a particular geometry.

Common Pitfall

Common mistake: “The contrast estimates are highly correlated, so Bonferroni should use a smaller critical value.”

That reasoning confuses dependence with Bonferroni’s stated calibration. The standard Bonferroni critical value is determined by \(m,\nu,\alpha\), and does not become smaller when correlations are high. The correction is to distinguish the conservative union-bound guarantee from the family’s geometry: correlations can affect how conservative the guarantee is, but they do not alter the ordinary Bonferroni formula. Likewise, do not conclude that Scheffé must be wider simply because it covers all contrasts; compare its multiplier \(\sqrt{pF_{p,\nu;\,1-\alpha}}\) directly with \(t_{\nu,\,1-\alpha/(2m)}\).

Key takeaway: For the same contrast, compare critical values to compare Bonferroni and Scheffé widths. Bonferroni pays for the number of specified contrasts; Scheffé pays for the dimension of the entire contrast space. Contrast geometry describes dependence and the potential conservatism of those guarantees, but does not enter the standard Bonferroni critical value.

Check Your Understanding

Use the model conditions and formulas above to answer these questions.

For a family of \(m\) two-sided contrasts, why does Bonferroni use the quantile \(t_{\nu,\,1-\alpha/(2m)}\) rather than \(t_{\nu,\,1-\alpha/m}\)?
If \(p,\nu,\alpha\), and each contrast standard error stay fixed while \(m\) increases, which method’s critical value changes?
How can the angle between two whitened contrast vectors be interpreted in terms of their estimated contrasts?
Suppose Bonferroni’s critical value is smaller than Scheffé’s. Which interval is narrower for a given contrast, and why does its standard error not change that conclusion?
Why can a highly correlated finite family make Bonferroni conservative without changing its standard critical value?