Why Adjust Pairwise Regions?
When several groups have multivariate means, a separate confidence region for each pairwise difference does not automatically provide reliable coverage for the whole collection. Even if every region has high individual coverage, examining many regions creates more opportunities for at least one to miss its target. Bonferroni adjustment controls that familywise risk without requiring the pairwise comparisons to be independent.
The adjustment works by making each individual region more conservative. The resulting guarantee is simple and robust: the probability that all pairwise regions cover their mean differences is at least the chosen familywise confidence level. The size of that guarantee, and the extra size of the regions, can both be quantified.
Exact Pairwise Hotelling Regions
Suppose \(k\) independent groups provide observations from \(p\)-variate normal distributions with group means \(\mu_1,\ldots,\mu_k\) and a common positive-definite covariance matrix \(\Sigma\). Let group \(i\) have sample size \(n_i\), sample mean \(\bar X_i\), and let \(N=\sum_{i=1}^k n_i\). The pooled within-group covariance estimator is
Assume \(\nu\ge p\), so \(S_p\) is positive definite with probability one under this model. For a pair \(i<j\), write \(\delta_{ij}=\mu_i-\mu_j\) and \(\hat\delta_{ij}=\bar X_i-\bar X_j\). The exact Hotelling confidence region at individual noncoverage probability \(\alpha_{ij}\) is
Here \(F_{p,\nu-p+1}(1-\alpha_{ij})\) is the \(1-\alpha_{ij}\) quantile of an \(F\) distribution with the indicated degrees of freedom. Under the stated normal, independence, and common-covariance assumptions, the exact Hotelling calibration gives \(P(\delta_{ij}\in C_{ij}(\alpha_{ij}))=1-\alpha_{ij}\). The factor \(n_i n_j/(n_i+n_j)\) accounts for the sampling variation of the difference between the two sample means.
This region is an ellipsoid centered at the observed difference. Equivalently, its estimated covariance shape is \((1/n_i+1/n_j)S_p\), since \((1/n_i+1/n_j)^{-1}=n_i n_j/(n_i+n_j)\). The confidence level enters through the \(F\) quantile: a smaller noncoverage probability yields a larger cutoff and a larger region.
Worked Example: Counting Comparisons and Setting the Local Level
A study compares all pairs among five treatment groups and seeks at least 95% simultaneous coverage. There are
distinct pairwise differences. With equal Bonferroni allocation, the noncoverage probability assigned to each pair is
Thus each exact pairwise Hotelling region is constructed at confidence level \(1-0.005=0.995\), or 99.5%. The corresponding cutoff is \(\frac{p\nu}{\nu-p+1}F_{p,\nu-p+1}(0.995)\). The family consists of all ten comparisons, not merely a selected subset. This example determines the adjustment; the numerical cutoff still depends on the dimension and pooled residual degrees of freedom.
Bonferroni Regions and Simultaneous Coverage
Let \(m=k(k-1)/2\) be the number of unordered pairs. For a desired familywise noncoverage bound \(\alpha\), construct every pairwise region using \(\alpha_{ij}=\alpha/m\). The resulting collection is called the Bonferroni family of pairwise regions.
Proof. Let \(A_{ij}\) be the event that the region for pair \(i,j\) misses \(\delta_{ij}\). Exact individual calibration gives \(P(A_{ij})=\alpha/m\). Failure of simultaneous coverage is the event that at least one pair is missed, so the union bound gives
Taking the complement shows \(P(\delta_{ij}\in C_{ij}(\alpha/m)\text{ for every }i<j)\ge 1-\alpha\). The inequality, rather than an equality in general, is the source of Bonferroni conservatism. \(\square\)
The proof only uses the marginal error probability for each pair. Pairwise differences share group means, and the regions also use the same \(S_p\); their coverage events are generally dependent. That dependence does not invalidate the bound.
Worked Example: Three Groups With an Explicit Ellipsoid Inflation
Consider three groups, each with \(n_i=2\), and two-dimensional responses, so \(p=2\), \(N=6\), and \(\nu=N-k=3\). Suppose the observed group means are \((0,0)'\), \((1,0)'\), and \((0,1)'\), and the pooled covariance estimate is \(S_p=I_2\). There are \(m=3\) pairwise comparisons.
For these degrees of freedom, the relevant distribution is \(F_{2,2}\). Its cumulative distribution function is \(x/(1+x)\) for \(x>0\), so its \(1-\beta\) quantile is \((1-\beta)/\beta\). The Hotelling multiplier is
For ordinary 95% individual regions, \(\beta=0.05\), giving an \(F\) quantile of \(0.95/0.05=19\) and a Hotelling cutoff of \(3\cdot19=57\). For 95% familywise coverage, allocate \(\beta=0.05/3=1/60\) to each pair. The adjusted \(F\) quantile is \((59/60)/(1/60)=59\), and the adjusted cutoff is \(3\cdot59=177\).
For each pair, \(n_i n_j/(n_i+n_j)=4/4=1\), and \(S_p=I_2\). Thus each region is a disk centered at its observed mean difference, with radius \(\sqrt{57}\) before adjustment and \(\sqrt{177}\) after adjustment. The radius ratio is
The adjusted regions are therefore about 76% larger in radius. Their area ratio is \(177/57=59/19\approx3.105\). The centers are \((-1,0)'\), \((0,-1)'\), and \((1,-1)'\) for differences ordered as group 1 minus group 2, group 1 minus group 3, and group 2 minus group 3. The adjustment changes the cutoffs, not these observed centers.
Quantifying the Conservatism
For any particular pair, define \(q_{ij}=1/n_i+1/n_j\). Its region has shape matrix \(q_{ij}S_p\) and cutoff \(c(\beta)=\frac{p\nu}{\nu-p+1}F_{p,\nu-p+1}(1-\beta)\). The volume of a \(p\)-dimensional ellipsoid with this shape matrix and cutoff is
where \(V_p\) is the volume of the unit ball in \(p\) dimensions. This follows by writing points in the ellipsoid as the center plus \((q_{ij}S_p)^{1/2}u\), where \(u'u\le c(\beta)\): the linear transformation scales volume by the square root of the determinant, and scaling a \(p\)-dimensional ball's radius by \(\sqrt{c(\beta)}\) scales its volume by \(c(\beta)^{p/2}\).
Derivation. The shape matrix \(q_{ij}S_p\) and the volume constant \(V_p\) are the same in the adjusted and unadjusted regions. Dividing their volume expressions cancels both, leaving \([c(\alpha/m)/c(\alpha)]^{p/2}\). Since an ellipsoid's linear dimensions scale with the square root of its cutoff, their ratio is \(\sqrt{c(\alpha/m)/c(\alpha)}\). For \(m>1\), \(\alpha/m<\alpha\), so the \(F\) quantile and cutoff increase. The adjusted region cannot be smaller. \(\square\)
This quantifies enlargement pair by pair, but it does not determine the actual simultaneous coverage. The union bound guarantees at least \(1-\alpha\); the true joint coverage can be greater. How much greater depends on the joint behavior of the comparisons, so it cannot be calculated from the individual confidence levels alone.
Worked Example: Six Comparisons With Unequal Group Sizes
Suppose four groups have sizes \(2,2,2,1\), and responses are two-dimensional. Then \(N=7\), \(k=4\), \(\nu=3\), and \(m=\binom{4}{2}=6\). Assume the pooled covariance estimate is nonsingular. For 95% familywise coverage, each pair receives noncoverage probability \(0.05/6=1/120\), so each exact region has confidence level \(119/120\).
As in the preceding calculation, \(p=2,\nu=3\), so the Hotelling multiplier is 3 and the \(F_{2,2}\) quantile at \(119/120\) is \(119\). The Bonferroni cutoff is therefore \(3\cdot119=357\). The corresponding unadjusted 95% cutoff is \(3\cdot19=57\). For any fixed pair, the shape matrix is unchanged by adjustment, so the volume ratio is
and the linear-dimension ratio is \(\sqrt{119/19}\approx2.503\). The differing group sizes affect each pair's shape through \(q_{ij}\): a pair of size-two groups has \(q_{ij}=1/2+1/2=1\), while a size-two versus size-one pair has \(q_{ij}=1/2+1=3/2\). These pair-specific shapes are retained; the same adjusted cutoff applies to every pair because the degrees of freedom and allocated level are common.
Common Mistake and Practical Interpretation
Mistake: Construct each of six pairwise regions at 95% confidence and conclude that the probability all six cover is 95%. Each region does have 95% marginal coverage, but that statement alone does not give 95% simultaneous coverage. The probability of at least one miss can exceed 5%.
Correction: For a familywise target of 95%, assign each of the six comparisons noncoverage probability \(0.05/6\), then use the corresponding exact Hotelling cutoff. Bonferroni controls the probability of any miss even when comparisons are dependent. The price is potentially substantial enlargement, especially for many comparisons or heavy-tailed \(F\) quantiles.
Bonferroni regions are useful when a transparent, assumption-light familywise guarantee is more important than obtaining the smallest possible regions. Report both the simultaneous target and the adjustment, and interpret the guaranteed coverage as “at least” the target. It is not generally correct to label the actual joint coverage as exactly 95%, or to treat the volume inflation as a measure of how much the actual joint coverage exceeds 95%.
Check Your Understanding
Use the definitions and results above to check your understanding.
- How many pairwise regions are in a family of seven group means, and what local noncoverage probability gives a Bonferroni familywise bound of 0.05?
- Why does the Bonferroni coverage guarantee remain valid when pairwise coverage events are dependent?
- For a fixed pair, what happens to its ellipsoid’s linear dimensions when the Hotelling cutoff is multiplied by four? What happens to its volume?
- Why is the simultaneous coverage guaranteed to be at least, rather than necessarily exactly, \(1-\alpha\)?
- With unequal group sizes, which part of a pairwise region changes from pair to pair, and which part of the Bonferroni adjustment is common to all pairs?