Bonferroni Intervals for Prespecified Contrasts

Why Adjust Intervals for a Family?

An interval for one mean contrast is designed to cover its target with a stated probability. But when several contrasts are reported together, the relevant question is whether all intervals cover their targets at once. Even if every individual interval has high coverage, the probability that at least one misses can be larger. Bonferroni intervals address this by assigning each contrast a smaller error probability, then bounding the chance of any miss.

The construction is especially useful when the contrasts are chosen before examining the data and their estimates are correlated. Its guarantee does not require those estimates, or their interval failures, to be independent.

Model and Interval Definition

Suppose $X_1,\ldots,X_n$ are independent observations from a $p$-variate normal distribution with mean vector $\mu$ and unknown positive-definite covariance matrix $\Sigma$. Let $\bar X$ be the sample mean and let $S$ be the usual unbiased sample covariance matrix:

X_i\sim N_p(\mu,\Sigma),\qquad S=\frac{1}{n-1}\sum_{i=1}^n(X_i-\bar X)(X_i-\bar X)'.

Consider $m$ fixed, nonzero vectors $c_1,\ldots,c_m$ in $\mathbb R^p$. The $j$th target is the linear mean contrast $\theta_j=c_j'\mu$, estimated by $c_j'\bar X$. In comparisons among coordinates, the entries of a contrast vector often sum to zero; the interval formula also applies to other fixed nonzero linear functionals of $\mu$. The vectors must be specified independently of the observed data for the stated coverage guarantee.

Definition: For $0<\alpha<1$, the two-sided Bonferroni interval for contrast $c_j'\mu$ is $$ c_j'\bar X\ \pm\ t_{n-1,\,1-\alpha/(2m)} \sqrt{\frac{c_j'Sc_j}{n}}, $$ where $t_{\nu,q}$ is the $q$ quantile of a Student $t$ distribution with $\nu$ degrees of freedom. The family consists of these $m$ intervals, one for each prespecified contrast.

The estimated standard error is the ordinary one-sample standard error for the projected observations $c_j'X_i$. The adjustment replaces the single-interval tail probability $\alpha/2$ with $\alpha/(2m)$ in each tail. Thus each interval has marginal error probability $\alpha/m$, not $\alpha$.

Why the Construction Has Its Coverage Guarantee

Theorem: Under the independent, identically distributed multivariate normal model above, for $n\geq2$, $0<\alpha<1$, and any fixed family of $m$ nonzero contrast vectors, the Bonferroni intervals cover all $m$ targets simultaneously with probability at least $1-\alpha$: $$ P\left( c_j'\mu\text{ belongs to its interval for every }j=1,\ldots,m \right)\geq 1-\alpha. $$

To establish the result, first consider one fixed contrast $c\neq0$. The projected observations $Y_i=c'X_i$ are independent normal random variables with mean $c'\mu$ and variance $\sigma_c^2=c'\Sigma c>0$. Their sample mean is $c'\bar X$, and their unbiased sample variance is $c'Sc$.

For normal observations, $Z=\sqrt n(c'\bar X-c'\mu)/\sigma_c$ is standard normal, and $U=(n-1)c'Sc/\sigma_c^2$ has a chi-square distribution with $n-1$ degrees of freedom, independently of $Z$. By the definition of the Student $t$ distribution, $Z/\sqrt{U/(n-1)}$ has a $t_{n-1}$ distribution. Substitution gives the pivot

\frac{c'\bar X-c'\mu}{\sqrt{c'Sc/n}}\sim t_{n-1}.

Consequently, for $q=t_{n-1,\,1-\alpha/(2m)}$, the probability that this pivot lies between $-q$ and $q$ is $1-\alpha/m$. The two tails each have probability $\alpha/(2m)$, so the probability that this one interval misses its target is $\alpha/m$.

Now let $A_j$ be the event that interval $j$ misses $c_j'\mu$. The event that at least one interval misses is $\bigcup_{j=1}^m A_j$. The union bound gives

P\left(\bigcup_{j=1}^m A_j\right) \leq \sum_{j=1}^m P(A_j) = m\frac{\alpha}{m} =\alpha.

Taking complements proves that the probability all intervals cover their targets is at least $1-\alpha$. No independence among the events $A_j$ was used. Equality is not promised: dependence among contrasts, or overlap among their failure events, can make the actual joint coverage greater than the lower bound. $\square$

Worked Interval Calculations

Worked Example: Two Coordinate Mean Contrasts

Suppose five independent observations are $(1,2),(2,1),(3,4),(4,3),(5,5)$, sampled from a bivariate normal distribution with unknown mean and covariance. Construct 95% simultaneous Bonferroni intervals for the two coordinate means.

The sample mean is $\bar X=(3,3)'$. The centered first coordinates are $-2,-1,0,1,2$, and the centered second coordinates are $-1,-2,1,0,2$. Their sums of squares are both 10, and the sum of their cross-products is $2+2+0+0+4=8$. Since $n-1=4$,

S=\frac14 \begin{pmatrix}10&8\\8&10\end{pmatrix} = \begin{pmatrix}2.5&2\\2&2.5\end{pmatrix}.

Here $m=2$, $\alpha=0.05$, and the degrees of freedom are 4. The critical value is $t_{4,\,0.9875}\approx3.495$. For $c_1=(1,0)'$, the estimate is $3$ and the standard error is $\sqrt{2.5/5}=\sqrt{0.5}\approx0.7071$. Its half-width is $3.495(0.7071)\approx2.471$, giving the interval $(0.529,5.471)$.

For $c_2=(0,1)'$, the estimate and standard error are also $3$ and $0.7071$, so the interval is again $(0.529,5.471)$. The two intervals have joint coverage at least $0.95$ under the stated model. This guarantee concerns repeated samples, not certainty that these particular observed intervals contain the unknown means.

Worked Example: Contrasts Between Three Coordinates

Suppose a sample of $n=10$ independent three-dimensional normal observations has $\bar X=(4,2,1)'$ and

S= \begin{pmatrix} 4&1&0\\ 1&9&2\\ 0&2&4 \end{pmatrix}.

Construct 90% simultaneous intervals for $c_1'\mu$ and $c_2'\mu$, where $c_1=(1,-1,0)'$ and $c_2=(0,1,-1)'$. These targets are $\mu_1-\mu_2$ and $\mu_2-\mu_3$. The covariance matrix is positive definite: its leading principal minors are $4$, $35$, and $124$, all positive.

There are $m=2$ intervals, so the critical value is $t_{9,\,0.975}\approx2.262$. The first estimated contrast is $4-2=2$, and $c_1'Sc_1=4+9-2(1)=11$. Its standard error is $\sqrt{11/10}\approx1.049$, giving half-width $2.262(1.049)\approx2.372$. The interval is approximately $(-0.372,4.372)$.

The second estimated contrast is $2-1=1$, and $c_2'Sc_2=9+4-2(2)=9$. Its standard error is $\sqrt{9/10}\approx0.949$, so its half-width is $2.262(0.949)\approx2.146$. The interval is approximately $(-1.146,3.146)$. Together, these intervals have coverage at least $0.90$.

Worked Example: A Family With Three Prespecified Contrasts

Suppose $n=8$, $\bar X=(2,5)'$, and

S=\begin{pmatrix}1&0.5\\0.5&4\end{pmatrix}.

For a 85% familywise guarantee, use $\alpha=0.15$ and the three prespecified contrasts $c_1=(1,0)'$, $c_2=(0,1)'$, and $c_3=(1,1)'$. The matrix $S$ is positive definite because its leading principal minors are $1$ and $4-0.25=3.75$. With $m=3$ and 7 degrees of freedom, the critical value is $t_{7,\,0.975}\approx2.365$.

The three estimates are $2$, $5$, and $7$. Their estimated contrast variances before division by $n$ are $c_1'Sc_1=1$, $c_2'Sc_2=4$, and $c_3'Sc_3=1+4+2(0.5)=6$. The standard errors are therefore $\sqrt{1/8}\approx0.354$, $\sqrt{4/8}\approx0.707$, and $\sqrt{6/8}\approx0.866$.

Multiplying each standard error by $2.365$ gives half-widths of approximately $0.836$, $1.672$, and $2.048$. The respective intervals are $(1.164,2.836)$, $(3.328,6.672)$, and $(4.952,9.048)$. The theorem guarantees that all three targets are covered with probability at least $0.85$.

When the Bonferroni Guarantee Is Useful

This method gives a finite-sample guarantee even when the contrasts are correlated and their joint distribution would be inconvenient to calculate. Each interval uses the same adjusted critical value, while its standard error reflects the variability of its particular contrast through $c_j'Sc_j$. Adding more contrasts increases the adjustment and generally widens the intervals.

The lower bound can be conservative because the union bound does not account for the dependence structure. If several contrasts tend to miss together, summing their separate error probabilities can overstate the probability of any miss. Conservatism affects the interval widths, not the validity of the stated lower coverage bound.

Common Mistakes

Using the unadjusted critical value for every interval. That gives each interval its own marginal confidence level, but does not establish that the whole family has that coverage. Use $t_{n-1,\,1-\alpha/(2m)}$ for a two-sided family of $m$ intervals.
Dividing $\alpha$ by $m$ in each tail. The total error for one two-sided interval is $\alpha/m$, split equally into two tails. Thus each tail receives $\alpha/(2m)$.
Assuming the contrasts must be independent. The proof uses the union bound and requires no such independence. It does require the stated marginal $t$ distributions.
Choosing contrasts after examining the data. The result is for a prespecified family. Selecting contrasts based on observed results changes the inferential problem and is not covered by this guarantee.
Claiming exact joint coverage of $1-\alpha$. Bonferroni guarantees at least $1-\alpha$; the actual coverage can be larger.

Key takeaway: For $m$ prespecified Gaussian mean contrasts, use a two-sided $t$ critical value with tail probability $\alpha/(2m)$ and the contrast-specific standard error $\sqrt{c_j'Sc_j/n}$. The union bound then ensures joint coverage of at least $1-\alpha$, regardless of dependence among contrasts.

Check Your Understanding

Use the model assumptions and definitions above to answer the following.

For $m=5$ two-sided intervals and familywise error level $\alpha$, what probability is assigned to each tail of each interval?
Why does the marginal $t$ pivot for a contrast use $c'Sc$, rather than a single diagonal entry of $S$ in general?
Which step in the joint coverage proof makes independence among contrasts unnecessary?
If each interval separately has noncoverage probability $\alpha/m$, what lower bound follows for the probability that all $m$ intervals cover?
Why does the stated guarantee not automatically apply to a family of contrasts selected after inspecting the observed data?