Unequal Bonferroni Tail Allocation

Why Allocate Error Unequally?

A simultaneous confidence statement may cover several scientifically meaningful contrasts, but those contrasts need not have equal importance. For example, a primary treatment contrast may deserve a narrower interval than several secondary contrasts. Bonferroni protection allows this priority to be expressed by assigning different portions of the total error budget to different intervals. The guarantee is conservative, but it does not require the contrasts to be independent.

The trade-off is direct: an interval assigned more error budget has lower marginal confidence and a smaller critical value, so it is narrower. An interval assigned less error budget has higher marginal confidence and a larger critical value. The allocation should therefore reflect scientific priorities established before examining the observed contrast estimates.

Model, Contrast Intervals, and Error Allocation

Suppose $X_1,\ldots,X_n$ are independent observations from $N_p(\mu,\Sigma)$, where $\Sigma$ is positive definite and unknown. Write $\bar X$ for the sample mean vector and $S$ for the usual unbiased sample covariance matrix. Consider a prespecified finite family of $m$ nonzero contrast vectors $c_1,\ldots,c_m$. The $j$th target is $c_j^{\mathsf T}\mu$, estimated by $c_j^{\mathsf T}\bar X$.

For each contrast, its estimated standard error is $s_j=\sqrt{c_j^{\mathsf T}Sc_j/n}$. Under this multivariate normal model, the Student pivot $(c_j^{\mathsf T}\bar X-c_j^{\mathsf T}\mu)/s_j$ has a $t$ distribution with $\nu=n-1$ degrees of freedom. This follows by applying the one-sample Student $t$ result to the scalar observations $c_j^{\mathsf T}X_i$, which are independent normal observations with mean $c_j^{\mathsf T}\mu$ and variance $c_j^{\mathsf T}\Sigma c_j$.

Definition: Choose allocations $\alpha_1,\ldots,\alpha_m$ such that $0<\alpha_j<1$ and $\sum_{j=1}^m\alpha_j\le\alpha$, where $\alpha$ is the desired upper bound on familywise error. Let $t_{\nu,q}$ denote the $q$ quantile of the $t_\nu$ distribution. The equal-tailed Bonferroni interval for contrast $j$ is $$ c_j^{\mathsf T}\bar X\ \pm\ t_{\nu,\,1-\alpha_j/2}s_j. $$ Each tail of this interval receives error probability $\alpha_j/2$; the allocations $\alpha_j$ need not be equal across contrasts.

If every contrast receives the same allocation, then $\alpha_j=\alpha/m$ is the familiar equal-allocation choice. Unequal allocation replaces that common value with contrast-specific values. The critical value must be recalculated for each interval; using one common critical value after changing the allocations would not implement the stated procedure.

Why the Familywise Guarantee Holds

The familywise coverage event is the event that all $m$ intervals cover their respective targets. The individual Student intervals have exact marginal coverage under the stated normal model. Their dependence does not invalidate the Bonferroni bound.

Theorem: Under the independent multivariate normal sampling model above, if $0<\alpha_j<1$ and $\sum_{j=1}^m\alpha_j\le\alpha$, the unequal-allocation intervals have familywise coverage at least $1-\alpha$.

Bound each miss probability.
For contrast $j$, the Student pivot gives $P(c_j^{\mathsf T}\mu\text{ is outside its interval})=\alpha_j$.

Express failure of joint coverage.
The event that at least one interval misses is the union of the individual miss events.

Apply the union bound.
The probability of that union is at most the sum of its event probabilities, which is at most $\alpha$.

In detail, let $A_j$ be the event that interval $j$ misses $c_j^{\mathsf T}\mu$. The union bound gives

P\!\left(\bigcup_{j=1}^m A_j\right) \le \sum_{j=1}^m P(A_j) =\sum_{j=1}^m\alpha_j \le\alpha.

Taking complements shows that the probability all intervals cover is at least $1-\alpha$. This proves the claim. No independence assumption among the contrast pivots or miss events was used. The bound need not be exact: dependence among intervals can make the actual familywise coverage larger. $\square$

How Allocation Changes Interval Width

For a fixed contrast and observed standard error $s_j$, the interval width is $2s_jt_{\nu,1-\alpha_j/2}$. The $t_\nu$ quantile is strictly increasing in its probability argument. Consequently, increasing $\alpha_j$ decreases $1-\alpha_j/2$, lowers the critical value, and narrows the interval. This is the mathematical basis for prioritizing an important contrast with a larger allocation.

Result: Holding the data and degrees of freedom fixed, if $0<\alpha_j'<\alpha_j<1$, then $$ 2s_jt_{\nu,\,1-\alpha_j/2} < 2s_jt_{\nu,\,1-\alpha_j'/2}. $$ Thus the interval using $\alpha_j$ is strictly narrower than the interval using $\alpha_j'$, while its marginal coverage is lower.

The inequality follows because $\alpha_j'>0$ and $\alpha_j'<\alpha_j$ imply $1-\alpha_j'/2>1-\alpha_j/2$. Strict monotonicity of the $t_\nu$ quantile then gives $t_{\nu,1-\alpha_j/2}<t_{\nu,1-\alpha_j'/2}$. Multiplication by the same positive standard error and then by two preserves the inequality. $\square$

There is no universally best unequal allocation. Increasing one $\alpha_j$ requires reducing allocations elsewhere if the total budget is fixed. The important interval becomes narrower at the expense of wider intervals, and the individual coverage levels change accordingly. A familywise lower bound is preserved, but allocation cannot make every interval narrower at once.

Worked Example: Three Contrasts With Different Priorities

Suppose $n=10$, so $\nu=9$, and the desired familywise error bound is $\alpha=0.10$. Three prespecified contrasts have estimates and standard errors $(1.20,0.50)$, $(0.40,0.20)$, and $(-0.30,0.25)$. Assign allocations $0.08,0.01,0.01$, respectively. They sum to $0.10$.

For contrast 1, each tail receives $0.08/2=0.04$. The critical value is $t_{9,0.96}\approx1.97265$, so its margin is $1.97265(0.50)=0.986325$. Its interval is approximately $[1.20-0.986325,\ 1.20+0.986325]=[0.2137,\ 2.1863]$, with marginal coverage $1-0.08=0.92$.

For contrasts 2 and 3, each tail receives $0.01/2=0.005$. Thus the critical value is $t_{9,0.995}\approx3.24984$. Contrast 2 has margin $3.24984(0.20)=0.649968$, giving $[-0.2500,\ 1.0500]$. Contrast 3 has margin $3.24984(0.25)=0.812460$, giving $[-1.1125,\ 0.5125]$. Each has marginal coverage $0.99$.

The familywise coverage is at least $1-(0.08+0.01+0.01)=0.90$. The first contrast receives most of the error budget, so its interval has lower marginal coverage than the other two. This is a deliberate priority choice, not a claim that the first interval has greater certainty.

Worked Example: Prioritizing a Primary Contrast

Consider two prespecified contrasts from a sample of $n=20$, giving $\nu=19$, with a familywise error budget of $0.05$. Suppose the primary contrast has estimate $0.80$ and standard error $0.15$; the secondary contrast has estimate $0.20$ and standard error $0.30$. Allocate $0.04$ to the primary contrast and $0.01$ to the secondary contrast.

For the primary contrast, the critical value is $t_{19,0.98}\approx2.2047$. Its margin is $2.2047(0.15)=0.330705$, and its interval is approximately $[0.4693,\ 1.1307]$. Its marginal coverage is $0.96$.

For the secondary contrast, the critical value is $t_{19,0.995}\approx2.8609$. Its margin is $2.8609(0.30)=0.85827$, and its interval is approximately $[-0.6583,\ 1.0583]$. Its marginal coverage is $0.99$. The allocations sum to $0.05$, so the familywise coverage is at least $0.95$.

To see the priority effect at the same standard error, a contrast assigned $0.01$ would use margin $2.8609(0.15)=0.429135$, larger than the primary contrast's margin $0.330705$. Spending more of the budget on the primary contrast narrows its interval, while the total familywise guarantee remains intact.

Worked Example: Comparing Equal and Unequal Allocation

Take two contrasts with $\nu=9$, standard errors $0.50$ and $0.20$, and a total error budget $\alpha=0.10$. Their estimates are $0.60$ and $-0.10$. Under equal allocation, each receives $0.05$, so the critical value is $t_{9,0.975}\approx2.26216$. The margins are $2.26216(0.50)=1.13108$ and $2.26216(0.20)=0.452432$; the intervals are approximately $[-0.5311,\ 1.7311]$ and $[-0.5524,\ 0.3524]$.

Now allocate $0.08$ to the first contrast and $0.02$ to the second. These still sum to $0.10$. The critical values are $t_{9,0.96}\approx1.97265$ and $t_{9,0.99}\approx2.82144$. The resulting margins are $1.97265(0.50)=0.986325$ and $2.82144(0.20)=0.564288$. The intervals become approximately $[-0.3863,\ 1.5863]$ and $[-0.6643,\ 0.4643]$.

The first interval narrows, while the second widens. The equal allocation gives each interval $0.95$ marginal coverage; the unequal allocation gives the first $0.92$ and the second $0.98$. Both choices have the same Bonferroni lower bound of $0.90$ for familywise coverage.

Choosing Allocations Responsibly

Allocations are part of the inferential design. They should be determined from the scientific aims—for example, by assigning more error budget to a primary contrast and less to exploratory contrasts—and documented along with the contrast family. The allocation does not depend on the observed estimates or on which intervals happen to exclude zero.

Prespecification matters because choosing the most favorable allocation after seeing the data is a data-dependent procedure, not the fixed Bonferroni construction proved above. Nor does the word “important” mean “statistically significant”: scientific priority should be defined independently of the observed result. If the scientific objectives do not justify unequal priorities, equal allocation remains a straightforward default.

Common Mistakes

Using the same critical value after changing the allocation. Each contrast uses $t_{\nu,1-\alpha_j/2}$, so unequal allocations generally require unequal critical values.
Dividing the total familywise budget by two only once. First choose each contrast's allocation $\alpha_j$; then divide that contrast's allocation equally between its two tails.
Assuming independence is needed. The union bound holds for dependent events. Bonferroni protection does not require independent contrasts.
Thinking the total guarantee is unchanged without a budget constraint. The stated bound requires $\sum_j\alpha_j\le\alpha$. Allocations exceeding the total budget do not establish the desired familywise coverage.
Allocating after inspecting the results. A larger allocation narrows an interval, but data-driven choice of which interval to narrow is not justified by the fixed-allocation guarantee.

Key takeaway: Assign each prespecified contrast a positive error allocation, split that allocation equally between its two tails, and use its own Student $t$ critical value. If the allocations sum to at most $\alpha$, the familywise coverage is at least $1-\alpha$; a larger allocation buys a narrower interval by reducing that contrast's marginal coverage.

Check Your Understanding

Use the definitions and coverage argument above to answer these questions.

For three contrast intervals, what condition on their allocations guarantees a Bonferroni familywise error bound of at most $0.05$?
If one contrast's allocation increases while its standard error and degrees of freedom stay fixed, what happens to its critical value and interval width?
Why does the Bonferroni familywise coverage argument not require independent contrast estimates?
With a per-contrast allocation of $0.02$, how much error probability is assigned to each tail of an equal-tailed interval?
Why should unequal allocations be chosen before examining the observed contrast estimates?