Bonferroni Correction Explained

Introduction

A statistical test performed at the 5% significance level has a 5% probability of producing a false-positive result when its null hypothesis is true.

That statement is straightforward when there is only one hypothesis. But clinical research frequently involves multiple statistical comparisons: several treatment groups, multiple endpoints, several time points, subgroup analyses, dose comparisons, or combinations of these.

Once multiple hypotheses are tested, the probability of obtaining at least one false-positive result can become substantially larger than 5%.

The Bonferroni correction is one of the simplest and most widely recognized methods for addressing this problem.

Why Multiple Testing Is a Problem

Suppose one hypothesis is tested at: $$ \alpha=0.05. $$ If the null hypothesis is true, there is a 5% chance of obtaining a result that is statistically significant purely by chance.

Now suppose we perform two independent tests, each at the 5% level. The probability that neither test produces a false positive is: $$ (1-0.05)^2=0.95^2=0.9025. $$ Therefore, the probability of at least one false positive is: $$ 1-0.9025=0.0975. $$ So the chance of at least one false-positive result is approximately 9.75%, not 5%.

With ten independent tests: $$ 1-(0.95)^{10}\approx0.4013. $$ The probability of at least one false positive is therefore approximately 40.1%.

What Is the Family-Wise Error Rate?

The family-wise error rate, or FWER, is the probability of making at least one Type I error within a defined family of hypotheses.

If there are \(m\) hypotheses, the FWER can be written as: $$ FWER=P(\text{at least one false rejection}). $$

The goal of a family-wise multiplicity adjustment is typically: $$ FWER\le\alpha. $$

For example, if the study specifies: $$ \alpha=0.05, $$ then the procedure should ensure that the probability of making at least one false-positive rejection across the specified family is no greater than 5%.

What Is a "Family" of Hypotheses?

The word family is critical. The Bonferroni correction is not automatically applied to every statistical test appearing anywhere in a clinical trial. Instead, the analyst must define which hypotheses constitute the relevant family for the multiplicity problem.

Examples might include:

• Three dose-versus-placebo comparisons for one primary endpoint
• Two co-primary hypotheses
• Five secondary endpoint hypotheses
• Multiple treatment comparisons performed as one confirmatory family
• Several prespecified pairwise comparisons in an ANOVA framework

The Bonferroni Inequality

The Bonferroni procedure is based on a simple probability inequality. For events \(A_1,\ldots,A_m\): $$ P\left(\bigcup_{i=1}^{m}A_i\right) \le \sum_{i=1}^{m}P(A_i). $$

In multiple testing, each event can represent making a Type I error for one of the hypotheses.

If each test has Type I error probability \(\alpha/m\), then: $$ FWER \le m\left(\frac{\alpha}{m}\right) = \alpha. $$

This is the mathematical reason the Bonferroni procedure controls FWER.

The Basic Bonferroni Formula

Suppose there are \(m\) hypotheses and the desired family-wise significance level is \(\alpha\). The Bonferroni-adjusted per-comparison significance level is: $$ \boxed{ \alpha^*=\frac{\alpha}{m} } $$

For the common case of \(\alpha=0.05\):

A Simple Example

Suppose a study has four prespecified hypotheses and wants to control FWER at 5%.

Then: $$ m=4 $$ and: $$ \alpha^* = \frac{0.05}{4} = 0.0125. $$

Each individual hypothesis must therefore have a p-value below 0.0125 to be declared statistically significant using the Bonferroni procedure.

Only H1 and H2 are significant because their p-values are below 0.0125.

Adjusted P-Values

There is an equivalent way to perform the Bonferroni procedure. Instead of dividing the significance level by \(m\), multiply each raw p-value by \(m\).

The Bonferroni-adjusted p-value is: $$ p_i^{\text{adj}} = \min(mp_i,1). $$

The value is capped at 1 because a p-value cannot exceed 1.

Using the previous example:

Notice that H1 has an adjusted p-value of: $$ 4(0.004)=0.016. $$ Because 0.016 is below 0.05, H1 is significant.

This gives exactly the same conclusion as comparing the raw p-value with 0.0125.

Bonferroni Adjustment and Confidence Intervals

The same multiplicity principle can be applied to confidence intervals. Suppose \(m\) confidence intervals are constructed and the desired simultaneous confidence level is \(1-\alpha\). The Bonferroni procedure uses: $$ \alpha_i=\frac{\alpha}{m} $$ for each interval.

Therefore, each individual interval is constructed at confidence level: $$ 1-\frac{\alpha}{m}. $$

For five intervals with an overall 95% confidence level: $$ 1-\frac{0.05}{5} = 0.99. $$

Thus, each individual confidence interval would use a 99% confidence level.

Clinical Trial Example: Multiple Treatment Comparisons

Consider a randomized clinical trial with one placebo group and three active dose groups. The investigators want to compare every active dose with placebo for the primary endpoint.

The hypotheses are: $$ H_1:\mu_{\text{Low}}=\mu_{\text{Placebo}} $$ $$ H_2:\mu_{\text{Medium}}=\mu_{\text{Placebo}} $$ $$ H_3:\mu_{\text{High}}=\mu_{\text{Placebo}}. $$

There are: $$ m=3 $$ comparisons.

To maintain a family-wise Type I error rate of 5%: $$ \alpha^* = \frac{0.05}{3} = 0.01667. $$

Suppose the observed p-values are:

Only the medium- and high-dose comparisons are statistically significant after Bonferroni adjustment.

Why the Unadjusted Analysis Can Be Misleading

Without multiplicity adjustment, the medium- and high-dose comparisons would also be significant because their raw p-values are below 0.05. But imagine instead that all three p-values were: $$ 0.041,\quad0.032,\quad0.018. $$ All three would be individually significant at the ordinary 5% level. Yet none would meet the Bonferroni threshold: $$ 0.01667. $$

This illustrates why reporting a collection of ordinary p-values can give an overly optimistic impression of evidence when the hypotheses form a prespecified confirmatory family.

Bonferroni in a Multi-Endpoint Trial

Suppose a clinical trial has two secondary endpoints:

• Change from baseline in biomarker A
• Change from baseline in biomarker B

If both hypotheses are intended to support one overall confirmatory claim and FWER must be controlled at 5%, a simple Bonferroni allocation is: $$ \alpha_1=\alpha_2=0.025. $$

However, equal allocation is not the only possibility.

The overall alpha can be divided unequally: $$ \alpha_1+\alpha_2=\alpha. $$

For example: $$ \alpha_1=0.03, \qquad \alpha_2=0.02. $$

The resulting procedure remains within the Bonferroni framework because: $$ 0.03+0.02=0.05. $$

Weighted Bonferroni

This idea can be generalized using weights. Let: $$ w_1,\ldots,w_m $$ be nonnegative weights satisfying: $$ \sum_{i=1}^{m}w_i=1. $$ Then hypothesis \(i\) can be tested at: $$ \alpha_i=w_i\alpha. $$

For example, suppose there are three hypotheses and the investigators assign weights: $$ w_1=0.50,\qquad w_2=0.30,\qquad w_3=0.20. $$ With: $$ \alpha=0.05, $$ the allocated significance levels are:

This can be useful when hypotheses have different scientific priorities.

Bonferroni Does Not Require Independence

An important advantage of the Bonferroni procedure is that its basic FWER guarantee does not require the tests to be independent.

The Bonferroni inequality remains valid when test statistics are correlated.

This is particularly relevant in clinical research, where endpoints and measurements are often correlated.

Why Is Bonferroni Conservative?

The Bonferroni procedure uses the inequality: $$ P\left(\bigcup A_i\right) \le \sum P(A_i). $$

This bound can be substantially larger than the true probability of at least one Type I error.

Consequently, Bonferroni may use a significance threshold that is more stringent than necessary.

The practical consequence is: fewer false positives, but potentially more false negatives.

Type I Error vs. Type II Error

A multiplicity correction protects against Type I error, but it can reduce power.

Suppose the unadjusted test uses: $$ \alpha=0.05. $$ With ten Bonferroni-adjusted tests: $$ \alpha^*=0.005. $$

Detecting a genuine effect at the much smaller significance level generally requires stronger evidence.

If sample size is not increased, the probability of rejecting a false null hypothesis can decrease.

A Five-Test Worked Example

Suppose five secondary endpoints are tested, with raw p-values: $$ 0.003,\quad0.009,\quad0.014,\quad0.021,\quad0.046. $$

The Bonferroni threshold is: $$ \frac{0.05}{5}=0.01. $$

The adjusted p-values are: $$ 5(0.003)=0.015, $$ $$ 5(0.009)=0.045, $$ $$ 5(0.014)=0.070, $$ $$ 5(0.021)=0.105, $$ $$ 5(0.046)=0.230. $$

Thus, the first two endpoints remain statistically significant after Bonferroni adjustment.

Adjusted P-Values Cannot Be Interpreted as Ordinary P-Values

An adjusted p-value is best understood as a multiplicity-adjusted measure used for decision-making within the specified family.

For example, if: $$ p_{\text{raw}}=0.009 $$ and: $$ p_{\text{Bonf}}=0.045, $$ the appropriate interpretation is that the Bonferroni-adjusted p-value is 0.045 and is below the family-wise threshold of 0.05.

It should not be described as though the original statistical test itself had a raw p-value of 0.045.

Bonferroni and the Minimum P-Value

The Bonferroni procedure can also be understood through the smallest observed p-value. Suppose there are \(m\) tests and: $$ p_{\min}=\min(p_1,\ldots,p_m). $$ A necessary condition for at least one Bonferroni-significant result is: $$ p_{\min}\le\frac{\alpha}{m}. $$

Equivalently: $$ m p_{\min}\le\alpha. $$

Thus, the smallest p-value receives the strongest chance of producing a significant result, but it still has to overcome the multiplicity penalty.

Bonferroni vs. No Correction

Consider ten independent null hypotheses. Without adjustment, each test uses: $$ \alpha=0.05. $$ The approximate FWER is: $$ 1-(1-0.05)^{10} \approx0.401. $$

With Bonferroni: $$ \alpha^*=\frac{0.05}{10}=0.005. $$ The FWER is guaranteed to be no greater than 0.05.

Bonferroni vs. Holm's Procedure

The Holm procedure is a step-down method that also controls FWER but is generally at least as powerful as the ordinary Bonferroni procedure.

Suppose the ordered p-values are: $$ p_{(1)}\le p_{(2)}\le\cdots\le p_{(m)}. $$ Holm's method compares the smallest p-value with: $$ \frac{\alpha}{m}, $$ then the second-smallest with: $$ \frac{\alpha}{m-1}, $$ and continues sequentially.

Therefore, when a study requires strong FWER control but there is no specific reason to use the simple Bonferroni rule, Holm's method is often an attractive alternative.

Bonferroni vs. Hochberg

The Hochberg procedure is another FWER-controlling method that can be more powerful than Bonferroni under appropriate assumptions.

The key distinction is that Hochberg uses a step-up strategy rather than Bonferroni's uniform threshold.

The choice among Bonferroni, Holm, Hochberg, and other methods depends on the study design, dependence structure, regulatory objective, and prespecified multiplicity strategy.

Bonferroni vs. Benjamini-Hochberg

The Benjamini-Hochberg procedure addresses a different error criterion: the false discovery rate (FDR).

FWER asks: What is the probability of making at least one false rejection?

FDR asks, approximately: Among the rejected hypotheses, what proportion are expected to be false discoveries?

Clinical Trial Multiplicity Is More Than Just Counting P-Values

In a clinical trial, multiplicity can arise from several sources:

• Multiple primary endpoints
• Multiple doses
• Multiple treatment comparisons
• Multiple time points
• Multiple estimands
• Multiple populations
• Multiple endpoints
• Multiple interim analyses
• Subgroup analyses

These sources do not necessarily belong to one enormous multiplicity family. The statistical strategy should define which hypotheses contribute to the same confirmatory claim.

Example: Multiple Doses and One Primary Endpoint

Suppose a Phase II dose-ranging study compares three doses against placebo. The primary endpoint is change from baseline in a continuous outcome. The hypotheses are: $$ H_1:\mu_1-\mu_P=0, $$ $$ H_2:\mu_2-\mu_P=0, $$ $$ H_3:\mu_3-\mu_P=0. $$

If all three comparisons are part of one family and the trial requires FWER = 5%, the simple Bonferroni thresholds are: $$ \alpha_1=\alpha_2=\alpha_3=\frac{0.05}{3}=0.01667. $$

Suppose the raw p-values are: $$ 0.012,\quad0.021,\quad0.044. $$ Only the first comparison is significant under Bonferroni because: $$ 0.012<0.01667. $$

What Happens to the Confidence Intervals?

Suppose the same three treatment comparisons are accompanied by confidence intervals. For simultaneous 95% confidence coverage: $$ \alpha=0.05, \qquad m=3. $$ Each interval uses: $$ 1-\frac{0.05}{3} = 0.98333. $$

Thus, the individual intervals are approximately 98.33% confidence intervals.

The intervals are wider than ordinary 95% intervals because simultaneous coverage is more demanding.

Bonferroni and Two-Sided Tests

Suppose the overall family-wise significance level is: $$ \alpha=0.05 $$ and there are: $$ m=4 $$ two-sided hypotheses. The Bonferroni-adjusted significance level for each hypothesis is: $$ \frac{0.05}{4}=0.0125. $$

For a two-sided confidence interval corresponding to this test, the confidence level is: $$ 1-0.0125=0.9875, $$ or 98.75%.

The two tails of that interval together contain 1.25% probability, or 0.625% in each tail under the usual symmetric construction.

R Implementation: Basic Bonferroni Adjustment

R provides a built-in method for Bonferroni adjustment through p.adjust().

p_values <- c(
  0.004,
  0.011,
  0.019,
  0.041
)

p.adjust(
  p_values,
  method = "bonferroni"
)

The result is:

0.016  0.044  0.076  0.164

These are the adjusted p-values.

R: Compare Several Multiplicity Methods

p_values <- c(
  0.004,
  0.011,
  0.019,
  0.041
)

data.frame(
  raw = p_values,
  bonferroni = p.adjust(
    p_values,
    method = "bonferroni"
  ),
  holm = p.adjust(
    p_values,
    method = "holm"
  ),
  hochberg = p.adjust(
    p_values,
    method = "hochberg"
  ),
  BH = p.adjust(
    p_values,
    method = "BH"
  )
)

This is useful for understanding how different multiplicity procedures behave. The methods should not, however, be selected simply because one produces more significant results.

R: Calculate the Bonferroni Threshold Manually

alpha <- 0.05
m <- 5

alpha_bonferroni <- alpha / m

alpha_bonferroni

The result is:

0.01

Therefore, each raw p-value must be below 0.01 for significance at an overall 5% family-wise error rate.

R: Manual Adjusted P-Values

p_values <- c(
  0.003,
  0.009,
  0.014,
  0.021,
  0.046
)

adjusted_p <- pmin(
  p_values * length(p_values),
  1
)

adjusted_p

This implements the basic formula: $$ p_i^{\text{adj}} = \min(mp_i,1). $$

R: Determine Which Hypotheses Are Significant

alpha <- 0.05

bonferroni_p <- p.adjust(
  p_values,
  method = "bonferroni"
)

significant <- bonferroni_p < alpha

data.frame(
  raw_p = p_values,
  adjusted_p = bonferroni_p,
  significant = significant
)

Bonferroni in SAS

Bonferroni adjustment is also available in SAS procedures that support multiplicity-adjusted comparisons. For example, in an analysis of variance framework:

proc glm data=analysis;
    class treatment;
    model change = treatment;
    lsmeans treatment / pdiff=control("Placebo")
                        adjust=bon;
run;

The exact syntax depends on the procedure and type of comparison being performed, so the statistical analysis plan should identify the intended comparison and multiplicity method explicitly.

A Practical Clinical Trial Reporting Example

Suppose three secondary endpoints were tested with raw p-values:

A clear statistical report would identify the adjustment and explain that the Bonferroni-adjusted p-value for Endpoint A was 0.024, which remains below the 5% family-wise significance threshold.

Endpoint B and Endpoint C would not be statistically significant after the prespecified Bonferroni adjustment.

Bonferroni Does Not Make an Insignificant Result "More Significant"

The correction can only make the evidence against the null hypothesis more difficult to declare significant.

For every p-value: $$ p_{\text{adjusted}} \ge p_{\text{raw}}. $$

Therefore, Bonferroni adjustment cannot turn a nonsignificant raw p-value into a significant adjusted p-value.

It can, however, prevent a raw p-value below 0.05 from being considered significant within the multiplicity-controlled family.

Bonferroni Does Not Change the Effect Estimate

Another important distinction is that the Bonferroni correction does not alter the estimated treatment effect itself.

For example, if the estimated treatment difference is: $$ \hat{\theta}=4.2, $$ the correction does not change 4.2 to another estimate.

Instead, it changes the inferential threshold associated with the hypothesis test or confidence interval.

Statistical Significance vs. Clinical Importance

Multiplicity adjustment addresses statistical error control. It does not determine whether an observed effect is clinically meaningful.

For example, an endpoint could have: $$ p_{\text{Bonf}}=0.003 $$ while the treatment difference is too small to be clinically important.

Conversely, a clinically important estimated effect may fail to reach the Bonferroni-adjusted significance threshold because the study has limited precision.

Common Mistake: Dividing Alpha Twice

A frequent error is to apply the Bonferroni correction incorrectly. Suppose: $$ \alpha=0.05 $$ and: $$ m=5. $$ The correct threshold is: $$ 0.05/5=0.01. $$

The analyst should not then divide 0.01 by five again unless a separate, prespecified multiplicity structure requires another adjustment.

Repeatedly applying corrections without understanding the hypothesis structure can be unnecessarily conservative.

Common Mistake: Correcting Every Analysis in the Trial

A clinical trial can contain hundreds of statistical outputs. Not every descriptive analysis necessarily forms part of the same confirmatory family.

For example:

• Baseline descriptive summaries
• Safety listings
• Exploratory subgroup summaries
• Graphical displays
• Prespecified confirmatory efficacy tests

may have different statistical purposes.

The multiplicity strategy should therefore be linked to the scientific claims being made rather than simply to the number of p-values appearing in the tables.

Common Mistake: Defining the Family After Seeing the Results

Suppose ten endpoints are analyzed but only the three with the most promising results are declared to be the "family."

This can undermine the intended Type I error control.

The relevant family should generally be determined based on the scientific questions and decision framework before the confirmatory results are examined.

Common Mistake: Thinking Bonferroni Controls False Discovery Rate

Bonferroni controls the probability of at least one false rejection within the family. That is FWER.

It does not primarily target the false discovery rate.

For exploratory high-dimensional analyses where many hypotheses may be investigated, an FDR-controlling procedure such as Benjamini-Hochberg may be more appropriate.

Common Mistake: Assuming Bonferroni Requires Independent Tests

It does not. The basic Bonferroni guarantee holds regardless of dependence among the hypothesis tests.

This is one reason the procedure is attractive when the dependence structure is complicated or unknown.

Common Mistake: Treating Adjusted P-Values as New Data

The Bonferroni-adjusted p-value is a transformation of the original p-value for multiplicity control. It does not represent a newly collected observation or a new test statistic.

For example: $$ p=0.008 $$ becoming: $$ p_{\text{Bonf}}=0.024 $$ does not mean that the underlying data changed. The inference threshold changed because multiple hypotheses were being considered together.

When Bonferroni Is a Good Choice

Bonferroni is particularly attractive when:

• Strong FWER control is required.
• The number of hypotheses is relatively small.
• The hypotheses have a clear common family.
• A simple and transparent method is desirable.
• The dependence structure is unknown or complicated.
• Regulatory or confirmatory interpretation favors conservative control.
• The analysis needs to be easily reproducible.

When Bonferroni May Be Too Conservative

Bonferroni may be inefficient when there are many correlated hypotheses. For example, if dozens of highly correlated biomarkers are analyzed, dividing alpha equally across every test can result in extremely small significance thresholds.

In such settings, other approaches may provide a better balance between multiplicity control and power.

Possible alternatives include:

• Holm's step-down procedure
• Hochberg's procedure
• Hommel's procedure
• Benjamini-Hochberg FDR control
• Gatekeeping procedures
• Hierarchical testing
• Graphical multiple-testing procedures
• Closed testing procedures

Bonferroni and Hierarchical Testing

Sometimes hypotheses have a natural hierarchy. For example, a clinical trial might specify:

A hierarchical testing strategy can sometimes preserve more power than simply dividing alpha equally across all hypotheses.

The correct method depends on the logical structure of the clinical claims.

Bonferroni and Gatekeeping

Gatekeeping procedures use the hierarchy among endpoint families. For example, one family may receive alpha first, while another family receives alpha only if specified hypotheses in the first family succeed.

This can be useful when a clinical development program has a logical sequence of claims.

Bonferroni and Sample Size

Multiplicity adjustment can affect sample size planning because stricter significance thresholds generally reduce power at a fixed sample size.

Suppose a study originally plans a test at: $$ \alpha=0.05. $$ If the final analysis instead uses: $$ \alpha=0.01, $$ the same sample size will generally have lower power.

If the target power must remain fixed, sample size may need to increase.

Therefore, multiplicity should be incorporated into the sample-size calculation rather than introduced after the sample size has already been finalized.

A Simple Sample-Size Concept

For a two-sided normal approximation, a simplified sample-size relationship contains critical values such as: $$ z_{1-\alpha/2} $$ and: $$ z_{1-\beta}. $$

Reducing \(\alpha\) increases the magnitude of the critical value. Thus, for the same effect size and desired power, the required sample size generally increases.

A Complete Bonferroni Workflow

Worked Example: Four Clinical Endpoints

Suppose a trial evaluates four clinically relevant endpoints. The family-wise error rate is: $$ \alpha=0.05. $$ The four raw p-values are: $$ 0.002,\quad0.013,\quad0.027,\quad0.061. $$

The Bonferroni threshold is: $$ \alpha^* = \frac{0.05}{4} = 0.0125. $$

The adjusted p-values are: $$ 0.008,\quad0.052,\quad0.108,\quad0.244. $$

The important point is that Endpoint 2, despite having a raw p-value very close to 0.01, does not meet the Bonferroni-adjusted criterion.

What Does "Significant After Bonferroni Adjustment" Mean?

A precise interpretation is: The hypothesis was statistically significant under a prespecified Bonferroni procedure controlling the family-wise Type I error rate at the specified level.

For example: $$ p_{\text{Bonf}}=0.008<0.05. $$

This means the hypothesis is significant under the adjusted family-wise decision rule.

It is preferable to report the adjusted p-value rather than simply saying "the p-value was 0.008" if 0.008 is actually the adjusted value.

What Should Be Reported?

A transparent clinical trial report should identify:

• The hypotheses included in the family
• The family-wise error rate
• The multiplicity procedure
• The number of hypotheses
• The raw p-values when appropriate
• The adjusted p-values
• Effect estimates
• Confidence intervals
• The resulting statistical conclusions

For example:

Four prespecified secondary endpoints were evaluated
with family-wise Type I error controlled at 5% using
the Bonferroni procedure. The adjusted p-values were
0.008, 0.052, 0.108, and 0.244, respectively.

Bonferroni and Exploratory Analyses

Exploratory analyses are often intended to generate hypotheses rather than provide definitive confirmatory evidence.

That does not mean exploratory analyses should be interpreted as though there were no multiplicity considerations.

Instead, the interpretation should clearly distinguish:

• Confirmatory hypotheses
• Prespecified secondary hypotheses
• Exploratory analyses
• Post hoc analyses

A p-value from an exploratory analysis should not automatically be presented with the same evidentiary interpretation as a prespecified multiplicity- controlled confirmatory hypothesis.

Bonferroni in Subgroup Analyses

Suppose a trial evaluates treatment effect separately in:

• Men
• Women
• Patients under 65
• Patients 65 or older

If these subgroup hypotheses are being used to make a common confirmatory claim, multiplicity becomes relevant.

However, simply dividing alpha by every subgroup analysis performed is not necessarily the correct strategy.

The analyst must determine which subgroup findings are confirmatory and which are exploratory.

Bonferroni Does Not Solve Every Multiplicity Problem

Bonferroni is a method for controlling a particular error criterion. It does not automatically address every form of adaptive or sequential analysis.

For example, repeated interim analyses of accumulating data may require a group-sequential or alpha-spending framework rather than simply counting the number of analyses and dividing alpha.

Similarly, complex endpoint hierarchies may be better handled by graphical or gatekeeping procedures.

Bonferroni vs. Group-Sequential Methods

Suppose a trial plans interim analyses at 50% and 75% information and a final analysis at 100%. The repeated looks at the accumulating data introduce a sequential testing problem.

A group-sequential design typically specifies efficacy and/or futility boundaries that preserve the overall Type I error.

Simply treating the three analyses as three independent hypotheses and using: $$ 0.05/3 $$ is generally not the standard way to construct an efficient group-sequential design.

Advantages of Bonferroni

• Extremely simple to understand.
• Easy to calculate.
• Easy to explain in a protocol or SAP.
• Controls FWER without requiring independence.
• Works for arbitrary numbers of hypotheses.
• Can be extended through alpha weighting.
• Easy to reproduce in statistical software.
• Provides a conservative and transparent approach to confirmatory testing.

Disadvantages of Bonferroni

• Can be conservative.
• Can reduce statistical power.
• Can become inefficient when the number of hypotheses is large.
• Equal alpha allocation may ignore scientific priorities.
• May be less powerful than stepwise FWER procedures.
• May not be appropriate for exploratory high-dimensional analyses.
• Does not by itself define the appropriate hypothesis family.

The Most Important Formula

If you remember only one formula from this tutorial, remember: $$ \boxed{ \alpha_{\text{Bonferroni}} = \frac{\alpha}{m} } $$ where:

• \(\alpha\) = desired family-wise Type I error rate
• \(m\) = number of hypotheses in the family

The equivalent adjusted p-value formula is: $$ \boxed{ p_{\text{Bonf}} = \min(mp,1) } $$

A Quick Mental Example

Suppose there are eight tests and the desired FWER is 5%. Immediately calculate: $$ 0.05/8=0.00625. $$

Therefore, any raw p-value below 0.00625 is significant under the simple Bonferroni procedure.

Alternatively, multiply every p-value by eight. A raw p-value of 0.004 becomes: $$ 8(0.004)=0.032, $$ which remains below 0.05. A raw p-value of 0.009 becomes: $$ 8(0.009)=0.072, $$ which does not.

Final Checklist

Bottom Line

References

Bonferroni, C.E. (1936). Teoria statistica delle classi e calcolo delle probabilità. Pubblicazioni del R. Istituto Superiore di Scienze Economiche e Commerciali di Firenze, 8, 3–62.

Dunn, O.J. (1961). Multiple comparisons among means. Journal of the American Statistical Association, 56, 52–64.

Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6, 65–70.

Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple tests of significance. Biometrika, 75(4), 800–802.

Benjamini, Y. & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B, 57(1), 289–300.

ICH E9. Statistical Principles for Clinical Trials. International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use.