Graphical Approaches to Multiple Testing

Introduction

Clinical trials frequently evaluate several hypotheses simultaneously. Examples include multiple primary or key secondary endpoints, multiple doses, different patient populations, or combinations of efficacy objectives.

If each hypothesis were tested independently at the conventional significance level, the probability of making at least one false positive decision could be substantially greater than the desired family-wise error rate.

Traditional procedures such as Bonferroni and Holm solve this problem by adjusting significance levels according to relatively simple rules.

But clinical-trial hypotheses often have a much richer structure.

For example, an investigator might want to:

• Prioritize a primary endpoint.
• Give a smaller amount of alpha to supportive endpoints.
• Test a key secondary endpoint only after success on the primary endpoint.
• Transfer unused alpha after a successful hypothesis.
• Recycle alpha among several hypotheses.
• Allow different hypotheses to receive different fractions of recycled alpha.

A graphical multiple-testing procedure provides a natural way to represent these relationships.

The Multiplicity Problem

Suppose a clinical trial tests three null hypotheses:

If each hypothesis is tested at the one-sided level 0.05, the probability of at least one false rejection can exceed 5%.

If the tests were independent, for example, the probability of no false rejection when all three null hypotheses are true would be:

Therefore the probability of at least one false rejection would be:

or approximately 14.3%.

The exact family-wise error rate depends on the dependence structure of the tests, but the basic problem remains: testing several hypotheses without multiplicity control can inflate the probability of at least one false-positive conclusion.

Family-Wise Error Rate

The family-wise error rate, or FWER, is the probability of making at least one false rejection among a family of hypotheses.

A multiplicity procedure is commonly designed to satisfy:

For a confirmatory clinical trial, a common one-sided family-wise error target is:

The graphical procedure determines how that overall alpha budget is distributed and subsequently transferred among the hypotheses.

Why a Graph?

A graphical procedure makes the testing strategy visible.

Each hypothesis is represented by a node. The initial allocation of alpha is represented by the node's starting weight. Directed arrows represent possible transfers of alpha after a hypothesis is rejected.

For example:

The initial local significance levels sum to the overall alpha:

If \(H_1\) is rejected, the arrow labeled 1.0 means that all of its available alpha can be transferred to \(H_2\).

If \(H_2\) is subsequently rejected, its available alpha can be transferred to \(H_3\).

Hypotheses as Nodes

Suppose a trial has three objectives:

The initial alpha weights are:

because:

Multiplying these weights by the family-wise alpha gives:

so:

Initial Alpha Weights

The initial weights describe how the total type I error budget is allocated before any hypothesis has been rejected.

In general, if there are \(m\) hypotheses:

with:

The initial local significance level for hypothesis \(H_i\) is:

A hypothesis with a larger initial weight receives a larger initial testing level.

Transition Weights

Initial weights tell us where alpha starts.

Transition weights tell us where alpha goes next.

Suppose \(H_i\) has an outgoing transition to \(H_j\) with weight \(g_{ij}\).

The transition weights from a node satisfy:

and, when all available alpha is distributed after rejection:

For example:

If \(H_1\) is rejected, 70% of its available alpha is transferred to \(H_2\).

If \(H_1\) had local alpha 0.025, the transfer would be:

If another 30% is directed toward \(H_3\), then:

and the full amount of \(H_1\)'s alpha has been redistributed:

Alpha Recycling

One of the most powerful features of graphical procedures is alpha recycling.

Instead of allowing alpha associated with a rejected hypothesis to disappear, the procedure can transfer that alpha to hypotheses that remain active.

For example:

If \(H_1\) is rejected, its available alpha moves to \(H_2\).

If \(H_2\) is then rejected, its accumulated alpha can be transferred again.

The alpha therefore follows a path through the graph.

A Simple Serial Graph

The simplest graphical procedure is a sequential chain:

The interpretation is:

1. Test \(H_1\) at 0.025.
2. If \(H_1\) is rejected, transfer its available alpha to \(H_2\).
3. Test \(H_2\) at its updated alpha.
4. If \(H_2\) is rejected, transfer its available alpha to \(H_3\).
5. Test \(H_3\) at its updated alpha.

This is closely related to a hierarchical or gatekeeping strategy.

Graphical Procedures Are More General Than Simple Gatekeeping

A strict hierarchy might require:

with all alpha moving sequentially from one hypothesis to the next.

A graphical procedure can be more flexible.

For example, after rejecting \(H_1\), alpha could be split between \(H_2\) and \(H_3\):

This allows the statistical strategy to reflect a more complex clinical decision structure.

Graphical Testing in Matrix Form

For a larger procedure, it is often useful to describe the graph using a transition matrix.

Suppose there are three hypotheses and the transition matrix is:

The first row says that when \(H_1\) is rejected, all available alpha is transferred to \(H_2\).

The second row says that when \(H_2\) is rejected, all available alpha is transferred to \(H_3\).

The third row contains no outgoing transition because \(H_3\) is the final hypothesis in this simple chain.

The Graph Changes After a Rejection

The graphical procedure should be thought of as a dynamic system.

Initially:

If \(H_1\) is rejected, its available alpha is transferred according to the transition weights.

With a 1.0 transition from \(H_1\) to \(H_2\):

The total alpha remains:

If \(H_2\) is then rejected and all of its available alpha moves to \(H_3\):

The total alpha budget remains 0.05.

A Complete Worked Example

Consider a confirmatory clinical trial with three hypotheses:

The total family-wise error rate is:

The graph is:

The transition rules are:

• If \(H_1\) is rejected, transfer all of its available alpha to \(H_2\).
• If \(H_2\) is rejected, transfer all of its available alpha to \(H_3\).
• If a hypothesis is not rejected, its alpha is not transferred through that rejection pathway.

Step 1: Test the Primary Hypothesis

Suppose the primary endpoint produces:

The initial significance level for \(H_1\) is:

Because:

we reject \(H_1\).

The full 0.025 alpha associated with \(H_1\) is now transferred to \(H_2\).

Step 2: Update the Alpha for \(H_2\)

Before the rejection of \(H_1\), \(H_2\) had:

After receiving the 0.025 transferred from \(H_1\):

Suppose the p-value for \(H_2\) is:

Since:

we reject \(H_2\).

Step 3: Transfer Alpha to \(H_3\)

The available alpha for \(H_3\) initially is:

The rejected \(H_2\) now transfers its entire 0.040 alpha to \(H_3\).

Therefore:

Suppose:

Since:

we reject \(H_3\).

Worked Example Result

The sequence is:

What Happens if \(H_1\) Fails?

Now consider a different result for the same prespecified procedure.

Suppose:

The primary hypothesis is tested at:

Since:

\(H_1\) is not rejected.

Under the simple serial graph, its alpha is therefore not transferred to \(H_2\).

If \(H_2\) has:

then:

so \(H_2\) is also not rejected.

If \(H_3\) has \(p_3=0.012\), then:

and \(H_3\) is not rejected either.

Partial Alpha Transfer

The graph does not have to transfer all alpha to one hypothesis.

Suppose \(H_1\) distributes its available alpha as follows:

If \(H_1\) has 0.025 available alpha, then \(H_2\) receives:

while \(H_3\) receives:

The total transferred alpha remains:

Why Partial Transfer Is Useful

Partial transfer can represent situations in which several hypotheses remain clinically important after a particular objective succeeds.

For example, a trial might want to prioritize:

• A key secondary endpoint at 70% of recycled alpha.
• A second key secondary endpoint at 30%.

This is more flexible than a strict serial hierarchy.

Alpha Recycling Back to an Earlier Hypothesis

Graphical procedures can also contain cycles.

For example, alpha from \(H_2\) might be partially returned to \(H_1\) while another portion moves to \(H_3\).

A conceptual graph might be:

This kind of recycling is useful when the hypotheses are not naturally arranged in a one-way hierarchy.

Graphical Procedures and Gatekeeping

Gatekeeping procedures divide hypotheses into families or testing sequences and determine when later hypotheses become eligible for testing.

For example:

A graphical procedure can represent this same logic using nodes and transition weights.

Graphical Procedures and Hierarchical Testing

Hierarchical testing is a particularly simple special case of a graphical strategy.

Consider:

If all alpha is transferred sequentially, the graph encodes a hierarchy.

The advantage of the graphical representation becomes more apparent when the strategy becomes more complicated.

For example:

• \(H_1\) may transfer 60% to \(H_2\) and 40% to \(H_3\).
• \(H_2\) may transfer alpha to \(H_3\).
• \(H_3\) may recycle some alpha to another hypothesis.

A graph can display all of these relationships explicitly.

Graphical Procedures and Bonferroni

The Bonferroni procedure can also be viewed through the lens of alpha allocation.

If \(m\) hypotheses receive equal weights:

then each hypothesis initially receives:

For three hypotheses and \(\alpha=0.05\):

This illustrates an important conceptual difference.

Bonferroni allocates alpha statically, whereas a graphical procedure can redistribute alpha dynamically according to prespecified testing relationships.

Graphical Procedures and Holm

Holm's step-down procedure also creates a sequence in which the testing level changes as hypotheses are processed.

However, graphical procedures are generally specified through an explicit network of initial weights and transition weights rather than simply ranking p-values and applying a step-down formula.

This makes graphical procedures particularly attractive when the order and relationships among clinical hypotheses have substantive meaning.

Local Alpha Versus Overall Alpha

A common source of confusion is the difference between the overall family-wise alpha and the local alpha assigned to an individual hypothesis.

Suppose:

and \(H_1\) initially receives weight 0.50.

Its local testing level is:

This does not mean the trial has a 2.5% family-wise error target.

The 0.025 is the current local level at which \(H_1\) is tested within the overall 0.05 multiplicity framework.

What Does a Rejected Hypothesis Contribute?

Suppose \(H_i\) has current local alpha:

and is rejected.

If its transition weight to \(H_j\) is \(g_{ij}\), the amount transferred is:

If several hypotheses receive transfers, then the receiving hypothesis may accumulate alpha from multiple sources.

For example:

where the summation is over rejected hypotheses that transfer alpha to \(H_j\).

Multiple Incoming Paths

Suppose \(H_3\) receives alpha from both \(H_1\) and \(H_2\).

If:

with transition weight 0.50 to \(H_3\), and:

with transition weight 0.40 to \(H_3\), then \(H_3\) receives:

or:

If \(H_3\) originally had 0.010 alpha, its updated level becomes:

This is one reason graphical procedures can become difficult to implement manually in large trials.

Graphical Procedures With Multiple Endpoints

A common clinical-trial application is a primary endpoint followed by several key secondary endpoints.

For example:

A graphical strategy might allocate more initial alpha to \(H_1\), while allowing successful testing of \(H_1\) to increase the levels available for the secondary objectives.

Graphical Procedures With Multiple Doses

The same framework can be used when multiplicity arises from multiple doses.

For example, suppose a study evaluates:

• Low dose
• Medium dose
• High dose

and each dose has an efficacy hypothesis.

The graph can encode a strategy such as:

Graphical Procedures With Multiple Families

More complex trials may contain several families of hypotheses.

For example:

• Family 1: primary efficacy endpoints
• Family 2: key secondary endpoints
• Family 3: dose-related hypotheses
• Family 4: patient-reported outcomes

A graphical procedure can connect individual hypotheses or groups of hypotheses depending on the desired testing strategy.

This can be especially useful when the trial has multiple clinically meaningful paths to success.

Closed Testing and Graphical Procedures

Graphical procedures are closely related to the broader framework of closed testing.

The closed-testing principle requires testing appropriate intersection hypotheses so that rejection of an individual hypothesis is supported by the required family of tests.

Many graphical procedures can be interpreted as computationally efficient ways of implementing multiple-testing strategies that maintain strong control of the FWER.

Strong Control of the FWER

A major goal of confirmatory multiplicity procedures is strong control of the family-wise error rate.

Strong control means that the probability of at least one false rejection is controlled at or below the target alpha for every configuration of true and false null hypotheses.

Symbolically:

for every possible configuration of the hypotheses.

This is stronger than controlling error only under the global null hypothesis.

Why the Graph Must Be Prespecified

The graph determines which hypotheses can receive additional alpha.

Therefore changing the graph after observing unblinded efficacy results can change the operating characteristics of the procedure.

The following should generally be prespecified:

• The family of hypotheses.
• The definition of each hypothesis.
• The overall alpha level.
• The initial alpha weights.
• The transition weights.
• The order or conditions for testing.
• The statistical test used for each hypothesis.
• The handling of multiple interim or final analyses, if relevant.
• The treatment of missing data and estimands.

A More Complex Worked Graph

Consider four hypotheses:

Suppose the initial weights are:

With \(\alpha=0.05\):

Suppose \(H_1\) is rejected.

Then 60% of its 0.025 alpha goes to \(H_2\):

and 40% goes to \(H_3\):

Therefore the updated levels are:

while \(H_4\) remains at:

The total available alpha remains:

Alpha Allocation Is Not the Same as Sample Size Allocation

A larger alpha allocation does not mean that a hypothesis automatically receives more patients.

The alpha allocation controls the significance level used for the hypothesis within the multiplicity procedure.

Sample size calculations must separately account for:

• Effect size
• Variance
• Event rate
• Power
• Allocation ratio
• Dropout assumptions
• Multiplicity-adjusted testing levels

Thus, a hypothesis receiving alpha 0.025 instead of 0.010 may require a different sample size to achieve the desired power.

Graphical Procedures and Power

Multiplicity adjustment affects power because the available significance level for each hypothesis may be smaller than 0.05.

For a simple one-sided test, the required sample size generally increases as the significance level becomes more stringent, holding the desired power and effect size constant.

Therefore, designing a graphical procedure and designing the sample size should be considered together.

Testing at Different Local Alpha Levels

Suppose three hypotheses have current levels:

Then the decision rule for each hypothesis is based on its current local level.

For example:

The fact that \(H_3\) has a smaller p-value than \(H_1\) does not mean that the same alpha level must be used for both hypotheses.

Their testing levels are determined by the prespecified multiplicity strategy.

What Happens When a Hypothesis Is Not Rejected?

This is one of the most important implementation details.

A graphical procedure must specify what happens to the alpha associated with a hypothesis that is not rejected.

In a simple one-way hierarchy:

failure to reject \(H_1\) prevents the transfer of \(H_1\)'s alpha to \(H_2\).

Other graphical procedures may have more complicated behavior.

Therefore, one should never infer the testing algorithm from the picture alone without examining the accompanying rules.

Graphical Procedures Are Decision Algorithms

A useful mental model is:

Why Graphical Methods Are Attractive in Clinical Trials

Clinical trials rarely have hypotheses that are all equally important.

For example, the primary endpoint may be substantially more important than several supportive endpoints.

A graphical method can encode that clinical hierarchy without requiring all hypotheses to receive identical alpha.

It can also make a complex multiplicity strategy easier to communicate among:

• Biostatisticians
• Clinical scientists
• Regulatory statisticians
• Medical reviewers
• Data-management teams
• Programming teams

Graphical Approaches and Regulatory Communication

In a confirmatory trial, the multiplicity strategy should be understandable before the confirmatory results are known.

A graphical representation can make several important features immediately visible:

• Which hypothesis is prioritized.
• How much alpha each hypothesis receives initially.
• Which hypotheses depend on successful testing of others.
• How alpha is recycled.
• Which secondary objectives can benefit from success on another endpoint.

Common Graphical Procedure Designs

Graphical Procedures Versus Fixed-Sequence Testing

Fixed-sequence testing uses a predetermined order:

The next hypothesis is tested only after the preceding hypothesis succeeds.

Graphical procedures can reproduce this strategy but can also allow more flexible redistribution.

Therefore: fixed-sequence testing can be represented as a particularly simple graphical structure, but graphical procedures are not limited to fixed sequences.

Graphical Procedures Versus Bonferroni

Graphical Procedures Versus Holm

Graphical Procedures and Closed Testing

Closed testing and graphical procedures address the same fundamental problem: how to make multiple confirmatory decisions while maintaining control of the family-wise error rate.

Closed testing is a general theoretical framework.

Graphical procedures provide a practical way to specify many useful testing strategies through alpha weights and transitions.

The two concepts should therefore be understood as complementary rather than competing ideas.

Handling Multiple Primary Endpoints

Suppose a trial has two primary endpoints:

The clinical objective might require:

• Both endpoints to succeed.
• At least one endpoint to succeed.
• One endpoint to succeed before another receives additional alpha.

These are fundamentally different multiplicity questions.

The graphical strategy should therefore be based on the clinical success criterion rather than chosen simply because a particular graph looks convenient.

Intersection Versus Union Success Criteria

Consider two primary hypotheses.

An intersection-style requirement might require:

before the treatment is considered successful on the overall primary objective.

A union-style objective might consider success if either endpoint is demonstrated.

These distinctions affect how alpha should be allocated and how the graph should be constructed.

Example: Two Co-Primary Endpoints

Suppose:

• \(H_1\) tests progression-free survival.
• \(H_2\) tests overall survival.

If both endpoints must be successful for the trial to meet its primary objective, the multiplicity problem differs from a trial in which either endpoint can establish success.

A simple alpha-recycling graph should not be used blindly in such a setting.

The clinical interpretation of success determines the appropriate statistical framework.

Graphical Procedures With Multiple Testing Across Time

Some trials also contain multiple testing over time, such as:

• Interim analyses
• Final analyses
• Multiple endpoints
• Multiple doses

These sources of multiplicity may need to be handled jointly.

A graphical endpoint procedure does not automatically solve every other multiplicity problem in the trial.

Graphical Procedures and Alpha-Spending

Alpha-spending is commonly associated with repeated or sequential interim analyses.

Graphical multiple testing is instead primarily concerned with relationships among hypotheses and the transfer of alpha among them.

These concepts can coexist.

For example, a trial could use an alpha-spending approach for interim analyses and a graphical multiplicity strategy for multiple endpoints.

The combined procedure must preserve the intended overall type I error.

Implementation in Statistical Programming

A graphical procedure should be implemented from a precise numerical specification rather than manually interpreting a figure.

At minimum, programmers need:

• Hypothesis identifiers
• Initial weights
• Transition weights
• Testing order or eligibility rules
• Local alpha calculation rules
• P-values or test statistics
• Decision rules
• Rules for updating alpha after rejection

The implementation should be independently checked against the statistical analysis plan.

Simple Pseudocode

initialize alpha_total = 0.05

set initial alpha:
  H1 = 0.025
  H2 = 0.015
  H3 = 0.010

test H1 at alpha = 0.025

if H1 is rejected:
    transfer H1 alpha to H2
    update H2 alpha

test H2 at updated alpha

if H2 is rejected:
    transfer H2 alpha to H3
    update H3 alpha

test H3 at updated alpha

For a more complicated graph, the update operation becomes a weighted redistribution of the rejected node's available alpha.

R Implementation of the Simple Example

The basic alpha allocation can be represented in R as:

alpha <- 0.05

weights <- c(
  H1 = 0.50,
  H2 = 0.30,
  H3 = 0.20
)

local_alpha <- alpha * weights

local_alpha

The result is:

H1    H2    H3
0.025 0.015 0.010

Implementing a Simple Transition

Suppose \(H_1\) transfers all of its available alpha to \(H_2\).

local_alpha["H2"] <-
  local_alpha["H2"] +
  local_alpha["H1"]

local_alpha["H1"] <- 0

local_alpha

The resulting alpha allocation is:

H1    H2    H3
0.000 0.040 0.010

The total remains 0.05.

sum(local_alpha)
# 0.05

Implementing Partial Transition Weights

Suppose \(H_1\) transfers 70% of its alpha to \(H_2\) and 30% to \(H_3\).

alpha_H1 <- local_alpha["H1"]

local_alpha["H2"] <-
  local_alpha["H2"] +
  alpha_H1 * 0.70

local_alpha["H3"] <-
  local_alpha["H3"] +
  alpha_H1 * 0.30

local_alpha["H1"] <- 0

This implements the graph numerically.

Testing Decisions in R

A simple decision function can compare a p-value with the current local alpha:

reject_hypothesis <- function(p_value, alpha_level) {
  p_value <= alpha_level
}

reject_hypothesis(
  p_value = 0.018,
  alpha_level = 0.040
)

The result is:

TRUE

because 0.018 is below the updated 0.040 significance level.

Representing the Transition Matrix in R

For a three-hypothesis serial graph:

G <- matrix(
  c(
    0, 1, 0,
    0, 0, 1,
    0, 0, 0
  ),
  nrow = 3,
  byrow = TRUE
)

rownames(G) <- c("H1", "H2", "H3")
colnames(G) <- c("H1", "H2", "H3")

G

The rows describe where alpha moves after each corresponding hypothesis is rejected.

A Generic Alpha-Transfer Function

transfer_alpha <- function(alpha,
                           rejected,
                           transition_row) {

  available <- alpha[rejected]

  alpha <- alpha +
    available * transition_row

  alpha[rejected] <- 0

  alpha
}

For example:

alpha <- c(
  H1 = 0.025,
  H2 = 0.015,
  H3 = 0.010
)

transition_H1 <- c(
  H1 = 0,
  H2 = 1,
  H3 = 0
)

alpha <- transfer_alpha(
  alpha,
  rejected = "H1",
  transition_row = transition_H1
)

alpha

The result is:

H1    H2    H3
0.000 0.040 0.010

Validation of the Implementation

Graphical procedures should be validated before they are used for a confirmatory analysis.

Useful checks include:

• Initial weights sum to 1.
• Transition weights satisfy the prespecified constraints.
• Local alpha levels are calculated correctly.
• Transferred alpha is not accidentally duplicated.
• Alpha does not exceed the permitted amount under the specified procedure.
• Each hypothesis is tested at the correct current level.
• Rejection decisions are correctly propagated through the graph.
• Independent programming reproduces the planned decisions.

Common Mistakes

1. Testing every endpoint at 0.05. This ignores the family-wise error problem.
2. Confusing initial alpha with final alpha. A hypothesis may begin with a small local alpha and later receive additional alpha through prespecified transitions.
3. Changing transition weights after seeing results. The graph is part of the prespecified multiplicity strategy.
4. Assuming rejected hypotheses automatically give all their alpha to the next hypothesis. The amount transferred depends on the transition weights.
5. Forgetting that a hypothesis may receive alpha from multiple sources. Its current testing level can reflect several incoming transfers.
6. Assuming a graph automatically guarantees FWER control. The underlying graphical procedure must satisfy the conditions required for valid error control.
7. Ignoring other sources of multiplicity. Multiple endpoints are only one possible source of multiplicity.
8. Failing to align the graph with the clinical success criterion. The statistical procedure should reflect what constitutes success scientifically.
9. Using a graph without numerical transition rules. The SAP should make the procedure reproducible without relying on visual interpretation alone.
10. Failing to validate the programming implementation. Complex alpha recycling is especially susceptible to implementation errors.

How to Design a Graphical Procedure

What Should Be Documented in the SAP?

A graphical multiple-testing procedure should be sufficiently explicit that an independent statistician could reproduce the testing decisions.

The statistical analysis plan should generally include:

• The full list of hypotheses.
• The clinical endpoint corresponding to each hypothesis.
• The estimand associated with each hypothesis.
• The overall family-wise alpha level.
• The initial alpha weights.
• The numerical initial significance levels.
• The transition matrix or equivalent transition rules.
• The graphical representation.
• The testing order or eligibility rules.
• The rule for transferring alpha after rejection.
• The rule for hypotheses that are not rejected.
• The statistical test used for each hypothesis.
• The treatment of missing data and intercurrent events.
• The relationship between the multiplicity procedure and interim analyses.
• The programming implementation and validation approach.

Graphical Approaches and Estimands

Multiplicity should be considered after the clinical questions themselves have been clearly defined.

Each hypothesis should correspond to a clearly specified estimand or statistical question.

For example, the trial should distinguish among:

• The treatment effect of interest.
• The population in which the effect is evaluated.
• The endpoint or variable.
• The handling of intercurrent events.
• The summary measure used for inference.

The graphical procedure then controls multiplicity among the resulting confirmatory hypotheses.

Graphical Approaches Do Not Replace Good Hypothesis Design

A sophisticated graphical procedure cannot rescue poorly chosen hypotheses.

If a trial has too many nominally confirmatory endpoints, assigning them to a graph does not automatically make the clinical strategy scientifically appropriate.

The first questions should be:

• Which outcomes are truly confirmatory?
• Which outcomes are supportive?
• Which outcomes define regulatory success?
• Which outcomes are exploratory?
• What clinical decisions depend on each result?

Only then should the graphical multiplicity strategy be constructed.

A Practical Three-Hypothesis Template

For a common primary-plus-secondary design, a useful starting structure is:

This produces initial local levels of:

If all hypotheses succeed sequentially, the alpha can ultimately accumulate at the later hypothesis.

But the actual weights and transitions should always be selected according to the trial's scientific objectives.

Example of a Parallel Graph

Suppose a primary hypothesis \(H_1\) is tested first, and after rejection its alpha is split between two secondary hypotheses.

Suppose \(H_1\) has 0.025 available alpha.

Then:

This allows both secondary endpoints to become more accessible following success on the primary endpoint.

Sequential Versus Parallel Strategies

Why Graphical Methods Are Especially Useful for Complex Trials

As the number of hypotheses increases, a simple list of adjusted p-values can become difficult to interpret.

A graph can instead answer three questions immediately:

1. Where does alpha start?
2. Where does alpha go after a rejection?
3. Which hypotheses become more powerful after earlier success?

This makes graphical methods particularly useful for complex confirmatory programs.

The Most Important Mathematical Idea

The central mathematical idea is the conservation and redistribution of the alpha budget.

Start with:

When a rejected hypothesis \(i\) transfers its available alpha according to weights \(g_{ij}\), the outgoing transfers satisfy:

provided the outgoing transition weights sum to one.

The receiving hypotheses then accumulate the transferred alpha.

This is the mathematical mechanism underlying the visual graph.

A Compact Worked Example

Suppose:

and initial weights:

giving:

Now suppose the p-values are:

The decisions are:

The graphical procedure therefore permits all three hypotheses to be rejected even though the initial alpha levels of \(H_2\) and \(H_3\) were much smaller than 0.05.

That is the principal practical benefit of alpha recycling: successful earlier hypotheses can unlock additional significance level for later hypotheses.

What the Graph Does Not Mean

A graphical procedure does not mean:

• Every hypothesis eventually receives 0.05.
• A failed hypothesis can simply be skipped.
• Alpha can be moved wherever the analyst wants.
• P-values can be compared directly without regard to their assigned levels.
• The procedure is automatically valid regardless of its graph.

Instead, every alpha transfer must follow the prespecified rules.

Final Practical Checklist

Summary of the Worked Example

The Most Important Concept

The most important idea behind graphical multiple-testing procedures is that alpha is treated as a prespecified resource that can be allocated and redistributed according to the clinical structure of the hypotheses.

The graph specifies:

• Where alpha starts.
• Which hypotheses receive the initial allocation.
• How much alpha is transferred after a rejection.
• Where transferred alpha goes.
• Whether alpha can be recycled.
• Which later hypotheses can benefit from earlier success.

This allows graphical procedures to represent complex confirmatory strategies while maintaining control of the family-wise error rate when the procedure is properly constructed.

References

Bretz, F., Maurer, W., Brannath, W. & Posch, M. (2009). A graphical approach to sequentially rejective multiple test procedures. Statistics in Medicine, 28, 586–604.
Bretz, F., Posch, M., Glimm, E., Klinglmueller, F., Maurer, W. & Rohmeyer, K. (2011). Graphical approaches for multiple comparison procedures using weighted Bonferroni, Simes, and parametric tests. Biometrical Journal, 53, 894–913.
Maurer, W. & Bretz, F. (2013). Memory and graphical approaches for simultaneous inference in multiple testing. Statistics in Medicine.
Goeman, J.J. & Solari, A. (2014). Multiple testing for exploratory research. Statistical Science, 29, 584–597.
Hochberg, Y. & Benjamini, Y. (1990). More powerful procedures for multiple significance testing. Statistics in Medicine, 9, 811–818.
Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6, 65–70.
Bretz, F., Hothorn, T. & Westfall, P. (2010). Multiple Comparisons Using R. Chapman & Hall/CRC.