Why Ask Whether a Graph Is a Closed-Testing Shortcut?
A multiplicity graph records how local significance levels are assigned to endpoint hypotheses and how alpha can be transferred after a rejection. Its appeal is practical: the investigator can follow a sequence of endpoint tests rather than separately evaluating every intersection hypothesis. But a graph is not, by itself, a proof that all the intersection hypotheses required by closed testing have been rejected. That claim depends on the tests represented by the graph, including how each test handles interim looks.
The distinction matters in a group sequential design. An endpoint’s local test must control its own type I error across the planned looks at the alpha level assigned to it. A graph update reallocates that endpoint-level alpha after a rejection; it does not make an invalid sequence of interim tests valid. Nor does a valid graph automatically implement every possible, potentially more powerful, closed-testing procedure.
Here, “valid level \(\alpha\)” means that under any parameter configuration satisfying \(H_I\), the probability of rejecting \(H_I\) is at most \(\alpha\). In a group sequential design, that probability is evaluated over the full planned monitoring procedure, not separately at each look. The closed-testing principle gives strong familywise error rate control: the probability of rejecting at least one true elementary hypothesis is at most \(\alpha\).
What a Graph Specifies—and What It Does Not
A common graph starts with weights \(w_1,\ldots,w_m\), where \(w_i\geq 0\) and \(\sum_i w_i\leq 1\). At familywise level \(\alpha\), endpoint \(i\) initially receives local level \(\alpha_i=\alpha w_i\). Directed edge weights \(g_{ij}\) specify the fraction of alpha at node \(i\) transferred to node \(j\) if \(H_i\) is rejected. The edge weights are nonnegative and each row satisfies \(\sum_{j\ne i}g_{ij}\leq 1\); any untransferred amount is not allocated to another endpoint.
A graph-based step-down rule tests each active endpoint with its current local group sequential test. After rejecting a node, it updates the allocations using the prespecified transition rules and can test again. For example, if all alpha at \(i\) goes to \(j\), then \(\alpha_j\) increases by \(\alpha_i\) when \(H_i\) is rejected. With more than one recipient, the increase to \(j\) is \(\alpha_i g_{ij}\).
These weights and transitions describe a testing algorithm; they do not uniquely specify the local tests for all intersection hypotheses. To call that algorithm a closed-testing shortcut, one needs a set of valid local intersection tests and a justification that the graph’s sequential decisions certify the closure decisions. In particular, the local tests must be compatible with the graph’s allocations and update rules. A graphical procedure may control the familywise error rate by a direct argument without being equivalent to a chosen, more powerful closure.
How the Closure Argument Establishes Strong Control
The key justification is short but important. Suppose the true elementary hypotheses form a nonempty set \(T\). The event of making at least one false rejection is contained in the event of rejecting the intersection of all true hypotheses, \(H_T\): if a true \(H_i\) is rejected, closure requires rejecting \(H_T\), since \(i\in T\). Therefore,
The final inequality follows because the local test of \(H_T\) is valid at level \(\alpha\). This argument does not require independence among endpoints. It does require that the local test of \(H_T\) really has the stated size under the joint null configuration in \(T\), including the repeated looks used to reach a decision.
A graphical shortcut uses less computation when it can infer the necessary intersection decisions from a sequence of local endpoint tests and alpha updates. In a weighted Bonferroni construction, for instance, an intersection test may reject when at least one member’s valid endpoint-level p-value is below its allocated local threshold. A graph can then track which such threshold applies after specified rejections. For a group sequential design, the p-values or boundary decisions in that construction must already account for the monitoring schedule.
Worked Example: Checking Closure for Two Endpoints
Consider two endpoints with elementary hypotheses \(H_1\) and \(H_2\), familywise level \(\alpha=0.05\), and hypothetical group-sequential p-values \(p_1=0.018\) and \(p_2=0.041\). Assume these p-values are valid for the full monitoring plans, so they already account for all planned interim looks. Define local tests as follows: test each singleton \(H_i\) at 0.05, and test \(H_{\{1,2\}}\) by weighted Bonferroni with weights 0.5 and 0.5. Thus the intersection is rejected if \(p_1\leq 0.025\) or \(p_2\leq 0.025\).
For \(H_1\), the singleton test rejects because \(0.018\leq 0.05\). The intersection test also rejects because \(0.018\leq 0.025\). Both closure conditions hold, so \(H_1\) is rejected.
For \(H_2\), the singleton test rejects because \(0.041\leq 0.05\). The intersection test rejects because \(p_1=0.018\leq 0.025\), even though \(p_2\) itself exceeds 0.025. Both closure conditions for \(H_2\) therefore hold as well. This illustrates a feature of closure: an intersection can be rejected because evidence against one member rejects the intersection, allowing closure to reject another member if that member’s other required tests also pass.
The conclusion is valid only under the stated local-test definitions and p-value validity. The numeric thresholds alone do not establish that a different graph, with different transitions or local tests, implements this closure.
When the Graphical Shortcut Is Justified
Before treating graph calculations as a substitute for explicit closure, check the following conditions. They are a practical way to make the claimed equivalence auditable.
List the elementary endpoint hypotheses and the intersection hypotheses whose tests define the closure. State what local test is used for each intersection, or give a precise rule that determines it.
For each local test, verify type I error control under its intersection null over the complete interim schedule. An endpoint test that is valid at one final analysis is not automatically valid when repeatedly applied at interim looks.
Check that initial allocations and transition weights are prespecified, nonnegative, and do not allocate more than the available alpha. Confirm that each update in the algorithm matches the update assumed by the local intersection tests.
For every rejection the graph can make, show that the corresponding elementary hypothesis passes every local intersection test required by closure. If this is not established, describe the graph as a separate FWER-controlling procedure only if its own error-control proof is available.
The last condition is the defining shortcut check. It guards against an easy but invalid inference: “the graph has a total alpha budget of 0.05, so every rejection it produces must be a closed-testing rejection.” Total budget bookkeeping is necessary for many graphical procedures, but it does not, by itself, show which intersection tests have been passed.
Worked Example: A Graphical Shortcut With Complete Alpha Recycling
Suppose two endpoints are tested at familywise level \(\alpha=0.05\). The graph starts with \(w_1=0.6\) and \(w_2=0.4\), so the initial local levels are \(\alpha_1=0.05(0.6)=0.030\) and \(\alpha_2=0.05(0.4)=0.020\). If either endpoint is rejected, all its alpha transfers to the other endpoint. Assume the endpoint group sequential tests are valid at their assigned local levels across all planned looks. For the closure, use singleton tests at 0.05 and a weighted Bonferroni test of \(H_{\{1,2\}}\), rejecting it if \(p_1\leq0.030\) or \(p_2\leq0.020\).
Suppose the hypothetical monitoring results give \(p_1=0.028\) and \(p_2=0.035\). Initially, endpoint 1 is rejected because \(0.028\leq0.030\), while endpoint 2 is not rejected at its initial level because \(0.035>0.020\). The graph transfers endpoint 1’s 0.030 to endpoint 2, giving endpoint 2 a current level of \(0.020+0.030=0.050\). Endpoint 2 is then rejected because \(0.035\leq0.050\).
Now check the closure conditions. The singleton tests pass because \(0.028\leq0.050\) and \(0.035\leq0.050\). The intersection test passes because \(p_1=0.028\leq0.030\). Thus both elementary hypotheses satisfy all their required closure tests. Here the graph sequence and the specified closure agree, given the stated local tests and the validity of their group sequential calibration.
The example does not establish a universal equivalence for all graphs or local tests. It shows what the argument must verify: the updated endpoint decision is supported by the required intersection test, not just by the fact that alpha was transferred.
When the Shortcut Does Not Apply
A common failure is to apply a graph’s nominal local level at every look. Suppose a local endpoint test has level 0.025, but the analyst tests at 0.025 at each of two independent interim looks under the null and rejects if either look crosses the threshold. The probability of at least one crossing is
This is approximately 0.0494, not 0.025. In this hypothetical case, the endpoint test is not a valid level-0.025 local test over the monitoring plan. A graph that assigns that endpoint 0.025 has not repaired the problem, and a closure argument using that local test cannot establish the claimed error control. The local test needs a valid group sequential calibration at its assigned level.
Other warning signs include changing transition weights after seeing unplanned results, transferring alpha after a non-rejection without a prespecified rule, or using an intersection test whose validity has not been established under its intersection null. A shortcut also fails as a claim of equivalence if the graph’s rejection rule does not imply that all required intersection tests reject. In these cases, one must use a separately justified procedure or explicitly test the closure; the graph diagram alone is not enough.
Worked Example: Spotting an Invalid Interim Shortcut
A hypothetical three-look endpoint test is allocated local level 0.020. An analyst proposes testing the endpoint at an ordinary p-value threshold of 0.020 at each look and stopping for rejection at the first crossing. Assume, only for this illustration, that the three look-specific p-values are independent and Uniform(0,1) under the null. The probability of at least one false rejection is
The proposed endpoint test has null rejection probability about 0.0588, exceeding its allocated level of 0.020. Consequently, it cannot serve as the valid level-0.020 local test needed by the graph or its closure. The independence assumption makes this a transparent calculation, not a general model for interim statistics; the diagnosis is simply that the stated local test fails its size requirement. The remedy is to use a group sequential test calibrated to level 0.020 over the specified looks, then verify that the graph and intersection tests use that calibration consistently.
How the Shortcut Compares With Alternatives
| Approach | What it does | What must be checked |
|---|---|---|
| Explicit closed testing | Tests every required intersection and rejects an endpoint only when all containing intersections are rejected. | Every local intersection test must be valid at its specified level over the monitoring plan. |
| Graphical shortcut to a closure | Uses graph updates to avoid evaluating closure decisions separately when sequential rejections certify them. | Local tests, transitions, and the rejection implication must match the specified closure. |
| Standalone graphical procedure | Applies prespecified local tests and alpha transfers with a separate justification for strong error control. | The procedure’s own error-control argument must apply; equivalence to a particular closure should not be presumed. |
A shortcut can reduce bookkeeping, but it does not make the underlying error-control problem disappear. Explicit closure may be easier to audit when the graph-to-intersection correspondence is difficult to establish. Conversely, a well-defined shortcut can be operationally convenient when its local tests and transitions have already been matched and justified. Neither representation automatically dominates every alternative in power; the comparison depends on the local tests and the evidence used by the procedure.
Common Mistakes and Limitations
- Confusing alpha conservation with a closure proof. A graph that never allocates more than the familywise budget still needs valid local tests and a rejection implication for the claimed closure.
- Treating each look as a fresh test at the same level. The local test must control error over the full monitoring plan; repeated nominal thresholds can exceed the assigned level.
- Assuming every graph is equivalent to full closure. A graph may be a valid FWER procedure without being identical to a particular closed procedure. State which claim has actually been justified.
- Ignoring the intersection null. A local test must be valid when all hypotheses in that intersection are true, regardless of the truth values of hypotheses outside it.
The approach also has limits. A graph may be less computationally demanding than explicit closure but can be difficult to verify when endpoints have different monitoring schedules or local tests. The validity argument must still reflect the actual analysis plan and the assumptions underlying each local test. The graph cannot substitute for those details.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- In a closed-testing procedure, what must be true before an elementary hypothesis \(H_i\) can be rejected?
- A graph allocates local level 0.01 to an endpoint and uses threshold 0.01 at each of two independent looks. What is the probability of at least one false rejection under the null?
- Why does a graph’s total alpha budget being at most 0.05 not, by itself, prove that the graph implements a specified closure?
- What is the role of a valid local test of the intersection of all true hypotheses in the strong familywise error argument?
Solutions. (1) Every intersection hypothesis containing \(H_i\) must be rejected by its specified local test. (2) The probability is \(1-(1-0.01)^2=0.0199\), which exceeds the allocated local level 0.01, so the repeated-threshold test is not a valid level-0.01 local test. (3) The budget does not establish that the local tests are valid over the interim schedule or that each graph rejection implies rejection of every required intersection. (4) Any false elementary rejection entails rejection of the intersection of all true hypotheses under closure; if that intersection test has size at most \(\alpha\), the probability of any false rejection is also at most \(\alpha\).