Why Closure Is Needed Across Endpoints and Looks
A group sequential test accounts for the repeated opportunities to reject a hypothesis at interim and final looks. With several endpoints, however, a false claim might arise on any endpoint at any look. Controlling the sequential error rate separately for each endpoint does not, by itself, control the probability of at least one false endpoint claim across the family.
The closure principle provides a general route to strong familywise error control. It requires tests not only of the individual endpoint hypotheses, but also of every intersection of them. The key is to make each intersection test valid over the full sequential monitoring process. Then, whenever any true endpoint null is rejected by the closed procedure, the intersection of all true endpoint nulls must also have been rejected.
For example, with two endpoints the procedure tests \(H_1\), \(H_2\), and \(H_{12}=H_1\cap H_2\). It may reject \(H_1\) only if both \(H_1\) and \(H_{12}\) are rejected. A test of the joint intersection is not an optional extra: it is part of the rule that supports a claim about either individual endpoint.
Sequential Validity of Each Intersection Test
Let \(\alpha\) be the desired familywise error rate. Assign a local significance level \(\alpha_I\) to each nonempty intersection \(H_I\), with \(\alpha_I\leq\alpha\). The local test must account for all the interim looks at which that intersection can be rejected. Write \(R_{I,k}\) for the event that the test rejects \(H_I\) at look \(k\), and \(R_I=\bigcup_{k=1}^{K}R_{I,k}\) for rejection at any of the \(K\) looks.
This condition is about the probability of rejection over the entire monitoring process, not the probability at one selected look. The group sequential test must use boundaries or another valid sequential procedure for the specified monitoring plan. If the look schedule or analysis decisions are adapted in a way not covered by that procedure, the displayed bound need not hold. Validity must also hold under the full intersection null, including any nuisance parameter values allowed by that null.
No independence assumption between endpoints is required for the closure argument. The test statistics may be strongly correlated, and their dependence may be complicated. The proof uses only the validity bound for the intersection that is true.
Why the Closed Procedure Controls Familywise Error
Strong familywise error control means that the probability of making at least one false endpoint rejection is at most \(\alpha\), for every configuration of true and false elementary null hypotheses. Let \(T\) denote the indices of the true elementary nulls at a particular parameter value. If \(T\) is empty, there are no true nulls to reject, so the probability of a false rejection is zero. Suppose \(T\) is nonempty.
If the closed procedure falsely rejects any \(H_i\) with \(i\in T\), its rule requires rejection of every intersection containing \(H_i\). In particular, it requires rejection of \(H_T=\bigcap_{i\in T}H_i\). Thus the event of at least one false endpoint rejection is contained in the event of rejecting \(H_T\). Since every hypothesis indexed by \(T\) is true, \(H_T\) is true at the parameter value under consideration.
The second inequality follows from the assumed sequential validity of the test of \(H_T\). Because the argument works for every possible nonempty set \(T\), the bound is strong: it holds whether all endpoint nulls are true, none are true, or only some are true. This reasoning does not require a separate probability calculation for every combination of endpoint test statistics.
Worked Examples
Worked Example: Two Endpoints and a Joint Intersection
Suppose a hypothetical trial monitors two endpoint null hypotheses, \(H_1\) and \(H_2\), at several prespecified looks. Each of the three intersection tests, \(H_1\), \(H_2\), and \(H_{12}\), has been calibrated as a valid group sequential test at \(\alpha_I=0.025\). At the end of monitoring, their sequential adjusted p-values are, respectively, \(0.012\), \(0.028\), and \(0.019\). For this example, reject an intersection when its sequential adjusted p-value is at most \(0.025\).
The test of \(H_1\) rejects because \(0.012\leq0.025\). The test of \(H_2\) does not reject because \(0.028>0.025\). The joint intersection test rejects because \(0.019\leq0.025\).
The closure rule therefore permits rejection of \(H_1\): both required tests, \(H_1\) and \(H_{12}\), reject. It does not permit rejection of \(H_2\), because \(H_2\) itself fails its test. The conclusion is that the procedure rejects \(H_1\), but not \(H_2\). Since every intersection test controls its probability of rejection over all looks at \(0.025\), closure gives familywise error control at no more than \(0.025\), under any configuration of true endpoint nulls.
Worked Example: Three Endpoints With Different Local Levels
Consider three hypothetical endpoint nulls. Assign level \(0.05\) to each singleton intersection, \(0.04\) to each pairwise intersection, and \(0.03\) to the three-way intersection. Each is assumed to have a valid group sequential test at its assigned level. The sequential adjusted p-values are:
| Intersection | Assigned level | Sequential adjusted p-value |
|---|---|---|
| \(H_1\) | 0.05 | 0.010 |
| \(H_2\) | 0.05 | 0.020 |
| \(H_3\) | 0.05 | 0.010 |
| \(H_{12}\) | 0.04 | 0.025 |
| \(H_{13}\) | 0.04 | 0.018 |
| \(H_{23}\) | 0.04 | 0.041 |
| \(H_{123}\) | 0.03 | 0.027 |
Each displayed value is compared with the assigned level for its own intersection. For \(H_1\), all required comparisons pass: \(0.010\leq0.05\), \(0.025\leq0.04\), \(0.018\leq0.04\), and \(0.027\leq0.03\). Thus \(H_1\) is rejected by closure.
For \(H_2\), the singleton, \(H_{12}\), and \(H_{123}\) tests pass, but the required \(H_{23}\) test fails because \(0.041>0.04\). Hence \(H_2\) is not rejected. The same failed \(H_{23}\) test is required to reject \(H_3\), so \(H_3\) is not rejected either. The procedure rejects only \(H_1\). All assigned levels are at most \(0.05\), so the theorem gives strong familywise error control at \(0.05\), provided the tests are valid across the looks as assumed.
Worked Example: Bounding Error When Two Endpoint Nulls Are True
In a hypothetical three-endpoint trial, suppose the true null hypotheses at the parameter value of interest are \(H_2\) and \(H_3\), while \(H_1\) is false. Thus \(T=\{2,3\}\), and \(H_T=H_{23}\) is true. Assume the group sequential test of \(H_{23}\) is valid at its assigned level \(\alpha_{23}=0.015\). The other intersection tests have valid assigned levels no larger than the familywise target \(\alpha=0.05\).
A false endpoint claim in this configuration can only be a rejection of \(H_2\) or \(H_3\). Under closure, either rejection requires rejection of \(H_{23}\). Therefore, $$ P_\theta(\text{at least one false rejection}) \leq P_\theta(R_{23}) \leq 0.015. $$
This bound covers the chance of rejecting either true endpoint null at any monitored look; it is not a separate \(0.015\) allowance for each endpoint. Rejection of \(H_1\) would not be a type I error in this configuration because \(H_1\) is false. The example shows why the test of the intersection of the true nulls is the crucial bound in the proof.
Diagnostics, Alternatives, and Limitations
The closure theorem is simple, but its conditions are substantive. Before relying on it, verify that each intersection has a specified test and that its sequential validity applies to the null parameter space for that intersection. Check especially that the claimed level accounts for the complete set of looks. A nominal level used separately at every look is not automatically a valid group sequential level: repeated testing can make the probability of at least one rejection exceed that level.
Also check that the rejection rule actually enforces closure. A favorable singleton test alone is not enough; every intersection containing that endpoint must also reject. Similarly, the levels must obey \(\alpha_I\leq\alpha\). If a true intersection is tested at a level above the familywise target, the proof only yields a bound above that target.
Compared with allocating error directly among separate endpoint tests, closure makes the required logical safeguards explicit and can use different tests or levels for different intersections. Its main cost is scale: \(m\) elementary hypotheses have \(2^m-1\) nonempty intersections. For a modest number of endpoints this may be manageable; for many endpoints, testing every intersection can be computationally burdensome. Any shortcut must preserve the logic that a rejection of a true elementary null entails rejection of the intersection of all true nulls.
Closure does not repair a misspecified sequential test, an unaccounted-for look, or a test whose level is invalid for some parameter values in its intersection null. Nor does it by itself prescribe how to construct the intersection hypotheses or how to calibrate each local group sequential test. Those are separate design and implementation tasks. The result here is conditional: if every intersection test has the stated validity, the closed rule controls the familywise error rate.
Check Your Understanding
Answer each question using the closure rule and the sequential validity condition.
- With two endpoints and a target \(\alpha=0.05\), sequential adjusted p-values for \(H_1,H_2,H_{12}\) are \(0.020, 0.030, 0.060\), respectively. At level \(0.05\), which elementary hypotheses can closure reject?
- If all local sequential tests are valid but one intersection is assigned level \(0.07\) while the target is \(0.05\), does the stated theorem guarantee control at \(0.05\)? Explain.
- Does the closure proof require independence of endpoint test statistics? Explain what probability bound it uses instead.
- If the true elementary null set is \(T=\{1,3\}\), name the intersection test that bounds the probability of at least one false rejection and state why.
Solutions. (1) Neither endpoint can be rejected. Although both singleton tests pass, \(H_{12}\) fails because \(0.060>0.05\), and rejection of either endpoint requires rejection of that intersection. (2) No. The proof bounds the familywise error by \(\alpha_T\); if that level can be \(0.07\), it does not establish a bound of \(0.05\). (3) No independence is required. The event of any false elementary rejection is contained in rejection of the true intersection \(H_T\), whose probability is at most its valid assigned level. (4) It is \(H_{13}\). Both true nulls belong to this intersection, and rejecting either one under closure requires rejecting \(H_{13}\), so its sequential rejection probability bounds the probability of any false rejection.