Why a Hierarchy Needs Two Decisions
A primary endpoint and a secondary endpoint may be measured at the same interim looks, but evidence for them does not automatically have equal confirmatory status. A hierarchy makes the primary endpoint the gatekeeper: the secondary endpoint can support a confirmatory claim only if the primary hypothesis has also been rejected. With interim analyses, this rule must be combined with each endpoint’s sequential rejection rule. The analyst therefore makes two distinct decisions: has the endpoint crossed its sequential boundary, and is the endpoint permitted to support a claim?
The example below uses one primary and one key secondary endpoint, both tested in a prespecified beneficial direction. It focuses on the claim rule and its rejection patterns, not on choosing a clinical endpoint or designing a trial. All data and boundaries are hypothetical.
This rule deliberately distinguishes the secondary endpoint’s statistical evidence from its permission to support a claim. A secondary statistic can cross its boundary while the gate is closed. That crossing is recorded, but it does not produce a secondary claim at that time. If the primary endpoint later crosses its boundary, the prespecified rule permits the secondary claim then, provided the secondary crossing has occurred by that look. If the primary endpoint never crosses, no confirmatory secondary claim is made.
Hypotheses, Statistics, and Boundaries
For endpoint \(j\), where \(j\) is \(P\) or \(S\), write the beneficial-direction effect estimate at look \(k\) as \(\hat{\theta}_{j,k}\), its null value as \(\theta_{j,0}\), and its standard error as \(SE_{j,k}\). The example uses the standardized statistic
Larger positive values support benefit. The one-sided null hypothesis is that the effect is no greater than the null value; the alternative is that it is greater. Each endpoint has a prespecified group-sequential boundary \(b_k\), and its sequential test rejects at or before look \(k\) if \(Z_{j,r}\ge b_r\) for at least one look \(r\le k\).
For transparent arithmetic, use three planned looks and the illustrative boundaries below. Treat these values as already calibrated for each endpoint’s prespecified spending rule and information schedule so that, under that endpoint’s null, the probability of crossing at least one boundary is at most one-sided \(\alpha=0.025\). In an actual design, boundaries must be calibrated to the planned looks, information fractions, spending rule, and statistical model; the numbers here are not universal cutoffs.
| Look | Boundary \(b_k\) | Interpretation |
|---|---|---|
| 1 | 2.80 | Crossing at the first look rejects that endpoint’s sequential test. |
| 2 | 2.30 | If not rejected earlier, crossing now rejects that endpoint’s sequential test. |
| 3 | 2.00 | If not rejected earlier, crossing now rejects that endpoint’s sequential test. |
The assumptions for interpreting these boundaries are important. The analyses occur at the three prespecified looks; the test statistics and their standard errors are computed as planned; the endpoint-specific boundaries control the probability of at least one false rejection across those looks at or below 0.025; and the hierarchy is applied exactly as specified. Dependence between endpoints need not be ignored or assumed absent for the error-control argument below, because that argument uses containment of events rather than independence. If the actual analysis changes the look schedule, statistic, or boundary without valid recalibration, the stated calibration cannot simply be presumed to hold.
Worked Example: The Secondary Crosses Before the Primary
Suppose positive effects favor treatment. At look 1, the primary estimate is 4.2 units with \(SE=2.0\), and the secondary estimate is 5.62 units with \(SE=2.0\). Both null values are zero. At look 2, the corresponding estimates and standard errors are 4.90 and 2.0 for the primary, and 4.80 and 2.0 for the secondary.
At look 1, the statistics are
The primary does not cross its first-look boundary because \(2.10<2.80\). The secondary does cross because \(2.81\ge2.80\), but its gate is closed, so no confirmatory secondary claim is permitted at look 1.
At look 2,
The primary now crosses because \(2.45\ge2.30\). The hierarchy gate opens. The secondary has already crossed its own boundary at look 1, so by look 2 both endpoints meet the stated claim rule. The secondary claim becomes eligible at look 2, not at look 1. This conclusion follows from the prespecified rule; it is not a claim that the secondary’s look-1 result alone established the hierarchy.
Evaluating the Possible Rejection Patterns
For each endpoint, first record whether its statistic crossed any boundary by the current look. Then apply the hierarchy. This produces four basic combinations of endpoint-specific crossing status, although the hierarchy prevents a secondary claim in one of them.
Compare each primary statistic with its boundary and record the first crossing, if any. A crossing makes the primary rejection absorbing: the primary remains rejected at later looks.
Independently compare the secondary statistics with their own boundaries. Record the first crossing, even if the hierarchy gate is not yet open.
Claim the primary if its sequential test has rejected. Claim the secondary only if its sequential test has rejected and the primary has been rejected by that time.
A “crossing” here means crossing the group-sequential boundary, not merely obtaining a single-look p-value below 0.025. The boundary accounts for repeated opportunities to reject across planned looks. A nominal single-look result cannot replace the prespecified sequential rule.
Worked Example: Secondary Evidence Without Primary Rejection
Suppose the primary statistics at the three looks are \(2.10\), \(2.25\), and \(1.98\). The secondary statistics are \(1.70\), \(2.40\), and \(2.15\). Use the boundary sequence \(2.80, 2.30, 2.00\) for each endpoint.
The primary does not cross at look 1 because \(2.10<2.80\), at look 2 because \(2.25<2.30\), or at look 3 because \(1.98<2.00\). Thus the primary has not been rejected by the final look.
The secondary does not cross at look 1, but it crosses at look 2 because \(2.40\ge2.30\). It remains sequentially rejected thereafter. Nevertheless, the primary gate never opens. The decision at the final look is therefore: no primary claim and no confirmatory secondary claim. Reporting the secondary boundary crossing as a confirmatory success would violate the hierarchy.
Worked Example: Primary Rejection Followed by a Final-Look Secondary Crossing
Suppose the primary statistics are \(2.90\), \(2.70\), and \(2.40\), while the secondary statistics are \(1.90\), \(2.00\), and \(2.05\). The boundary sequence remains \(2.80, 2.30, 2.00\).
At look 1, \(Z_{P,1}=2.90\ge2.80\), so the primary is rejected and the gate opens. The secondary does not cross at look 1 because \(1.90<2.80\). It also does not cross at look 2 because \(2.00<2.30\). At look 3, however, \(Z_{S,3}=2.05\ge2.00\), so the secondary sequential test rejects. Since the primary was already rejected, both endpoints support confirmatory claims by look 3.
The secondary claim is not supported at looks 1 or 2, and it is supported at look 3 under this rule. The fact that the primary crossed early does not remove the need for the secondary to meet its own sequential boundary.
Why the Hierarchy Controls Familywise Error
Familywise error is the probability of making at least one false confirmatory rejection in the endpoint family. A short event argument justifies the hierarchy, assuming each endpoint’s sequential test has null crossing probability no greater than \(\alpha=0.025\).
If the secondary null \(H_S\) is true, a false secondary claim requires that the secondary’s own sequential test cross its boundary. The hierarchical claim event is a subset of that crossing event. Therefore,
If \(H_P\) is true, a false primary claim requires a primary boundary crossing, whose probability is at most \(\alpha\). If both null hypotheses are true, a secondary claim also requires primary rejection; consequently, any false claim requires primary boundary crossing. The familywise error in that configuration is at most \(\alpha\). If \(H_P\) is false but \(H_S\) is true, the only possible false claim is a secondary claim, already bounded by \(\alpha\). If \(H_P\) is true but \(H_S\) is false, the only possible false claim is primary rejection, also bounded by \(\alpha\). These cases show strong familywise error control for this two-endpoint hierarchy.
This justification does not require the endpoints to be independent. It does require valid endpoint-specific sequential tests under their null hypotheses and the stated gatekeeping rule. The argument also explains why the procedure does not add the two endpoint alpha levels: each marginal sequential test is calibrated to the familywise level, while the hierarchy restricts which secondary rejections can be claimed.
Diagnostics, Alternatives, and Limits
Before applying a hierarchical decision rule, check that the protocol or analysis plan identifies the endpoint order, direction of testing, look schedule, boundary rule for each endpoint, and the exact condition for opening the gate. Confirm that a reported boundary crossing uses the correct endpoint-specific statistic and boundary. In particular, check that a missed or delayed endpoint result has not been silently treated as though it were available at a planned look.
A simple hierarchy is attractive when the primary endpoint is clinically essential and the secondary claim is intended to depend on primary success. Compared with a fallback or graphical procedure, it is easier to explain but offers less flexible alpha reallocation among hypotheses. A graphical procedure can encode more complex priorities and transfers; it requires careful tracking of local levels and their sequential validity. Whichever approach is selected, interim boundaries and multiplicity rules must be treated as one prespecified testing procedure, not as separate opportunities to spend the same error allowance.
The example also has limits. The illustrative boundary values cannot be copied into a real trial without confirming their calibration. The event argument controls type I error; it does not guarantee adequate power for either endpoint. And a statistically supported claim is not, by itself, a judgment about clinical importance, benefit-risk balance, or the quality of endpoint measurement.
- Mistake: “The secondary crossed its boundary, so it can be claimed even though the primary did not.” Correction: A secondary crossing is necessary but not sufficient; the gate must also have opened.
- Mistake: “The primary crossed, so the secondary is automatically successful.” Correction: The secondary must independently cross its own sequential boundary.
- Mistake: “A nominal p-value below 0.025 at one look proves the sequential test rejected.” Correction: Compare the statistic with its prespecified group-sequential boundary, which accounts for repeated looks.
Check Your Understanding
Use the three-look boundaries \(2.80, 2.30, 2.00\) and the hierarchy defined above. Each question is followed by a concise solution.
- The primary statistics are \(2.20, 2.35, 2.10\); the secondary statistics are \(2.85, 2.20, 1.95\). At which look, if any, can each endpoint be claimed?
- The primary crosses at look 1. The secondary statistic is \(2.10\) at look 1, \(2.28\) at look 2, and \(2.01\) at look 3. Which endpoint claims are supported by the final look?
- Why does dependence between the two endpoint statistics not invalidate the event-containment argument for the secondary null?
- What is wrong with deciding that the secondary is significant because its ordinary single-look p-value is below 0.025, without checking its sequential boundary?
Solutions. (1) The primary first crosses at look 2 because \(2.35\ge2.30\). The secondary crosses at look 1 because \(2.85\ge2.80\), but the gate is not open until look 2. Thus both claims are permitted at look 2. (2) The primary claim is supported from look 1. The secondary does not cross at look 1 or 2, but crosses at look 3 because \(2.01\ge2.00\); both claims are supported by the final look. (3) A false secondary claim is a subset of the event that its own sequential test crosses under \(H_S\). Bounding the probability of that event does not require independence from the primary statistic. (4) The ordinary p-value does not account for repeated looks. The sequential rejection decision must use the calibrated boundary for that look.