Gatekeeping Graphs For Primary And Secondary Endpoints

Why Gatekeeping Matters Across Interim Looks

A trial may have one primary endpoint and several secondary endpoints, each monitored at interim analyses. Investigators may want secondary claims to be possible if the primary objective is met, but not to let a promising secondary result substitute for a failed primary claim. A gatekeeping graph makes that priority explicit: secondary endpoints receive no testing opportunity until the primary endpoint has been rejected under its prespecified sequential test.

The graph governs allocation across endpoints; group sequential boundaries govern when evidence at an endpoint is strong enough during repeated looks. These are related but distinct decisions. An endpoint can have alpha allocated to it and still fail to cross its boundary. Conversely, a secondary endpoint with no alpha allocated to it cannot support a confirmatory claim, even if its observed result looks compelling.

Assume the confirmatory family, endpoint hypotheses, direction of each test, look schedule, graph weights, and transition rules are prespecified. Each endpoint’s group sequential test must be valid at the local alpha assigned to it: under that endpoint’s null, the probability of crossing a rejection boundary over the monitoring plan must be no greater than that local alpha. The method controls familywise error, the probability of at least one false rejection among the endpoint hypotheses, at level \(\alpha\), provided the graph procedure and the sequential tests are valid and their alpha allocations are implemented as specified. No independence between endpoint statistics is required for the usual graphical error-control guarantee.

Building a Primary-to-Secondary Graph

Let \(H_P\) denote the null hypothesis for one primary endpoint, and let \(H_{S1}\) and \(H_{S2}\) denote null hypotheses for two secondary endpoints. Suppose the familywise significance level is \(\alpha=0.025\), using one-sided tests in the prespecified favorable direction. The initial allocation is \(\alpha_P=0.025\), \(\alpha_{S1}=0\), and \(\alpha_{S2}=0\). Give the primary node outgoing transition weights of \(0.60\) to \(S1\) and \(0.40\) to \(S2\). The weights sum to one, so rejection of the primary transfers its full allocation: \(0.015\) to \(S1\) and \(0.010\) to \(S2\).

If \(S1\) is then rejected, a transition from \(S1\) to \(S2\) with weight 1 transfers \(S1\)'s \(0.015\) allocation to \(S2\). The new allocation for \(S2\) is \(0.025\). The graph therefore allows a secondary claim to become testable only after the primary rejection; it can also allow alpha to move between secondary hypotheses after the specified rejection sequence.

Outgoing weights from a node must be nonnegative and sum to at most one. Any unassigned portion is not transferred. A transition redistributes alpha; it does not create alpha. Across the graph, the total available allocation cannot exceed the familywise level. The graph must also state what happens if a hypothesis is not rejected: absent a prespecified transition, its alpha stays where it is and no downstream node receives it.

Combining Graph Transitions With Interim Looks

At each endpoint, the sequential procedure must account for all planned looks at which that endpoint can be tested. A local alpha of \(0.015\), for example, is the total error budget for that endpoint’s valid group sequential test—not \(0.015\) at every look. Its interim boundaries must be calibrated so that the probability of rejecting a true endpoint null over the monitoring plan is at most \(0.015\).

If a graph transition opens a secondary endpoint partway through the trial, the implementation must use the prespecified rules for that transition time and the remaining looks. It cannot simply apply the full \(0.015\) at every future look, nor can it treat a transferred amount as if the secondary endpoint had been open at earlier looks. The sequential procedure must account for the alpha already spent, if any, and calibrate valid boundaries for the eligible looks after the transition. The earlier material on spending alpha after a graph transition addresses this operational calibration.

A useful way to report an analysis is to state, for each endpoint, its allocated alpha, the look at which it became eligible, and whether its valid sequential test crossed a boundary. A nominal p-value at one look is not by itself a substitute for that sequential decision. If results are summarized by sequentially adjusted p-values, they must correspond to the endpoint’s specified sequential procedure and the relevant allocation and transition timing.

Why the Gatekeeping Rule Protects the Family

The error-control argument combines two safeguards. First, each endpoint’s sequential procedure controls its chance of a false rejection at the local alpha assigned through the graph. Second, the graph’s transfers conserve the familywise error budget. For example, when \(H_P\) is rejected, its \(0.025\) is divided into \(0.015\) and \(0.010\), not added on top of those amounts. A later transfer from \(S1\) to \(S2\) likewise moves the \(0.015\); it does not duplicate it.

More generally, the established error-control result for weighted graphical procedures applies when the endpoint tests are valid at the allocated local levels and the transition rules are followed. Sequential monitoring does not remove the need for that condition: each node’s test must control its error across looks, and transitions must be coordinated with the remaining-look boundaries. Under these conditions, the graph controls the probability of one or more false endpoint rejections at no more than the familywise level, including when endpoint statistics are correlated.

The gatekeeping restriction also has a direct interpretive consequence. A secondary result may be described as exploratory if the primary gate is not rejected, but it is not a confirmatory rejection under this graph. A small secondary p-value does not open the gate; only the specified primary rejection does.

Comparison, Diagnostics, and Limitations

A parallel allocation gives multiple endpoints alpha from the outset, so secondary endpoints can be tested even if the primary endpoint fails. That approach can be appropriate when the scientific objectives are genuinely parallel, but it does not enforce the primary-first claim structure. Gatekeeping instead reserves the initial budget for the primary endpoint and releases alpha to secondary endpoints only after the designated rejection. A fixed hierarchy is a stricter alternative: it can require one secondary endpoint to be rejected before the next becomes testable, whereas a weighted graph can open several secondary nodes at once using prespecified shares.

Several checks help identify implementation failures:

• Check the initial allocations. Secondary nodes intended to be gated must not receive positive alpha before the primary rejection.
• Check every transition. Confirm that the edge weights and the rule for untransferred alpha match the protocol; do not silently change weights after seeing results.
• Check sequential calibration. Local alpha is a budget across looks, not a per-look threshold. Verify that the boundaries match the endpoint’s allocation and the timing of its eligibility.
• Check the claim set. Distinguish confirmatory rejections allowed by the graph from descriptive or exploratory findings outside it.

A common mistake is to say, “The primary endpoint missed its boundary, but the secondary p-value is below \(0.05\), so the secondary claim is significant.” This reasoning ignores both the closed gate and the familywise allocation. The correction is to follow the prespecified graph: without primary rejection, the secondary endpoint receives no confirmatory alpha, so its nominal result cannot be used as a rejection under this design.

Gatekeeping graphs do not repair an invalid endpoint test, an unplanned change in the look schedule, or an incorrectly implemented transition. They also do not make a secondary endpoint clinically meaningful merely because it becomes statistically testable. The graph specifies the order and allocation of statistical claims; scientific interpretation still requires the endpoint’s clinical context, measurement quality, and effect estimate.