Hierarchical Endpoints With Interim Monitoring

Why Combine an Endpoint Hierarchy With Interim Monitoring?

A confirmatory study may have several clinically important endpoints but a clear order of priority: for example, a primary endpoint followed by two secondary endpoints. A prespecified hierarchy allows the study to make a claim about a later endpoint only if the claims ahead of it have succeeded. Interim monitoring adds another dimension: each endpoint may be examined at more than one planned look. The design must respect both the endpoint order and the repeated opportunities to reject a null hypothesis.

The procedure here is a fixed sequence: hypotheses are tested in a prespecified order, and the sequence stops for confirmatory claims at the first hypothesis that is not rejected. Each endpoint uses a group sequential test whose error rate is controlled across that endpoint’s planned looks. The hierarchy then controls error across endpoints. It is not a procedure for reallocating alpha among endpoints after a rejection.

Endpoint Tests Across Planned Looks

Suppose endpoint \(j\) is assessed at planned looks \(k=1,\ldots,K_j\). For illustration, take a one-sided benefit hypothesis \(H_j:\theta_j\leq 0\) against \(H_{aj}:\theta_j>0\), where \(\theta_j\) is the treatment effect on endpoint \(j\), measured in that endpoint’s units. At look \(k\), let \(\hat{\theta}_{j,k}\) be its estimated treatment effect and \(SE_{j,k}\) its standard error. A standardized statistic is \(Z_{j,k}=\hat{\theta}_{j,k}/SE_{j,k}\), with larger values favoring benefit under this orientation.

The boundaries for an endpoint must be chosen as a group sequential design: under its null, the probability of crossing any of that endpoint’s boundaries over the full planned monitoring schedule must not exceed its local significance level. A boundary-crossing rule can be written as follows.

Here \(R_j\) is the event that endpoint \(j\)'s group sequential test rejects its null, and \(b_{j,k}\) is the boundary at look \(k\). The supremum means that the crossing-probability guarantee holds for every parameter value in \(H_j\). The boundary sequence may be calibrated using the joint distribution of the statistics across looks, or conservatively using a valid bound on the probability of any crossing. Either way, the guarantee is for the entire sequence of looks—not for each look separately.

This statement relies on suitable endpoint-specific statistical assumptions. The endpoint, estimand, beneficial direction, analysis method, and look schedule must be prespecified; the statistic and its null distribution (or calibration method) must be valid under the endpoint null; and the monitoring schedule must be the one used to calibrate the boundaries. For example, a normal-statistic design needs its stated normal or asymptotic approximation to be adequate. Unplanned looks, changing an endpoint’s definition after seeing results, or using an invalid standard error can break the local guarantee.

Why the Hierarchy Controls Familywise Error

The familywise error rate (FWER) is the probability of making at least one false rejection among the endpoint null hypotheses. The key fact is that the first true null in the hierarchy must be rejected before any later true null can be reached. Its own group sequential crossing event therefore bounds the chance of any false confirmatory claim.

To see this, consider any configuration of true and false null hypotheses. If all nulls are false, there is no false rejection. Otherwise, let \(r\) be the index of the first true null. Every index before \(r\) is false. A false confirmatory rejection can occur only if the procedure reaches and rejects \(H_r\): if \(H_r\) is not rejected, the hierarchy blocks every later claim, and earlier hypotheses are false. Thus the event of at least one false rejection is contained in \(R_r\). The local group sequential guarantee gives the result.

This is a strong-control argument because it applies to every pattern of true and false nulls. It does not require endpoint statistics to be independent. Nor does it require a conditional type I error guarantee given that earlier endpoints passed: the event-containment argument uses the unconditional crossing probability of the first true null’s test. This distinction matters when endpoints are correlated or when success on an earlier endpoint changes which claims are eligible.

Worked Decisions With a Two-Look Design

The examples use the same simple, hypothetical two-look boundary rule for each endpoint: reject at look 1 if \(Z_{j,1}\geq 2.90\), or at look 2 if \(Z_{j,2}\geq 2.10\). Suppose the design’s null statistic is standard normal at each look. The rule controls each endpoint’s crossing probability conservatively at no more than \(0.025\), since the union bound gives

This calculation does not assume independence between looks. The example uses a conservative rule to make the error guarantee transparent; a practical design may use a more efficient boundary calibrated to the joint distribution across looks.

Implementation, Diagnostics, and Alternatives

A reliable implementation makes the sequence and the monitoring rule operational before enrollment. At every planned look, the team should know which endpoint tests are running, which claims are eligible, what boundaries apply, and what happens if a gate has not opened. One transparent rule is to assess each endpoint at its prespecified looks using its original full-schedule boundary sequence, but to release a claim only at a look when all preceding claims have already been established. A boundary crossing while an endpoint is ineligible does not itself authorize a claim; the protocol should specify how eligibility and crossing are handled.

Diagnostics should check that the implemented boundary-crossing probability matches the design, that the data cutoff and information times correspond to the planned looks, and that the endpoint order has not changed in response to interim results. Review the analysis code for accidental per-look use of \(\alpha\) instead of a boundary calibrated across looks. Also check that the intended test statistic is valid for the endpoint’s estimand and that missing outcomes, treatment deviations, or data-quality problems have not undermined its assumptions.

A strict hierarchy is most useful when the scientific claims naturally have a priority order and the study can accept that a failure early in the sequence blocks later confirmatory claims. In contrast, a parallel strategy can allow several endpoint claims to be pursued without requiring every earlier claim to pass; it needs a multiplicity procedure suited to that different goal. The fixed sequence does not make a failed endpoint’s result disappear, and it does not show that a later treatment effect is absent. It limits which later findings can be presented as confirmatory under the specified familywise error guarantee.

Common Mistakes and Limitations

• Using a per-look threshold as if it were a sequential boundary. A level-\(\alpha\) test repeated at every look can have a crossing probability above \(\alpha\). Calibrate the whole look sequence or use a valid conservative bound.
• Letting a later endpoint leapfrog the hierarchy. A very large statistic on a blocked endpoint does not create a confirmatory claim. The order is part of the procedure, not a ranking reconsidered after results arrive.
• Assuming correlation invalidates the fixed-sequence proof. Correlation may affect the endpoint test calibration, but the FWER argument itself needs only the marginal full-look guarantee for the first true null.
• Calling a nonrejection proof of no effect. Failure to cross a boundary means the specified test did not reject at the planned looks; it is not proof that the null is true or that a clinically important effect is impossible.
• Changing the schedule or estimand after observing results. The original local crossing guarantee need not apply after unplanned looks, altered endpoint definitions, or uncalibrated analyses.

The main limitation is intentional: a strict hierarchy may sacrifice the ability to make later confirmatory claims when an earlier endpoint narrowly misses its boundary. It also depends on a defensible clinical ordering. The design is not a substitute for appropriate endpoint measurement, valid estimation, or transparent reporting of all results.