Why Endpoint-Specific Statistics Need Endpoint-Specific Boundaries
A clinical trial may monitor several outcomes on different scales. A treatment effect measured in millimetres of blood pressure change is not numerically comparable with a difference in response proportions. Even after both are standardized, their sampling distributions and information patterns may differ. A group sequential design therefore needs two linked choices for each endpoint: a statistic that is valid for that endpoint at each look, and a boundary calibrated to the endpoint’s local type I error allocation over all its looks.
The aim is not to make unlike outcomes share a raw cutoff. It is to make each endpoint’s sequential test valid at its assigned local error level. The introductory material on multiplicity graphs supplies the logic for allocating local alpha; here the focus is on how the test statistic and boundary must match.
For a one-sided efficacy test, the rejection rule is \(Z_{jk}\ge b_{jk}\) at any look \(k\). For a two-sided test, it is typically \(|Z_{jk}|\ge b_{jk}\), with the boundary calibrated for the total two-sided local alpha. The statistic’s sign must be defined consistently with the clinical direction of benefit. A large statistic is not evidence for the intended alternative if the contrast has been coded in the opposite direction.
Models and Calibration
A common working model for a group sequential statistic is that, under the null, its standardized values are approximately standard normal and have the canonical joint distribution across looks. In particular, if information increases from \(I_{j1}\) through \(I_{jK_j}\), the standardized statistics have approximately unit variance and
This structure follows when the unstandardized score accumulates independent increments with variance equal to the increase in information. It is an approximation for many practical statistics, not an automatic property of every statistic or every analysis. The analysis plan must justify the null distribution and information measure at each look. For example, a known-variance normal contrast has a direct normal standardization; a binary-response score statistic is generally justified by a large-sample approximation. Small samples, changing variance estimates, informative missingness, or non-independent increments can make a nominally standardized statistic depart from the assumed process.
A simple conservative construction is the Bonferroni boundary. With \(K_j\) looks and equal allocation across them, choose \(b_j=\Phi^{-1}(1-\alpha_j/K_j)\) for a one-sided test, where \(\Phi\) is the standard normal cumulative distribution function. For a symmetric two-sided test, use \(b_j=\Phi^{-1}(1-\alpha_j/(2K_j))\). These boundaries do not exploit correlation between looks, but they control the probability of any false crossing under arbitrary dependence, provided each marginal null statistic has the stated standard normal distribution.
Worked Example: Mean Contrast and Binary Score at Two Looks
Worked Example: Mean Contrast and Binary Score at Two Looks
Suppose a hypothetical two-endpoint trial has two efficacy looks. A multiplicity procedure assigns local alpha \(0.04\) to a continuous endpoint and \(0.01\) to a binary endpoint. The two local allocations sum to \(0.05\). For this example, use equal Bonferroni allocation over the two looks, with one-sided tests. The continuous outcome is analyzed as a difference in treatment and control means with known common standard deviation \(10\) units; the binary outcome is analyzed with a two-sample score statistic. Assume independent observations within and between arms, the stated normal model for the continuous outcome, and sufficiently large binary samples for the score’s normal approximation.
At the first look, each arm has 100 continuous-outcome observations. The treatment mean is 103 units and the control mean is 100 units. The standard error and statistic are
The local boundary for the continuous endpoint is \(\Phi^{-1}(1-0.04/2)=\Phi^{-1}(0.98)=2.0537\). Since \(2.1213>2.0537\), the continuous endpoint crosses its local efficacy boundary at the first look. The calculation uses the unrounded standard error in the statistic; dividing by the displayed rounded value \(1.4142\) gives the same result to four significant figures.
For the binary endpoint at the first look, suppose 38 of 100 treatment participants and 26 of 100 control participants respond. Under the null of equal response probabilities, the pooled estimate is \(\hat p=(38+26)/200=0.32\). The score statistic for treatment minus control is
The binary endpoint’s local boundary is \(\Phi^{-1}(1-0.01/2)=\Phi^{-1}(0.995)=2.5758\). Thus \(1.819<2.5758\), so the binary endpoint does not cross its local boundary. The larger threshold reflects the smaller alpha allocation, not a different clinical unit.
At the second look, suppose the cumulative binary counts are 72 responses among 200 treatment participants and 54 among 200 controls. Then \(\hat p=126/400=0.315\), and
This remains below 2.5758, so the binary null is not rejected at either look. These conclusions concern the specified local tests; they do not imply that the two endpoints have equal clinical importance or that one endpoint’s result changes the other endpoint’s statistic.
Why the Local Error Bound Works
The Bonferroni construction follows directly from the probability of a union of events. For a one-sided endpoint test with the same boundary \(b_j\) at each of \(K_j\) looks,
Setting \(b_j=\Phi^{-1}(1-\alpha_j/K_j)\) makes the right-hand side equal to \(\alpha_j\). This argument does not require independence between looks. For multiple endpoints, if the local allocations for the true null hypotheses sum to no more than the overall familywise level \(\alpha\), another union bound gives
where \(\mathcal{T}\) is the set of true null hypotheses. The conclusion holds when every local test is valid at its assigned level. This is why a valid endpoint-specific sequential test is essential: an allocation alone cannot repair a miscalibrated statistic or boundary.
Worked Example: Three-Look Boundary for a Single Endpoint
Consider a hypothetical one-sided endpoint test with three looks and local alpha \(0.03\). Use equal Bonferroni allocation. Each look receives \(0.03/3=0.01\), so the common boundary is
Suppose the standardized statistics are \(Z_1=1.10\), \(Z_2=1.90\), and \(Z_3=2.40\). The first two are below the boundary; the third satisfies \(2.40>2.3263\). The endpoint null is rejected at the third look. Under the assumed marginal normal null distributions, the probability of crossing at any of the three looks is at most \(3(0.01)=0.03\), regardless of the dependence among those looks. This construction is conservative compared with a boundary calibrated to the full joint distribution, which can use the positive correlation between cumulative statistics.
Worked Example: A Two-Sided Local Test
Worked Example: A Two-Sided Local Test
Suppose a hypothetical endpoint has two looks and receives local two-sided alpha \(0.02\). Under equal Bonferroni allocation, each look and each tail receive \(0.02/(2\times2)=0.005\). The common absolute-value boundary is therefore \(\Phi^{-1}(0.995)=2.5758\). If the observed statistics are \(Z_1=-2.20\) and \(Z_2=-2.70\), then \(|Z_1|=2.20<2.5758\), while \(|Z_2|=2.70>2.5758\). The endpoint is rejected at the second look in the negative direction. The two-sided calibration protects the total probability of a false crossing in either direction; using a one-sided cutoff here would incorrectly allocate the full local alpha to each tail.
Choosing and Checking the Statistic
The analysis should define the estimand, contrast direction, statistic, variance estimate, information, and look-specific boundary as a matched set. For a continuous endpoint, a standardized mean contrast may use a known variance under a normal model, or an estimated standard error under an appropriate large-sample or model-based justification. For a binary endpoint, a score statistic tests equality of response probabilities using the null-based pooled variance. The statistic must be computed according to the planned method at every look; replacing a score statistic by a Wald statistic midway through monitoring changes its null behavior and can invalidate the boundaries.
Several diagnostics help identify trouble before interpreting a crossing:
- Check that the contrast sign, units, and treatment direction match the stated alternative.
- Check that the score’s null variance and the continuous endpoint’s standard error use the specified analysis model.
- Verify that looks use cumulative data and the planned information definition, rather than treating repeated cumulative estimates as independent observations.
- Investigate sparse binary counts, major departures from the normal approximation, unequal variances not covered by the model, and missing data patterns that undermine the planned standardization.
- Confirm that each boundary is calibrated for the correct endpoint, number of looks, sidedness, and current local alpha allocation.
A useful alternative to conservative Bonferroni boundaries is joint calibration under a specified joint distribution of the sequential statistics. It can yield less conservative boundaries by accounting for correlation across looks, and sometimes across endpoints. That approach requires a credible joint model and accurate implementation. Conversely, separate endpoint-wise calibration combined with Bonferroni control is more robust to unknown dependence but may lose power. An unadjusted nominal threshold at every look is not a valid substitute: it generally allows the chance of at least one false crossing to exceed the intended local alpha.
Common Mistake and Limitations
Wrong reasoning: “Both statistics are Z statistics, so the same cutoff can be used for both endpoints, and each endpoint can be checked at every look at that cutoff.” The correction is that standardization makes statistics comparable to a reference scale under their respective null models; it does not determine their allocated alpha or account for repeated looks. Each endpoint needs a boundary calibrated to its local allocation, sidedness, and sequential schedule.
Even correctly calibrated boundaries do not guarantee that the scientific conclusions are reliable if the endpoint measurement, model, or data collection is flawed. Bonferroni boundaries may be conservative, while joint calibration can be sensitive to misspecified dependence or information. Asymptotic score behavior may be inaccurate with very sparse responses. The design must use a method whose assumptions fit the endpoint and sample size, and results should be interpreted as evidence from the planned local tests—not as proof that a null hypothesis is true or that an observed effect is clinically important.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A one-sided endpoint receives local alpha \(0.02\) and has two looks. What common Bonferroni boundary is used at each look?
- For a two-sided endpoint with local alpha \(0.03\) and three looks, what is the per-tail, per-look error allocation?
- Why can a binary score statistic and a mean-contrast statistic not be assigned the same boundary merely because both are standardized as \(Z\) statistics?
- Under what condition can endpoint-wise local tests support familywise error control when endpoint statistics are dependent?
Solutions. (1) Each look receives \(0.02/2=0.01\), so \(b=\Phi^{-1}(0.99)=2.3263\). (2) The allocation is \(0.03/(2\times3)=0.005\) per tail per look, giving an absolute boundary \(\Phi^{-1}(0.995)=2.5758\). (3) The local alpha, number of looks, sidedness, and validity of each statistic’s null distribution determine calibration; standardization alone does not set the cutoff. (4) Each local test must control its endpoint’s error at the allocated level, and the allocations for the true null hypotheses must sum to no more than the overall familywise level; a union bound then controls the chance of any false rejection without requiring endpoint independence.