Why Dependence Matters for an Intersection Test
In a multi-endpoint group sequential design, a local test may concern an intersection null: the claim that several endpoint-specific null hypotheses are all true. If a local test rejects that intersection, the closed-testing procedure may use the result to support claims about individual endpoints. Its type I error must therefore be controlled when the entire intersection null is true. Calibration of each endpoint test separately does not, by itself, establish that control for a procedure that rejects when any endpoint crosses a boundary.
Dependence matters because endpoint statistics measured on the same participants are often correlated. Their statistics across interim looks are also related because later analyses include accumulating information from earlier participants. A procedure that uses the joint distribution of these statistics can account for both relationships, but it is only valid if the assumed joint model is justified. When dependence is unknown, a dependence-robust local test is available, usually at the cost of some power.
Marginal Validity Is Not Joint Validity
Suppose endpoint \(j\) has a sequential test whose probability of ever rejecting its null is at most \(u\) when that null is true. This is a marginal statement about endpoint \(j\). It does not say that the probability of at least one rejection among several true endpoint nulls is at most \(u\). That latter probability depends on the joint distribution of the endpoint statistics and can be larger than \(u\).
The distinction applies even if every endpoint has already been calibrated across its own interim looks. Sequential calibration handles repeated opportunities to reject within that endpoint; it does not automatically handle the additional opportunities created by testing multiple endpoints. A local intersection test must account for both dimensions, either through a valid joint model or through a bound that remains valid regardless of dependence.
A useful dependence-robust construction starts with a valid sequential p-value \(p_j\) for each endpoint \(j\). Here, validity means that under \(H_j\), \(P(p_j\leq u)\leq u\) for every \(u\) between 0 and 1; the p-value must reflect the entire sequential procedure, not just the nominal p-value at the observed look. If there are \(m\) endpoints in \(I\), reject \(H_I\) when at least one \(p_j\leq\alpha_I/m\). The union bound establishes validity without requiring independence or any other specific dependence structure.
The proof uses only the marginal sequential validity of each p-value and the fact that all the component nulls in \(H_I\) are true. No assumption about correlation is needed. In a closed-testing procedure, this property is needed for each intersection test at its assigned level. A local test that relies on an unjustified dependence assumption can invalidate the closure argument, even if the other parts of the procedure are implemented correctly.
Worked Example: A Robust Test of a Three-Endpoint Intersection
Consider a hypothetical trial with three endpoints and an intersection local level of \(\alpha_I=0.05\). Assume each endpoint has a sequential p-value valid under its null across all planned looks. At the analysis, the sequential p-values are 0.012, 0.031, and 0.080. The dependence among these p-values is unknown.
The Bonferroni threshold is \(0.05/3=0.01667\). Only the first p-value, 0.012, is at or below this threshold. The local test therefore rejects the three-endpoint intersection.
This conclusion has level at most 0.05 under the intersection null for any dependence among the endpoints: the probability that one or more of three valid sequential p-values falls below 0.01667 is at most \(3(0.01667)\), which is approximately 0.05. The procedure does not use the observed p-values to estimate dependence; it trades some potential power for a guarantee that does not depend on that estimate.
When a Joint Dependence Model Is Used
A less conservative local test may use a joint reference distribution. For example, a one-sided maximum-statistic test rejects an intersection if the largest endpoint statistic exceeds a critical value selected so that the probability of crossing that value under the intersection null is at most \(\alpha_I\). In a group sequential design, the event being calibrated is typically a crossing at any endpoint at any monitored look. The critical value must therefore reflect the joint distribution across endpoints and across looks.
A common modeling framework assumes that the standardized statistics are jointly normal under the null, with a specified covariance structure. Under a canonical group sequential model and a common information schedule \(0<t_1<\cdots<t_K=1\), the within-endpoint correlation between looks \(k\) and \(\ell\) is often modeled as \(\sqrt{t_k/t_\ell}\) when \(k\leq\ell\). Under a particular separable joint model, the correlation between endpoint \(a\) at look \(k\) and endpoint \(b\) at look \(\ell\) is modeled as \(\rho_{ab}\sqrt{t_k/t_\ell}\) when \(k\leq\ell\), where \(\rho_{ab}\) is the endpoint correlation at a common information time. These are model assumptions, not consequences of having standardized statistics. Their appropriateness depends on the estimands, analysis statistics, information measures, and data structure.
To calibrate a maximum-statistic rule under such a model, choose a cutoff \(c\) satisfying \(P(\max_{j\in I,\,k} Z_{jk}>c\mid H_I)\leq\alpha_I\), where \(Z_{jk}\) is the standardized statistic for endpoint \(j\) at look \(k\). This probability depends on the full covariance structure. Simulation or numerical integration can be used to find \(c\), provided the simulated distribution represents the planned null model and monitoring schedule.
Worked Example: How an Incorrect Correlation Assumption Changes Calibration
Consider a hypothetical one-look, one-sided test of the intersection of two endpoint nulls at \(\alpha_I=0.05\). Suppose both statistics are standard normal under the intersection null. First assume independence and use the maximum statistic. The cutoff \(c\) must satisfy \(P(\max(Z_1,Z_2)\leq c)=0.95\).
Under independence, this gives a rejection probability of 0.05. Now suppose instead that the statistics are perfectly positively correlated, so \(Z_1=Z_2\). Using the independence-based cutoff, the rejection probability is \(P(Z_1>1.955)\approx0.0253\), which is below 0.05. In this particular direction, the independence-based cutoff is conservative.
The reverse modeling error can be more serious. Under perfect positive correlation, the exact 0.05 cutoff for the maximum is the one-sided normal quantile \(1.645\). If that cutoff is used when the statistics are actually independent, the rejection probability is \(1-\Phi(1.645)^2\approx1-(0.95)^2=0.0975\), nearly twice the target level. Thus a cutoff calibrated under an assumed dependence structure does not automatically control error under a different one. The example has one look; with multiple looks, the assumed dependence across looks also affects the crossing probability.
Assumptions, Diagnostics, and Alternatives
A model-based intersection test needs assumptions strong enough to determine the joint null distribution relevant to its rejection rule. A joint normal approximation may be reasonable for some large-sample score or estimate statistics, but marginal normality alone is insufficient: the joint dependence must also be modeled adequately. The covariance matrix must be valid, its parameters must be specified or estimated using a defensible method, and the information-time relationship across looks must reflect the actual design. Any approximation used to construct boundaries must control the probability of crossing over the planned monitoring process, not merely at the final look.
Before relying on model-based calibration, examine how the endpoint statistics are constructed and whether their participants, measurements, and analysis times overlap. Review the derivation or estimation of the covariance parameters and check that the proposed covariance matrix is positive semidefinite. Where feasible, compare the nominal calibration with simulations across plausible dependence structures, including values near the boundaries of the scientifically credible range. Such checks can reveal sensitivity, but simulation under one fitted model cannot prove validity under an unknown true model. If the design needs a guarantee without a sufficiently justified joint model, use a robust procedure such as Bonferroni.
| Approach | Dependence requirement | Main tradeoff |
|---|---|---|
| Bonferroni local test using valid sequential p-values | No endpoint-dependence assumption; each sequential p-value must be marginally valid | Robust, but may be conservative, especially for many endpoints |
| Joint maximum-statistic calibration | A justified joint null distribution across endpoints and looks | Can gain power by using dependence information; vulnerable to model misspecification |
| Other dependence-robust bounds or procedures | Depends on the particular procedure; guarantees must be established under the stated conditions | May improve on simple Bonferroni, but requires its own validity argument |
Neither approach removes the need to check the monitoring design. For the robust approach, an ordinary fixed-look p-value substituted for a sequentially valid p-value can fail to account for repeated looks. For the model-based approach, omitting interim looks from the joint crossing calculation can understate the chance of rejection. Also distinguish the local level \(\alpha_I\) assigned to an intersection from the overall familywise error target: the closed-testing procedure's local tests must be valid at their assigned levels for the overall guarantee to follow.
List the endpoint nulls in \(H_I\), its local level \(\alpha_I\), and the planned interim monitoring for every endpoint.
Confirm that each component p-value or test accounts for all planned looks under its own null.
Use a dependence-robust bound if the joint structure is unknown or inadequately justified; otherwise specify the joint model and its assumptions.
For model-based testing, calibrate crossings across endpoints and looks. For a robust method, apply its stated bound to the valid sequential tests.
Check plausible covariance alternatives and state whether the error guarantee is model-based or dependence-robust.
Common Mistake and Practical Limitation
Common mistake: “Each endpoint has a sequential test at level 0.05, so rejecting the intersection whenever either endpoint rejects is also a level-0.05 test.” This confuses marginal validity with joint validity. If two endpoint rejection events are independent and each has probability 0.05 under the intersection null, the probability of at least one rejection is \(1-(1-0.05)^2=0.0975\), not 0.05. The correction is to calibrate the joint rejection event under a justified joint model or to use a dependence-robust adjustment such as Bonferroni.
A robust adjustment is not cost-free: dividing the local level among many endpoints can reduce power, particularly when the endpoint statistics are strongly correlated. Conversely, model-based methods can be more efficient but their guarantees depend on the model. The choice is therefore not “assumption-free versus assumption-based” in an absolute sense. It is a choice between a conservative bound built from marginal validity and a potentially more powerful calibration whose dependence assumptions must be defended.
Worked Example: A Two-Look Bonferroni Calibration
Suppose a hypothetical intersection contains two endpoint nulls, has local level \(\alpha_I=0.04\), and each endpoint will be monitored at two interim looks. Assume a valid group sequential test has already been calibrated for each endpoint at level 0.02 across both looks. Under the intersection null, each endpoint's probability of any rejection over its two looks is therefore at most 0.02.
The probability of any rejection for either endpoint over either look is at most the sum of those two probabilities: \(0.02+0.02=0.04\). Thus rejecting the intersection if either endpoint's calibrated sequential test rejects controls its local type I error at 0.04, regardless of dependence between endpoints or across their looks.
The calculation does not require dividing 0.02 again by two looks: the premise is that each endpoint's group sequential test already controls its error across both looks. If instead the analysis used unadjusted fixed-look tests, that premise would not hold, and an additional sequential calibration would be needed before applying the endpoint-level union bound.
Check Your Understanding
Answer each question, then compare with the brief solutions.
- For an intersection of four endpoint nulls at local level 0.08, what Bonferroni threshold applies to valid sequential p-values?
- Why does calibrating each endpoint's sequential test at 0.025 not automatically make an “either endpoint rejects” rule level 0.025 for two endpoints?
- Under the independence model in the two-endpoint worked example, what is the probability of rejection if the cutoff 1.645 is used?
- What extra information is required to use a joint maximum-statistic test across multiple endpoints and interim looks?
- If each endpoint test already controls its type I error over all planned looks, should a Bonferroni adjustment across endpoints also divide by the number of looks? Explain.
Solutions. (1) \(0.08/4=0.02\). (2) The probability of the union of two rejection events can exceed either marginal probability; under independence with marginal probability 0.025 it is \(1-(0.975)^2=0.0494\). (3) Under independence, \(1-\Phi(1.645)^2\approx0.0975\). (4) A justified joint null distribution or covariance structure across both endpoints and looks, together with the planned monitoring schedule. (5) No. If the endpoint-level tests are already sequentially valid across all looks, the union bound is applied across endpoints; dividing by the number of looks again would be an unnecessary extra adjustment.