Why Endpoint Looks May Not Line Up
Different endpoints can accrue information at different rates. A time-to-event endpoint may gain information as events occur, while another endpoint’s information may depend on repeated measurements or a different event process. If analyses are scheduled when each endpoint reaches its own information targets, one endpoint can have a look when another has none. A valid procedure must account for every planned test, not just for the occasions when all endpoints happen to be analyzed together.
The key distinction is between calendar time and information time. Calendar time says when an analysis occurs. Information time describes how much statistical information has accrued for a particular endpoint. Because endpoints can have different information processes, a shared calendar date does not imply a shared information fraction, and matching information fractions does not imply a shared calendar date.
Endpoint-Specific Schedules and Assumptions
To describe sequential testing precisely, suppose each endpoint has a one-sided null hypothesis \(H_{0j}\), a valid standardized test statistic \(Z_j(t_{jk})\) at each planned look, and a prespecified boundary \(b_{jk}\). Under \(H_{0j}\), assume the statistic has the standard group sequential canonical model: each \(Z_j(t_{jk})\) is standard normal, and within endpoint \(j\),
This model applies when the information fractions are correctly specified and the sequential statistic has the canonical information-based form. The schedule must be prespecified or generated by a design with valid sequential error control; simply changing analysis times after examining unblinded results is not justified by this model. For the familywise argument below, no independence assumption between endpoints is required.
An endpoint-specific local significance allocation \(\alpha_j\) is a bound on the probability of rejecting \(H_{0j}\) at any of that endpoint’s planned looks when \(H_{0j}\) is true. The allocations must satisfy \(\sum_{j=1}^{m}\alpha_j\leq\alpha\), where \(\alpha\) is the desired familywise significance level. Boundaries can be calibrated to spend \(\alpha_j\) over endpoint \(j\)’s own information schedule, or a simpler conservative procedure can divide \(\alpha_j\) among its looks.
Why Endpoint-Specific Information Time Matters
For the canonical model, the sequential statistics can be represented as \(Z_j(t)=B_j(t)/\sqrt{t}\), where \(B_j(t)\) is Brownian motion on the endpoint’s information scale. For \(s\leq t\), Brownian motion has covariance \(\operatorname{Cov}\{B_j(s),B_j(t)\}=s\). Therefore,
This explains why the correlation structure is indexed by information fractions rather than by calendar spacing. Two analyses separated by six months can be highly correlated or less correlated depending on how much information accrued between them. For a spending procedure, boundaries must be calibrated using the endpoint’s actual information fractions and this within-endpoint correlation structure; treating its looks as equally spaced in time can give the wrong crossing probability.
For example, if a one-sided spending function specifies cumulative type I error expenditure \(A_j(t)\) by information fraction \(t\), the boundary at look \(k\) is calibrated so that the null probability of crossing by that look matches the specified cumulative expenditure, subject to the chosen spending convention. In general, the boundary is not obtained by treating \(A_j(t_{jk})-A_j(t_{j,k-1})\) as an independent test level: earlier looks and the current statistic are correlated. A local allocation \(\alpha_j\) is the endpoint’s total error budget across its schedule, not a fresh allowance at every analysis.
A Familywise Error Guarantee With Asynchronous Looks
The proof is the union bound. Let \(R_j\) be the event that endpoint \(j\)’s procedure rejects its true null at one or more looks. The familywise error event is contained in the union of these events over the true null hypotheses. Hence,
where \(\mathcal{T}\) is the set of endpoints whose null hypotheses are true. The proof does not require the endpoints to be tested simultaneously, nor does it require their statistics to be independent. What it does require is that each endpoint’s error bound cover its entire sequence of looks. Counting only one look per endpoint, or applying \(\alpha_j\) anew at every look, would not establish the theorem’s premise.
Worked Example: Writing Down Two Asynchronous Schedules
Consider a hypothetical study with two one-sided endpoints. Endpoint A has planned analyses at weeks 8, 16, and 24, with information values \(40\), \(75\), and \(100\) units. Endpoint B has planned analyses at weeks 12, 20, and 24, with information values \(30\), \(70\), and \(100\) units. For each endpoint, compute the information fractions and identify which analyses occur at week 16.
For endpoint A, the final information is \(100\), so its fractions are \(40/100=0.40\), \(75/100=0.75\), and \(100/100=1.00\). Its schedule is therefore weeks \(8,16,24\) at fractions \(0.40,0.75,1.00\).
For endpoint B, the final information is also \(100\), giving fractions \(30/100=0.30\), \(70/100=0.70\), and \(100/100=1.00\). Its schedule is weeks \(12,20,24\) at fractions \(0.30,0.70,1.00\).
At week 16, endpoint A is at its second look, with information fraction \(0.75\). Endpoint B has no planned look at week 16; its first look was at week 12 and its second is at week 20. The week-16 analysis must not be described as a joint second look for both endpoints. The two schedules coincide at week 24, but even then their matching calendar time does not mean that their earlier information histories were the same.
Worked Example: A Conservative Error Allocation Across Unequal Schedules
Suppose the familywise target is one-sided \(\alpha=0.05\). Allocate \(\alpha_A=0.03\) to endpoint A and \(\alpha_B=0.02\) to endpoint B. Endpoint A has three planned looks and endpoint B has two. Use a conservative equal division of each endpoint’s allocation among its own looks. Find the per-look test levels and critical values, then evaluate hypothetical statistics: A has values \(2.10,2.40,1.80\), and B has values \(2.05,2.32\).
For A, the per-look level is \(0.03/3=0.01\). For B, it is \(0.02/2=0.01\). Each look therefore uses the one-sided standard-normal critical value \(z_{0.99}\approx2.326\). Reject an endpoint’s null if its statistic exceeds this value at any of that endpoint’s looks, stopping further testing of that endpoint after rejection if the design specifies that convention.
Endpoint A’s second statistic is \(2.40>2.326\), so A’s null is rejected at week 16. Its first statistic, \(2.10\), did not cross the boundary. Endpoint B’s largest statistic is \(2.32<2.326\), so B’s null is not rejected at either look. In particular, the final B statistic misses the boundary by \(2.326-2.32=0.006\).
For a true endpoint null, the chance of at least one false rejection across its looks is at most the sum of its per-look levels. Thus A’s endpointwise error is at most \(3(0.01)=0.03\), and B’s is at most \(2(0.01)=0.02\). The familywise error is at most \(0.03+0.02=0.05\), regardless of dependence between A and B. This allocation is conservative because the repeated statistics within an endpoint are correlated; it does not claim that the actual crossing probability equals the sum of the per-look levels.
Calibrating Each Endpoint on Its Own Schedule
Equal division is transparent but can be inefficient. A more tailored procedure calibrates a group sequential boundary separately for each endpoint using its own information fractions and a prespecified spending rule. The endpoint’s total local level remains \(\alpha_j\). Its boundary sequence is chosen to control the probability of crossing at any of its scheduled looks under its null, accounting for the correlation between those looks. Asynchronous calendar times do not require a common boundary index: endpoint A’s second boundary is defined by A’s second look, and endpoint B’s second boundary by B’s second look.
Worked Example: Different Numbers of Looks, Same Familywise Target
In a hypothetical one-sided design, let \(\alpha=0.05\), with fixed allocations \(\alpha_A=0.03\) and \(\alpha_B=0.02\). Suppose A has two looks, at weeks 10 and 22, while B has four looks, at weeks 6, 13, 19, and 25. For a simple conservative illustration, divide each allocation equally across that endpoint’s looks. A’s per-look level is \(0.03/2=0.015\), giving critical value \(z_{0.985}\approx2.170\). B’s per-look level is \(0.02/4=0.005\), giving critical value \(z_{0.995}\approx2.576\).
Suppose A’s statistics are \(1.90\) and \(2.20\), and B’s are \(2.10,2.30,2.45,2.40\). A does not cross at its first look, then rejects at its second because \(2.20>2.170\). B does not reject: its largest value, \(2.45\), remains below \(2.576\). The endpointwise error bounds are at most \(2(0.015)=0.03\) for A and \(4(0.005)=0.02\) for B, so the familywise bound is at most \(0.05\).
The more frequent looks make B’s simple per-look boundary more stringent. A spending-based calibration could use B’s information fractions and a chosen spending function to allocate its \(\alpha_B\) less uniformly, but it must still account for the correlation among B’s four statistics. It cannot borrow A’s calendar schedule as a substitute for B’s information schedule.
Diagnostics, Alternatives, and Limitations
Before implementation, check that each endpoint has a clearly documented sequence of analysis times, information values, information fractions, local alpha allocation, and boundary rule. At each interim review, record which endpoints were actually analyzed and which look number and information fraction applied to each. A calendar-time table is helpful, but it should not conceal the endpoint-specific look index.
- Information fractions do not match the data. If projected event counts or variances are used to set information times, compare the observed information with the plan and document the update rule. A fraction calculated from an outdated final-information assumption can misrepresent the sequential correlation structure.
- Looks are added or moved after seeing unblinded results. The original spending or per-look allocation does not automatically cover unplanned analyses. The design needs a valid rule for the changed schedule and its error expenditure.
- The endpoint budget is counted repeatedly. Using \(\alpha_j\) at every look is not an endpointwise level-\(\alpha_j\) procedure. Either calibrate the whole boundary sequence to \(\alpha_j\), or explicitly allocate portions whose sum is no greater than \(\alpha_j\).
- Endpoint allocations are changed without recalibration. If a multiplicity procedure changes local levels after a rejection, later boundaries must respect the updated allocation and each endpoint’s remaining information schedule. A past boundary cannot simply be relabelled with a new alpha level.
The main alternative is a common interim schedule across endpoints. It simplifies operational coordination and can make a shared analysis calendar easier to communicate, but it may require waiting for a slower endpoint or analyzing a faster endpoint at information fractions that were not its natural targets. Asynchronous schedules can align analyses more closely with each endpoint’s information accrual, at the cost of more complicated implementation and reporting.
The union-bound procedure is robust to dependence across endpoints but may be conservative, especially when endpoint rejection events overlap substantially. More efficient closed or graphical multiplicity procedures can use structured local testing and alpha reallocation, as established in the earlier material on closure and endpoint graphs. Those procedures still need valid sequential tests for the endpoint-specific schedules; asynchronous timing does not remove the need to calibrate each sequential component correctly. The guarantee also depends on valid endpointwise tests and the specified testing family: it does not correct a misspecified test statistic, omitted endpoint, or invalid data-dependent schedule.
Common Pitfall
Wrong reasoning: “There are two endpoints, so I can test each at \(\alpha=0.025\) at every analysis date. At a date when only one endpoint is observed, the other endpoint contributes no test, so the familywise level is still 0.05.”
Correction: The number of distinct calendar dates is not the number of opportunities for a false rejection. Each endpoint can be tested repeatedly at its own looks. Its local level must cover the probability of crossing anywhere on its schedule, and the local allocations across endpoints must sum to at most the familywise target. Whether another endpoint is analyzed on the same date does not reduce the error opportunity from an already scheduled look.
Check Your Understanding
Answer the questions, then compare your reasoning with the solutions below.
- Endpoint C has information \(50,80,100\) at weeks 10, 18, and 26. What are its information fractions, and which fraction applies at week 18?
- Two endpoint procedures have endpointwise type I error bounds \(0.025\) and \(0.015\). What familywise bound follows from the union bound, without assuming independence?
- Why is it generally incorrect to use the increments of a cumulative spending function as independent per-look test levels?
- With endpoint allocation \(\alpha_D=0.024\) and three looks, what conservative equal per-look level and one-sided normal critical value apply? Use \(z_{0.992}\approx2.409\).
- At week 20, endpoint E reaches its second look but endpoint F has no planned analysis. Does the absence of an F analysis change E’s boundary or local error allocation?
Solutions. (1) Divide each information value by the final value \(100\): the fractions are \(0.50,0.80,1.00\). At week 18, C is at fraction \(0.80\). (2) The familywise error is at most \(0.025+0.015=0.040\); the union-bound result does not require independence. (3) Repeated statistics within an endpoint are correlated, so a spending procedure calibrates cumulative crossing probability using their joint sequential distribution; the increments are not automatically independent test levels. (4) The per-look level is \(0.024/3=0.008\), and the one-sided critical value is \(z_{0.992}\approx2.409\). (5) No. E’s boundary is determined by E’s own look number, information fraction, and calibration; F’s lack of a simultaneous analysis does not alter those quantities or E’s error budget.