Why Endpoint Allocation Rules Differ
A trial can monitor several endpoint hypotheses at interim looks, but an endpoint’s statistical importance does not by itself determine the testing rule. The allocation rule specifies which endpoint may be tested at what level, and what happens to its allocated alpha after rejection. Two familiar choices make different tradeoffs: fixed-sequence testing concentrates testing authority along a strict priority order, while a fallback procedure gives endpoints initial allocations and can pass the allocation of a rejected endpoint to later endpoints.
Sequential monitoring adds a second layer. Each endpoint’s test must account for all of its looks, not just the look at which a result first appears significant. In the discussion below, a sequential test is understood to be calibrated over its planned looks at the endpoint’s allocated level. A sequentially adjusted p-value summarizes that test: rejection at level \(a\) means the endpoint’s group-sequential test would reject at local level \(a\). If an allocation changes during monitoring, its sequential boundary or conditional-error calculation must be updated by a valid method; comparing an ordinary, unadjusted interim p-value with a newly enlarged level is not sufficient.
A transfer rule must say exactly which allocation moves. Here, it is the rejected endpoint’s current allocation, including any alpha it received earlier. That amount is divided using the prespecified transfer fractions, and the rejected endpoint is removed from further testing. Fractions may direct allocation only to later endpoints. This makes the procedure auditable and prevents a rejected endpoint from retaining or transferring the same allocation more than once.
Sequential Decisions and Transfer Steps
For endpoint \(i\), let \(p_i\) denote its sequentially adjusted p-value, or equivalently let its group-sequential test have a rejection decision calibrated across all planned looks. At the current local level \(a_i\), reject \(H_i\) if \(p_i\leq a_i\). In practice, the relevant p-value or boundary must correspond to the endpoint’s actual look schedule and the local level then in force. A transfer changes the level available to a later endpoint; it does not retroactively change the evidence or the rejection status of the endpoint that triggered the transfer.
The following calculation describes a serial implementation: each endpoint’s sequential test is resolved before the next endpoint is processed. More generally, if endpoint tests overlap in calendar time, an alpha increase during another endpoint’s monitoring requires a calibrated update to the continuing sequential test. Such an update must preserve the relevant error guarantee rather than simply replacing its boundary with a more permissive one.
Specify \(a_i\) for each endpoint and verify that their sum is no greater than the familywise level \(\alpha\).
Use the planned looks and the boundary or sequential p-value calibrated for its current local level.
If \(H_i\) is rejected, distribute its current allocation to eligible later endpoints using the prespecified \(\gamma_{ij}\). If it is not rejected, its allocation is not recycled.
Record the rejection, local level, look, and transfer for every endpoint. Do not treat a futility decision as an alpha transfer unless the procedure explicitly specifies one.
Worked Example: Full Transfer Along an Endpoint Sequence
Consider three hypothetical endpoint hypotheses with \(\alpha=0.05\) and initial allocations \(a_1=0.025\), \(a_2=0.015\), and \(a_3=0.010\). Each endpoint has a sequentially adjusted p-value calculated over its planned looks. Suppose the values are \(p_1=0.018\), \(p_2=0.034\), and \(p_3=0.047\). On rejection, each endpoint passes all of its current allocation to the next endpoint.
The initial budget checks: \(0.025+0.015+0.010=0.050\). Since \(p_1=0.018\leq0.025\), reject \(H_1\). Its \(0.025\) allocation moves to \(H_2\), giving \(a_2=0.015+0.025=0.040\). Because \(p_2=0.034\leq0.040\), reject \(H_2\); its current \(0.040\) moves to \(H_3\), giving \(a_3=0.010+0.040=0.050\). Finally, \(p_3=0.047\leq0.050\), so reject \(H_3\).
These are decisions under the specified fallback rule, not evidence that all three null hypotheses are false with a particular probability. The numbers \(p_i\) are assumed here to be sequentially adjusted, so each comparison represents the test across all looks—not an unadjusted comparison at a selected interim look.
Worked Example: A Partial Transfer and a Failed Test
Suppose three hypothetical endpoints have initial allocations \(0.020\), \(0.020\), and \(0.010\), totaling \(0.050\). If \(H_1\) is rejected, transfer 75% of its current allocation to \(H_2\) and 25% to \(H_3\). If \(H_2\) is rejected, transfer all its current allocation to \(H_3\). Let the sequentially adjusted p-values be \(p_1=0.016\), \(p_2=0.041\), and \(p_3=0.012\).
Because \(0.016\leq0.020\), reject \(H_1\). The transfer is \(0.020(0.75)=0.015\) to \(H_2\) and \(0.020(0.25)=0.005\) to \(H_3\). Thus \(H_2\) has level \(0.020+0.015=0.035\), and \(H_3\) has level \(0.010+0.005=0.015\). Since \(0.041>0.035\), do not reject \(H_2\); its allocation is not passed on. The third endpoint remains testable at \(0.015\), and \(0.012\leq0.015\), so reject \(H_3\).
The failed test of \(H_2\) does not erase \(H_3\)’s original allocation or its \(0.005\) transfer from \(H_1\). It only prevents \(H_2\)’s current \(0.035\) allocation from being transferred onward.
Worked Example: Reading Rejections Across Interim Looks
In a hypothetical two-endpoint design, \(H_1\) starts at level \(0.020\), and \(H_2\) starts at \(0.015\). If \(H_1\) is rejected, its full current allocation transfers to \(H_2\). Assume the sequential analyses have been calibrated for the specified look schedules and allocations. For \(H_1\), the sequentially adjusted p-value at its first look is \(0.032\), and after its second look it is \(0.014\). For \(H_2\), the corresponding values are \(0.061\) at its first look and \(0.028\) at its second.
At the first look, \(H_1\) is not rejected because \(0.032>0.020\). At its second look, the adjusted value is \(0.014\leq0.020\), so reject \(H_1\) and transfer \(0.020\). The resulting level for \(H_2\) is \(0.015+0.020=0.035\). At its first look \(H_2\) is not rejected because \(0.061>0.035\); at its second look it is rejected because \(0.028\leq0.035\).
This example uses values already adjusted for the sequential monitoring. If a transfer instead arrives while \(H_2\) is already being monitored, the design needs a valid boundary update or conditional-error recalibration for that change. The sequence of comparisons above cannot be justified merely by comparing raw interim p-values with an enlarged threshold.
Why Fallback Can Control Familywise Error
The familywise error rate (FWER) is the probability of rejecting at least one true endpoint hypothesis. A data-dependent threshold needs care: knowing that a test is valid at each fixed level does not, by itself, show that it remains valid when the chosen level depends on earlier endpoint results. The following result therefore states the condition that makes the fallback argument work.
The conditional assumption is stronger than marginal validity. It explicitly supports the fact that an endpoint’s current level is determined by earlier results. It also treats each endpoint’s sequential test as a whole: the conditional bound applies to rejection over all of that endpoint’s looks, not separately at every look.
Justification by Conditional Error and Allocation Accounting
Let \(R_i\) indicate rejection of \(H_i\), and let \(\mathcal F_{i-1}\) denote the observed history before endpoint \(i\) is processed. Its current allocation \(A_i\) is determined by that history. For a true \(H_i\), conditional validity gives
For each true endpoint \(i\), consider the event that no earlier true endpoint was rejected and that \(H_i\) is the first true hypothesis rejected. These events are disjoint and their union is the familywise-error event. Conditioning on the history before each endpoint and applying the displayed bound gives
It remains to bound the sum inside the expectation. Think of each unit of initial allocation as a token. A token can move through rejected endpoints according to the transfer rule, but it cannot be duplicated: the transfer fractions sum to at most one, and any untransferred amount is discarded. If a true endpoint is tested and not rejected, the tokens in its allocation are not transferred onward. On any realized path, therefore, a token can contribute to the allocation tested at at most one true endpoint before the first true rejection. The sum of allocations tested at true endpoints up to and including that first rejection is consequently no greater than the total initial allocation, which is at most \(\alpha\). The random sum in the expectation is bounded pathwise by \(\alpha\), so its expectation is also at most \(\alpha\). This proves the claimed FWER bound.
This argument addresses both sources of adaptivity: the local level may depend on earlier results, and the first true null rejected is not known in advance. Conditional validity handles the random level at each true endpoint; the allocation accounting handles the sequence of possible first true rejections.
Fixed Sequence, Fallback, and Practical Limits
| Feature | Fixed-sequence testing | Fallback testing |
|---|---|---|
| Initial testing level | Tests the first endpoint at \(\alpha\). | Assigns initial levels whose sum is at most \(\alpha\). |
| After a rejection | Moves to the next hypothesis, tested at \(\alpha\). | Transfers the rejected endpoint’s current allocation according to a prespecified rule. |
| After a failure | Stops; later hypotheses are not rejected. | Does not transfer the failed endpoint’s allocation; other endpoints may still be tested at their remaining levels. |
| Key tradeoff | High level for the top-priority claim, but strict dependence on earlier rejection. | More flexible allocation across endpoints, but levels may be smaller before transfers occur. |
Fixed sequence has a simple FWER argument under marginal sequential validity. If any true hypothesis is rejected, then the first true hypothesis in the prespecified order must have been reached and rejected at level \(\alpha\). Thus the familywise-error event is contained in the event that this particular true hypothesis rejects, whose probability is at most \(\alpha\). Later hypotheses cannot be reached after a failed test.
Fallback can be useful when endpoints have distinct initial priorities but the analysis team wants some unused alpha to support later claims after earlier rejections. Its guarantee, however, depends on the assumptions used to justify the changing levels. The conditional-validity condition in the theorem is sufficient, but it is not implied by the statement that each endpoint p-value is marginally valid under its own null. With dependent endpoint statistics, earlier outcomes can carry information about a later endpoint’s statistic. If conditional validity is not defensible, use a procedure with a separately established guarantee under the relevant dependence structure rather than assuming the transfer proof still applies.
Implementation should also specify the order of testing, initial allocations, transfer fractions, treatment of ties, and what happens after a futility decision or an unavailable endpoint result. At interim looks, verify that the endpoint-specific sequential procedure remains calibrated after any allocation change. The mechanics of transferring alpha and the validity of the resulting sequential boundary are separate checks: correct transfer arithmetic does not repair an invalid boundary.
Common Mistake
- Wrong reasoning: “Every endpoint test is valid at its assigned level, and the current levels sum to at most \(\alpha\), so FWER is controlled even when those levels depend on earlier results.”
- Correction: A random level can be associated with the endpoint’s data, so fixed-level validity alone does not justify the comparison. Establish conditional validity for the adaptive testing history, or use another procedure with an appropriate error-control proof. Then account for all possible first true rejections and show that the allocation tokens available to true endpoints cannot exceed the initial total.
- Another pitfall: Treating an interim nominal p-value as sequentially adjusted, or transferring alpha after a nonrejection. The former ignores repeated looks; the latter changes the procedure and invalidates the stated allocation argument.
Check Your Understanding
Use the stated fallback rule in each question. Assume sequential p-values and validity conditions are as described above where needed.
- Three endpoints start at \(0.020\), \(0.015\), and \(0.015\). If \(H_1\) is rejected, all its allocation goes to \(H_2\). If \(H_2\) is rejected, all its current allocation goes to \(H_3\). Given adjusted p-values \(0.018\), \(0.030\), and \(0.040\), determine the decisions and levels.
- In a fixed-sequence procedure, why does rejection of a later true hypothesis imply rejection of the first true hypothesis in the fixed order?
- Suppose two true endpoint hypotheses each have marginally valid p-values, but the second endpoint’s local level depends on the first endpoint’s result. Why is marginal validity alone not the conditional-validity proof used for fallback?
- With initial allocations \(0.020\) and \(0.030\), \(H_1\) is not rejected. Under the fallback rule, what level is available to \(H_2\) if \(H_1\)'s allocation is not transferred after failure?
- What must be checked if an endpoint receives additional alpha after its sequential monitoring has already begun?
Solutions. (1) \(H_1\) rejects because \(0.018\leq0.020\), so \(H_2\)'s level becomes \(0.015+0.020=0.035\). It rejects because \(0.030\leq0.035\), passing \(0.035\) to \(H_3\), whose level becomes \(0.015+0.035=0.050\). Since \(0.040\leq0.050\), reject \(H_3\). (2) Fixed sequence reaches a later endpoint only after rejecting every earlier hypothesis, including the first true one. (3) The second endpoint’s threshold is selected using earlier data; its marginal null distribution does not establish the required rejection bound conditional on that data. (4) \(H_2\) remains at its initial level, \(0.030\); \(H_1\)'s \(0.020\) is not recycled. (5) Verify a valid boundary or conditional-error update for the remaining looks. A larger threshold applied to ordinary interim p-values is not, by itself, a calibrated sequential test.