Why Allocate Alpha Across Arms and Interim Decisions?
A trial with several candidate treatments may examine efficacy at more than one decision time. Each opportunity to reject a null hypothesis creates a possible route to a false efficacy claim. A prespecified allocation makes those routes explicit: it assigns portions of the trial’s type I error budget to the treatment hypotheses and the decisions at which they may be tested.
The goal is familywise error rate (FWER) control. The FWER is the probability of making at least one false rejection among the treatment hypotheses in the family. Strong control means that this probability is at most the chosen significance level for every configuration of true and false null hypotheses, not only when all null hypotheses are true.
The allocation must cover every possible formal rejection route. For example, if efficacy can be declared for a treatment at an interim analysis or at the final analysis, both are opportunities. An interim rule used only to rank or drop arms, with no efficacy claim, is not by itself a false-rejection event. However, that rule can affect the later analysis, so the final test must remain valid for the adaptive design that actually produced the data.
A useful construction separates arm weights from decision-time weights. Choose nonnegative weights \(w_i\) for the arms and \(v_k\) for the opportunities, with \(\sum_i w_i=1\) and \(\sum_k v_k=1\). Then set \(\alpha_{ik}=\alpha w_i v_k\). This product allocation is one convenient option, not a requirement: allocations may instead be specified directly, provided their sum does not exceed \(\alpha\).
Why the Allocation Controls Error
Let \(R_{ik}\) denote the event that the test of \(H_i\) at opportunity \(k\) rejects. If \(H_i\) is true, assume that its test is valid at level \(\alpha_{ik}\), meaning \(P(R_{ik})\leq\alpha_{ik}\). This probability is evaluated under the trial’s design, including any adaptation that affects the test. Let \(I_0\) be the set of indices of true null hypotheses. A false efficacy claim occurs if at least one event \(R_{ik}\) occurs for an \(i\) in \(I_0\).
The key justification is the union bound. It does not require the tests to be independent, nor does it require the events to be disjoint.
This argument is a sufficient guarantee, and can be conservative when rejection events overlap. Its strength is that it applies under arbitrary dependence if each opportunity’s error bound is valid. In particular, it provides strong control: the argument holds for any subset of true null hypotheses.
There are two common ways to interpret an allocation. In a simple Bonferroni plan, \(\alpha_{ik}\) is the nominal level for a marginal test at opportunity \((i,k)\); a valid p-value is compared with that level. Alternatively, a sequential design can calibrate its boundaries so that the probability of crossing a boundary over a specified set of looks is at most an allocated amount. These approaches must not be mixed casually: the stated error bound must match the procedure actually used.
Worked Example: Splitting Alpha Across Three Arms and Two Looks
Suppose a hypothetical trial has three one-sided treatment hypotheses and a total FWER target of \(\alpha=0.05\). Assign arm weights \(w_1=0.50\), \(w_2=0.30\), and \(w_3=0.20\). Split each arm’s allocation across an interim and a final testing opportunity using weights \(v_1=0.40\) and \(v_2=0.60\). Assume all six marginal p-values are valid under their respective null hypotheses under the trial design.
The cell allocation is \(\alpha_{ik}=0.05w_iv_k\). For arm 1, the interim and final levels are \(0.05(0.50)(0.40)=0.0100\) and \(0.05(0.50)(0.60)=0.0150\). For arm 2, they are \(0.00600\) and \(0.00900\). For arm 3, they are \(0.00400\) and \(0.00600\).
The total allocated error is \(0.0100+0.0150+0.00600+0.00900+0.00400+0.00600=0.0500\). Suppose the hypothetical p-values at the interim and final decisions are, respectively, \(0.0080\) and \(0.0130\) for arm 1; \(0.0070\) and \(0.0080\) for arm 2; and \(0.0030\) and \(0.0070\) for arm 3.
Arm 1 passes both comparisons: \(0.0080\leq0.0100\) and \(0.0130\leq0.0150\). Arm 2 does not pass the interim comparison because \(0.0070>0.00600\), but it passes the final comparison because \(0.0080\leq0.00900\). Arm 3 passes the interim comparison because \(0.0030\leq0.00400\), but does not pass the final comparison because \(0.0070>0.00600\). These decisions follow the allocated thresholds; the FWER guarantee follows from the total allocation and validity assumptions, not from how many of these particular p-values pass.
Choosing Weights and Handling Selection
Weights express the prespecified priorities of the testing plan. Equal arm weights are simple when all treatments have equal confirmatory priority. Unequal weights can reserve more of the budget for a primary candidate or a particularly important decision time. A larger weight gives an opportunity a less stringent threshold, but it does not change the total budget. Weights should be set before examining unblinded results; choosing them in response to observed p-values would invalidate the stated calculation unless the selection is itself handled by a valid procedure.
An arm may be selected for a later analysis based on interim data. If the planned rule can reject only for selected arms, a false rejection for a true null is still a member of the set of possible false-rejection events covered by the allocation. A conservative way to account for this is to reserve budget for every arm and every formal testing opportunity that could occur, even if some arms will later be dropped. Unused allocations do not automatically become available elsewhere; any reallocation or recycling needs to be specified and justified as part of the procedure.
Selection alone does not guarantee that a later p-value is valid. For example, if the final p-value is computed using a reference distribution that ignores a data-dependent change in the design, its null rejection probability may exceed the allocated level. The allocation argument requires validity under the actual adaptive design, not just under an imagined fixed design. The earlier material on selection events and their sampling distributions motivates why this distinction matters.
Worked Example: Allocating a Budget When Only Two Arms Can Continue
Consider a hypothetical trial with four candidate arms and an overall target \(\alpha=0.04\). Before interim data are reviewed, allocate arm weights \(0.40, 0.30, 0.20,\) and \(0.10\), and split each arm’s allocation between interim and final efficacy decisions using time weights \(0.25\) and \(0.75\). The resulting interim and final levels are: arm 1, \(0.00400\) and \(0.0120\); arm 2, \(0.00300\) and \(0.00900\); arm 3, \(0.00200\) and \(0.00600\); and arm 4, \(0.00100\) and \(0.00300\). Their sum is \(0.0400\).
Suppose the interim selection rule retains arms 1 and 3, and the trial’s valid final p-values for those arms are \(0.0100\) and \(0.0050\), respectively. Arm 1 passes its final threshold because \(0.0100\leq0.0120\). Arm 3 passes because \(0.0050\leq0.00600\). No final efficacy test is performed for arms 2 and 4.
The two observed rejections do not imply that the trial has spent \(0.0120+0.00600=0.0180\) of its error budget in the probabilistic sense, nor do the dropped arms’ allocations automatically transfer to the selected arms. The prespecified error bound still covers every possible route: all four interim opportunities and all four possible final opportunities. Assuming the tests at those opportunities are valid under the adaptive design, the union bound gives FWER at most \(0.0400\), including when some null hypotheses are false.
Interim Decisions, Diagnostics, and Alternatives
A practical plan should make clear what each interim action means. A futility decision that merely stops an arm and makes no efficacy claim is not an additional rejection opportunity in the FWER calculation. An interim efficacy decision that permits a confirmatory claim is. If an interim test is used only to select an arm, the selection rule may still affect final-test validity, even though selection itself is not an efficacy rejection.
Useful checks include verifying that all allocations are nonnegative, that their sum is at most \(\alpha\), and that the plan includes every path to a formal claim. Also verify that each p-value or boundary has the stated null error probability under the actual data collection and adaptation rules. A simulation under the null can be a useful diagnostic for a complicated design: repeat the full decision process under representative null configurations and estimate the proportion of simulated trials with at least one false rejection. Simulation is a check on implementation and calibration, not a substitute for a valid argument when an exact guarantee is required.
The main tradeoff is simplicity versus efficiency. Direct alpha splitting is transparent and robust to dependence but may be conservative when test statistics are correlated or rejection events overlap. Correlation-aware procedures such as Dunnett or MaxT calibration can use the joint distribution to set less conservative thresholds, provided the joint model and calibration are appropriate. Sequential boundary methods can calibrate error across repeated looks jointly rather than applying a simple sum of marginal levels. These alternatives require their own assumptions and calibration; an allocation table by itself does not establish their validity.
| Approach | What is allocated or calibrated | Key consideration |
|---|---|---|
| Bonferroni alpha allocation | A level for each arm-by-decision testing opportunity | Simple and robust to dependence when each test is valid; may be conservative |
| Joint correlation-aware testing | A shared threshold based on the joint distribution of test statistics | Can gain power, but depends on appropriate joint calibration |
| Sequential boundary calibration | A crossing-probability budget across repeated looks | Must account for the full sequence of looks and the design’s adaptation |
Common Mistake
A frequent error is to divide \(0.05\) among treatment arms and then test each arm at that level at both interim and final analyses. For example, assigning \(0.025\) to each of two arms but using \(0.025\) at each of two looks creates four potential rejection events whose nominal levels sum to \(0.10\), not \(0.05\). Dependence between looks does not make that sum a valid upper bound of \(0.05\).
The correction is to allocate across both dimensions, such as assigning \(0.025\) per arm and splitting each arm’s amount across looks, or to use a sequential procedure calibrated to the desired crossing probability. In either case, include every formal rejection route and verify that its test is valid under the adaptive design. Do not assume that a selection rule or a dropped arm makes unallocated error disappear.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A trial has three arms and two formal testing opportunities per arm. What condition on the six allocations is sufficient for an overall FWER target of \(0.05\)?
Solution: Require \(\alpha_{ik}\geq0\) for every cell and \(\sum_{i=1}^{3}\sum_{k=1}^{2}\alpha_{ik}\leq0.05\), along with valid tests at their allocated levels under each true null. - Why does the union-bound argument not require independent tests?
Solution: For any events \(R_{ik}\), \(P(\bigcup R_{ik})\leq\sum P(R_{ik})\) holds regardless of their dependence. Bounding each true-null event by its allocation therefore bounds the probability of at least one false rejection. - A futility rule drops an arm without making an efficacy claim. Does that action, by itself, require an additional FWER allocation?
Solution: No. A non-efficacy stopping or selection action is not itself a false rejection. However, any later test affected by that action must still have a valid error bound under the adaptive design. - Two arms each receive \(0.02\), and each is tested at both interim and final analyses using \(0.02\) at each look. Does this establish FWER control at \(0.05\)?
Solution: No. The four opportunities have allocations summing to \(4(0.02)=0.08\), so the union-bound argument only gives an upper bound of \(0.08\). The levels must be divided across looks or replaced by a separately calibrated sequential procedure. - What must be checked before applying allocated thresholds to p-values after an adaptive interim decision?
Solution: Check that each p-value is valid under its null for the actual adaptive design and decision rule. A p-value valid only for a fixed, nonadaptive analysis does not automatically retain its error bound after adaptation.