Graphical Alpha Recycling in Sequential Multi-Arm Trials

Why Recycle Alpha Across Trial Stages?

A multi-arm trial may stop testing one treatment hypothesis after evidence against it is sufficiently strong. A fixed allocation of the familywise error budget leaves that arm’s unused allocation unavailable to the remaining arms. A graphical procedure can instead transfer some or all of the rejected hypothesis’s unspent alpha to other hypotheses, according to rules chosen before the trial.

In a sequential trial, those rules must work alongside repeated looks at the accumulating data. The graph determines how much error budget each active hypothesis holds; a sequential testing rule determines how that budget is spent across looks. Keeping these roles separate is essential: a valid graph does not make an invalid repeated-testing rule valid.

Definition: Let \(H_1,\ldots,H_m\) be the treatment hypotheses, and let \(\alpha\) be the target familywise error rate. A graphical alpha-recycling specification consists of initial weights \(w_i\) and transition weights \(g_{ij}\). The initial local alpha for \(H_i\) is \(a_i^{(0)}=\alpha w_i\), where \(w_i\geq 0\) and \(\sum_i w_i\leq 1\). The transition weight \(g_{ij}\geq 0\) is the fraction of the currently unspent local alpha for \(H_i\) transferred to \(H_j\) if \(H_i\) is rejected. Usually \(g_{ii}=0\) and \(\sum_j g_{ij}\leq 1\). Any fraction not transferred is discarded.

The weights and transitions are prespecified. At any time, \(a_i\) denotes the local alpha still available to hypothesis \(H_i\), not the amount already spent at earlier looks. A hypothesis that has been rejected is removed from further testing. If a transition points to a hypothesis already rejected, that portion is not available to another hypothesis unless a different transition rule was specified in advance.

Conditions for Sequential Validity

Suppose the trial has looks \(t=1,\ldots,T\), and let \(\mathcal{F}_{t-1}\) denote the observed trial history before look \(t\). To give a simple error-control argument, assume that each active \(H_i\) has a stagewise conditional p-value \(P_{i,t}\) satisfying, whenever \(H_i\) is true,

P(P_{i,t}\leq u\mid\mathcal{F}_{t-1})\leq u \qquad\text{for every }0\leq u\leq 1.

The test at look \(t\) uses a prespecified error debit \(\delta_{i,t}\), chosen using only the history available before that look, with \(0\leq\delta_{i,t}\leq a_i\). It rejects \(H_i\) if \(P_{i,t}\leq\delta_{i,t}\), and debits \(\delta_{i,t}\) from that hypothesis’s available alpha whether or not a rejection occurs. If \(H_i\) is rejected, its remaining alpha is then distributed by the graph. The debit may be zero at a look when no testing is planned.

These conditions require more than an ordinary p-value that is valid only at one final analysis. For example, one cannot repeatedly compare the same ordinary p-value to a local alpha at every look and infer sequential validity. Instead, the stagewise p-values and debit rule must be calibrated for the sequential design; conditional-error methods are one way to construct such rules. The earlier material on stagewise p-values and conditional-error recursion provides relevant foundations.

Assumptions for the error-control argument: The hypotheses, initial weights, transition weights, look schedule, and debit rules are prespecified. At each look, the debit is determined before observing that look’s test result. For every true null hypothesis, the stagewise conditional p-value is conditionally super-uniform given the prior history. The testing and recycling rules are followed as specified. No independence assumption between treatment-arm statistics is needed for the union-bound argument.

Updating the Graph and Its Budgets

At a look, first test each active hypothesis using its planned debit. Subtract each debit from its local alpha. If \(H_i\) is rejected, distribute its remaining amount \(a_i\) according to the row of transition weights for \(i\), then set \(a_i=0\). Thus, for each still-active \(H_j\), the update from rejecting \(H_i\) is \(a_j\leftarrow a_j+g_{ij}a_i\). If more than one hypothesis is rejected at the same look, apply the prespecified transfers for all of them; the ordering does not affect the result when transitions are applied to the remaining hypotheses according to the same fixed rule.

A transition row need not sum to one. If it sums to less than one, the difference is alpha that the design deliberately does not recycle. This can be useful when some hypotheses are less important or when the design aims to direct error only to selected arms. Transition weights should not be silently renormalized after an arm is rejected: changing them after seeing trial results changes the prespecified procedure.

Set the graph.
Choose the total \(\alpha\), initial weights, and transition weights before the trial. Check that initial weights sum to at most one and each transition row sums to at most one.

Plan stagewise debits.
For each look and active hypothesis, specify a debit no greater than its current local alpha. Use a sequentially valid conditional testing rule.

Test and debit.
Compare each stagewise conditional p-value with its debit. Subtract the debit from local alpha, regardless of whether the hypothesis is rejected.

Recycle after rejection.
Transfer the rejected hypothesis’s remaining alpha using its prespecified transition row. Do not transfer alpha already spent at earlier looks.

Worked Example: Specifying a Three-Arm Graph

Consider a hypothetical three-arm trial with \(H_1,H_2,H_3\) and familywise level \(\alpha=0.05\). Choose initial weights \((0.50,0.30,0.20)\). The initial local levels are

(a_1,a_2,a_3) =(0.05\times0.50,\;0.05\times0.30,\;0.05\times0.20) =(0.025,\;0.015,\;0.010).

Suppose the transitions are: after rejecting \(H_1\), send 60% of its remaining alpha to \(H_2\) and 40% to \(H_3\); after rejecting \(H_2\), send 50% to \(H_1\) and 50% to \(H_3\); after rejecting \(H_3\), send 25% to \(H_1\) and 75% to \(H_2\). Each row sums to one, so no alpha is discarded by the transition rule. The numbers specify destinations for remaining alpha, not for the original local levels.

If \(H_1\) is rejected after \(0.010\) alpha has been spent on it, its remaining amount is \(0.025-0.010=0.015\). The graph transfers \(0.60\times0.015=0.009\) to \(H_2\) and \(0.40\times0.015=0.006\) to \(H_3\). If neither other hypothesis has yet been tested, their resulting local levels are \(0.015+0.009=0.024\) and \(0.010+0.006=0.016\). The total remaining budget is \(0.040\), because \(0.010\) has already been spent.

Why the Procedure Controls Familywise Error

The familywise error rate (FWER) is the probability of rejecting at least one true null hypothesis. The key accounting fact is that the graph never creates alpha. Initially, the total available amount is at most \(\alpha\); debits reduce it, and transfers only move remaining amounts between hypotheses or discard them.

For a true \(H_i\), conditional super-uniformity gives \(P(P_{i,t}\leq\delta_{i,t}\mid\mathcal{F}_{t-1})\leq\delta_{i,t}\). Taking expectations bounds the probability of a false rejection of \(H_i\) at that look by the expected debit. A union bound over all looks and all true hypotheses therefore bounds FWER by the expected sum of their debits. On every path, the sum of all debits, including debits spent on false hypotheses, cannot exceed the initial total budget \(\alpha\). Consequently the sum restricted to true hypotheses cannot exceed \(\alpha\) either.

\begin{aligned} \mathrm{FWER} &\leq \sum_{i\in I_0}\sum_{t=1}^{T} P(\text{false rejection of }H_i\text{ at look }t)\\ &\leq \mathbb{E}\!\left[\sum_{i\in I_0}\sum_{t=1}^{T}\delta_{i,t}\right] \leq \alpha, \end{aligned}

Here \(I_0\) is the set of true null hypotheses. This argument gives strong control: it applies to any configuration of true and false hypotheses, not only the configuration in which every null is true. It also does not require independent test statistics. Its protection comes from conditional validity at each testing decision and the pathwise budget bound.

Worked Example: Two Looks With a Rejection and Recycling

Use the three-arm graph above. For illustration, at look 1 plan debits equal to 40% of each initial local level: \(\delta_{1,1}=0.010\), \(\delta_{2,1}=0.006\), and \(\delta_{3,1}=0.004\). Suppose the hypothetical conditional p-values are \(0.009\), \(0.008\), and \(0.0045\). Thus \(H_1\) is rejected because \(0.009\leq0.010\); \(H_2\) and \(H_3\) are not rejected because their p-values exceed their respective debits.

After the debits, the remaining amounts are \(0.015\), \(0.009\), and \(0.006\). Rejecting \(H_1\) transfers \(0.60\times0.015=0.009\) to \(H_2\) and \(0.40\times0.015=0.006\) to \(H_3\). The active hypotheses therefore start look 2 with \(a_2=0.018\) and \(a_3=0.012\).

Suppose the prespecified look-2 debits are one quarter of these amounts: \(\delta_{2,2}=0.0045\) and \(\delta_{3,2}=0.0030\). With hypothetical conditional p-values \(0.0040\) and \(0.0035\), respectively, \(H_2\) is rejected and \(H_3\) is not. The remaining amount on \(H_2\) is \(0.018-0.0045=0.0135\). Its transition row assigns half to \(H_1\) and half to \(H_3\). Since \(H_1\) has already been rejected, its half is unavailable; \(H_3\) receives \(0.50\times0.0135=0.00675\). No further testing is implied by this calculation; it shows how the prespecified graph updates the budget.

The total debited across these two looks is \(0.010+0.006+0.004+0.0045+0.0030=0.0275\), below \(0.05\). The difference is still available or has been discarded through transitions to an already rejected node; it is not evidence that the full familywise budget has been spent.

Alternatives, Diagnostics, and Limitations

A fixed weighted Bonferroni allocation uses initial levels such as \(\alpha w_i\) but does not increase the surviving arms’ allocations after another hypothesis is rejected. It is simpler to implement, but can leave useful error budget unused. A graphical procedure offers prespecified adaptation through recycling, at the cost of more bookkeeping and more demanding implementation checks. Closed testing is another route to strong FWER control, often expressing the procedure through tests of intersections; it can be less compact when many hypotheses are involved. These methods are not interchangeable without checking their testing rules and assumptions.

Several diagnostics help detect an implementation that departs from the intended method:

Budget audit: after every debit and transfer, verify that the sum of active local alpha does not exceed the initial total minus all debits and discarded amounts.
Transition audit: check each row sum and confirm that transfers to rejected hypotheses are handled exactly as prespecified.
Look-by-look audit: verify that each debit is fixed before the current look’s test result and that the conditional p-value is valid for the chosen sequential design.
Null-pattern audit: evaluate operating characteristics when different subsets of hypotheses are true. Simulation can reveal coding mistakes, but does not replace the conditional-validity argument.

A common error is to say, “\(H_1\) was rejected, so its original \(\alpha w_1\) can now be added to the other arms.” That transfers alpha already spent on \(H_1\), allowing the total budget to exceed \(\alpha\). The correction is to transfer only the amount remaining after all prior debits, and then to use the prespecified transition weights. Another serious error is to increase a surviving arm’s local alpha at an interim analysis while retaining boundaries calibrated only for its earlier fixed alpha. A new local level does not automatically yield a valid sequential test: the subsequent rule must satisfy the sequential validity conditions for the updated budget.

Graphical recycling does not decide which arms should be tested first, guarantee high power, or repair misspecified stagewise tests. It also cannot justify unplanned changes to transition weights after seeing results. Its role is narrower and precise: it allocates a finite familywise error budget among hypotheses according to prespecified rejection-triggered rules.

Worked Example: Spotting an Invalid Recycling Calculation

Suppose an active hypothesis has local alpha \(0.020\). At an earlier look, \(0.008\) was debited. A programmer rejects the hypothesis later and transfers its full original \(0.020\), rather than its remaining \(0.012\), to another arm. The transfer is too large by \(0.008\): that amount has already been spent and cannot be used again.

The valid calculation is \(0.020-0.008=0.012\) remaining. If the transition weight to the other arm is \(0.75\), that arm receives \(0.75\times0.012=0.009\), not \(0.75\times0.020=0.015\). The difference, \(0.006\), is improperly recycled under the programmer’s calculation. A budget audit would detect the discrepancy.

Check Your Understanding

Answer each question using the definitions and rules above. Concise solutions follow.

With \(\alpha=0.05\) and initial weights \((0.4,0.6)\), what are the initial local levels? Do they exhaust the initial budget?
A rejected hypothesis has \(0.012\) local alpha remaining. It transfers 70% to \(H_2\) and 30% to \(H_3\). How much does each receive?
Why is it not sufficient to use an ordinary level-\(\alpha\) p-value repeatedly at every look?
Does the budget argument require independent test statistics across treatment arms? State the key condition it does require.

Solutions. (1) The levels are \(0.05\times0.4=0.020\) and \(0.05\times0.6=0.030\); their sum is \(0.050\), so they exhaust the initial budget. (2) \(H_2\) receives \(0.70\times0.012=0.0084\), and \(H_3\) receives \(0.30\times0.012=0.0036\). (3) Repeated use of an ordinary p-value does not in general control the probability of crossing a threshold at any look; the sequential procedure needs valid stagewise conditional testing or an equivalent calibrated rule. (4) Independence is not required for the union-bound argument. It requires conditionally super-uniform stagewise p-values for each true null and a prespecified budget rule whose total debits never exceed the initial alpha.