Alpha-Spending Functions for Multi-Arm Boundaries

Why Spend Error Across Arms and Stages?

A multi-arm, multi-stage trial can produce a false positive in more than one way: an ineffective arm may cross its boundary at an early look, a later look, or after other arms have been dropped. The type I error budget must therefore describe the whole monitoring rule, not just the significance level of one test at one stage. Alpha spending gives a way to specify how much cumulative error is available over time and how that budget is assigned across arms or intersection hypotheses.

The key distinction is between a budget and a boundary. A spending function specifies a permitted cumulative probability of crossing by a given stage. A boundary is then chosen so the probability of crossing under the relevant null hypothesis respects that budget. When the statistics are correlated, the cumulative crossing probability depends on their joint distribution; it is not generally obtained by adding their marginal crossing probabilities exactly.

Definition: Let \(K\) be the number of planned stages, \(t_k\) the information fraction at stage \(k\), with \(0<t_1<\cdots<t_K=1\), and \(Z_{jk}\) the one-sided test statistic for arm \(j\) at stage \(k\), with larger values favoring that arm. A cumulative alpha-spending function \(A_j(t)\) for arm \(j\) is a nondecreasing function with \(A_j(0)=0\) and \(A_j(1)\leq\alpha_j\), where \(\alpha_j\) is the error budget assigned to that arm. The stage-\(k\) spending increment is \(A_j(t_k)-A_j(t_{k-1})\), taking \(A_j(t_0)=0\).

A spending increment is an amount of cumulative error budget, not automatically the probability of crossing at that stage. Because observations across stages are correlated, the probability of crossing a stage-\(k\) boundary depends on whether earlier boundaries were crossed. Accordingly, boundaries are chosen against a cumulative crossing event.

Armwise Spending and a Correlation-Robust Guarantee

Suppose the null hypothesis for arm \(j\) is \(H_j\), and let \(c_{jk}\) be its boundary at stage \(k\). The event that arm \(j\) rejects by stage \(k\) is \(\bigcup_{\ell=1}^{k}\{Z_{j\ell}\geq c_{j\ell}\}\). A standard armwise spending requirement is that, under \(H_j\), the probability of this event is no greater than \(A_j(t_k)\). If it is calibrated to equal \(A_j(t_k)\), the spending schedule is met exactly at that look; if equality is not attainable, a conservative boundary can keep the probability below the target.

If the arm budgets satisfy \(\sum_j\alpha_j\leq\alpha\), then strong familywise error control follows from the union bound, provided each true null arm has its stated armwise error bound. This guarantee does not require independence. Correlation may make the bound conservative, but it does not invalidate it.

P(\text{at least one true } H_j \text{ is rejected}) \leq \sum_{j\in I_0} P(\text{arm }j\text{ is rejected}) \leq \sum_{j\in I_0}\alpha_j \leq \sum_{j=1}^{J}\alpha_j \leq \alpha ,

Here \(I_0\) denotes the set of truly null arms. The first inequality holds for any dependence among test statistics. The second uses the armwise error guarantee under each true null. This approach is simple and robust, but it can be conservative because it does not use the correlation to reduce the probability of at least one false rejection.

Worked Example: Allocating Arm and Stage Budgets

Consider three one-sided arm tests and a total familywise level of \(\alpha=0.05\). Assign arm weights \(w_1=0.50\), \(w_2=0.30\), and \(w_3=0.20\), so that \(\alpha_j=w_j\alpha\). Use the linear spending function \(A_j(t)=\alpha_j t\) at information fractions \(t_1=0.25\), \(t_2=0.60\), and \(t_3=1\).

The arm budgets are \(\alpha_1=0.025\), \(\alpha_2=0.015\), and \(\alpha_3=0.010\). For arm 1, cumulative spending at the three looks is \(0.025(0.25)=0.00625\), \(0.025(0.60)=0.0150\), and \(0.025(1)=0.0250\). Its increments are therefore \(0.00625\), \(0.0150-0.00625=0.00875\), and \(0.0250-0.0150=0.0100\).

For arm 2, cumulative spending is \(0.00375\), \(0.00900\), and \(0.0150\), giving increments \(0.00375\), \(0.00525\), and \(0.00600\). For arm 3, cumulative spending is \(0.00250\), \(0.00600\), and \(0.0100\), giving increments \(0.00250\), \(0.00350\), and \(0.00400\). The final arm budgets sum to \(0.025+0.015+0.010=0.050\). Boundaries still need to be calibrated so each arm's cumulative crossing probability respects its spending function; the increments alone do not specify those boundaries.

Joint Spending for Correlated Families

Instead of assigning separate budgets to arms, one can specify a cumulative budget for an intersection hypothesis. For a set of arms \(I\), define \(H_I=\bigcap_{j\in I}H_j\). Given boundaries \(c_{j\ell}^{(I)}\) for the local test of \(H_I\), the family crossing event by stage \(k\) is \(\bigcup_{j\in I}\bigcup_{\ell=1}^{k}\{Z_{j\ell}\geq c_{j\ell}^{(I)}\}\). A cumulative joint spending target \(B_I(t_k)\) requires the probability of this event under \(H_I\) to be no greater than \(B_I(t_k)\), with \(B_I(1)\leq\alpha\). If calibrated to equality at each look, \(B_I(t_k)-B_I(t_{k-1})\) is the increase in cumulative crossing probability from the previous look to look \(k\).

For a correlated family, joint calibration uses the joint null distribution of the statistics in the local test. The boundary vector must account for correlation across arms and stages; separate marginal quantiles do not, by themselves, determine the probability of any crossing. The earlier material on closed testing provides the framework for strong control: a hypothesis is rejected only when the relevant intersection hypotheses containing it are rejected by their local tests. Spending can be specified for those local tests, and each must maintain its stated level over all stages.

A spending rule calibrated only under the global null controls the probability of any rejection when all arms are null. That is weak familywise control. To claim strong control—control for every configuration of true and false nulls—one needs valid local tests for the intersections used by closed testing, or conditions that justify transferring the global-null calibration to every true-null subset. Subset pivotality is one such condition when it applies; it should not be assumed without checking the model and testing procedure.

Worked Example: A Conservative Correlated-Family Boundary

Suppose two arms are monitored at two stages with a one-sided familywise level \(\alpha=0.05\). Allocate \(\alpha_1=0.03\) and \(\alpha_2=0.02\). Within each arm, assign half its budget to each stage, giving cell allocations \(q_{11}=q_{12}=0.015\) and \(q_{21}=q_{22}=0.010\). Their sum is \(0.015+0.015+0.010+0.010=0.050\).

For a simple conservative rule, choose each cell boundary so that its marginal null tail probability is no greater than its allocation: \(P_{H_j}(Z_{jk}\geq c_{jk})=q_{jk}\). If each \(Z_{jk}\) has a standard normal null distribution, the boundaries are \(c_{11}=c_{12}=\Phi^{-1}(0.985)\approx2.170\) and \(c_{21}=c_{22}=\Phi^{-1}(0.990)\approx2.326\), where \(\Phi\) is the standard normal cumulative distribution function.

Regardless of the correlations between the four statistics, the probability of any false rejection is at most the sum of the four allocated tail probabilities, which is \(0.050\). This rule is valid but can be conservative: it counts each possible arm-stage rejection separately and does not exploit dependence. It is a marginal-tail allocation, not an exact joint alpha-spending calibration.

From Spending Targets to Boundaries

The general translation is to choose critical values so the probability of crossing by each look is within the cumulative target. For armwise spending, that means controlling the probability of the maximum of that arm's statistics through the current stage. For joint spending, it means controlling the probability that any statistic in the specified local family has crossed. A critical value at look \(k\) therefore cannot ordinarily be computed by treating that look as the only test.

The following justification shows why cumulative constraints give an error guarantee. It also clarifies the distinction between a joint target and a collection of marginal targets.

Define the cumulative event.
For each relevant null or intersection hypothesis, include every arm and every stage monitored by its local test. The event is that at least one corresponding statistic crosses its boundary by the current look.

Set a nondecreasing spending target.
Specify cumulative targets from zero up to a final value no greater than the local test's assigned level. A cumulative target cannot decrease as stages are added, since the crossing event can only grow.

Choose boundaries under the null model.
Use marginal tail constraints for a conservative allocation, or the joint distribution for a less conservative calibration. The probability constraint applies to the cumulative event, not just the current statistic.

Apply the local tests in the familywise procedure.
For strong control, ensure the relevant intersection tests meet their error bounds and combine them using a valid procedure such as closed testing. Check the dependence assumptions used for any more powerful joint calibration.

P_{H_I}\!\left(\bigcup_{j\in I}\bigcup_{\ell=1}^{k} \{Z_{j\ell}\geq c_{j\ell}^{(I)}\}\right) \leq B_I(t_k) \quad\Longrightarrow\quad P_{H_I}(\text{local test rejects by stage }K) \leq B_I(1)\leq\alpha_I .

The implication follows because the event of rejection by the final stage is exactly the cumulative crossing event at \(K\). Thus, controlling that probability controls the local test's type I error. The same logic applies at each earlier look, which prevents the procedure from spending more cumulative error than planned at an interim analysis.

Worked Example: Reading a Boundary Crossing in Context

Use the two-arm, two-stage conservative rule above. Suppose a hypothetical trial produces standardized statistics \(Z_{11}=2.05\), \(Z_{12}=2.25\), \(Z_{21}=2.30\), and \(Z_{22}=2.10\). The arm 1 boundaries are approximately \(2.170\) at both stages, and the arm 2 boundaries are approximately \(2.326\) at both stages.

At stage 1, arm 1 does not cross because \(2.05<2.170\); arm 2 does not cross because \(2.30<2.326\). At stage 2, arm 1 crosses because \(2.25\geq2.170\), while arm 2 does not because \(2.10<2.326\). Under this rule, the arm 1 null is rejected at the second look. The familywise error guarantee comes from the preassigned tail allocations and the union bound, not from treating the observed statistics as independent.

This example illustrates the decision rule only. In an actual trial, the test statistics and their null distributions must match the specified analysis, and any closed-testing requirements for strong control must also be met.

Comparisons, Diagnostics, and Limitations

Armwise spending with a Bonferroni combination is a dependable option when a simple guarantee under arbitrary dependence is important. Its cost is potential conservatism, particularly when many arms or looks are included. Joint calibration can use known dependence to set less conservative boundaries, but depends on a correctly specified joint null distribution. A single-stage multiplicity procedure, such as a maximum-statistic approach, addresses multiplicity at one analysis; a multi-stage spending design must additionally account for repeated looks. These approaches serve different purposes and should not be treated as interchangeable.

Several checks help reveal when a spending design is being misapplied:

Check the cumulative schedule. The function must start at zero, be nondecreasing, and finish no higher than its assigned error level. Stage increments should be nonnegative and sum to the final cumulative budget.
Check what the allocation refers to. A marginal tail probability at one stage is not the same as a cumulative probability of crossing by that stage. Repeated looks require cumulative control.
Check the dependence used for joint boundaries. Shared controls and repeated measurements can induce correlation across arms and stages. Ignoring it can make a purported exact joint calibration incorrect.
Check the error-control claim. A boundary calibrated under the global null alone need not control error strongly across partial-null configurations. Use valid intersection tests or justify the conditions that permit the calibration to transfer.
Check changes to the monitoring plan. Adding a look, changing the information schedule, or changing which arms are included can alter the crossing event and invalidate boundaries tied to the original plan.

Alpha spending is a design principle, not a unique boundary shape. Different nondecreasing functions can allocate more error early or late, and different valid methods can translate those targets into boundaries. The schedule does not, by itself, establish power, guarantee a desired stopping pattern, or fix a misspecified test model.

Key takeaway: An alpha-spending function specifies a cumulative error budget. Valid boundaries must control the probability of crossing by each look under the relevant null. Dependence can be handled conservatively by allocating marginal error across arm-stage tests, or used in joint calibration when the joint null model and strong-control requirements are justified.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

An arm has an assigned level of 0.024 and uses linear spending at information fractions 0.40 and 1.00. What are its cumulative spending targets and stage increments?
Why does summing allocated arm-stage tail probabilities give a valid familywise bound even when the test statistics are correlated?
What does a joint cumulative target \(B_I(t_k)\) bound for an intersection hypothesis \(H_I\)?
Why is a boundary calibrated only under the global null not automatically a strong familywise procedure?

Solutions. (1) The cumulative targets are \(0.024(0.40)=0.0096\) and \(0.024(1)=0.024\). The increments are \(0.0096\) and \(0.024-0.0096=0.0144\). (2) The probability of a union of events is no greater than the sum of their probabilities, with no independence assumption; if each event's null probability is bounded by its allocation and the allocations sum to at most \(\alpha\), the familywise probability is at most \(\alpha\). (3) It bounds the probability, under \(H_I\), that any statistic included in that local test crosses its boundary by stage \(k\). (4) Strong control also covers configurations with some true and some false nulls. Global-null calibration alone does not guarantee the required error bound for every true-null subset; valid intersection tests or justified conditions such as subset pivotality are needed.