Why Weight the Boundaries?
In a multi-arm, multi-stage trial, investigators may have good design reasons to give some treatment arms more opportunity to demonstrate efficacy than others. An arm addressing a priority indication, for example, might receive a larger share of the familywise error budget. Unequal allocation can improve sensitivity for that arm, but it does not make the familywise error requirement disappear: the chance of falsely rejecting at least one true null hypothesis in the family must still be controlled.
The central safeguard is to choose the arm weights and stage allocations in advance, then apply the resulting boundaries whether or not an arm is retained at an interim analysis. The familywise error argument does not require independent arm statistics. It does require each true-null arm’s stagewise p-values to remain valid under the configurations in which that arm is null, including when other arms are effective.
Here, \(w_i\) expresses the arm’s prespecified priority in the family, and \(q_{i,k}\) divides that arm’s budget among its planned looks. The weights and stage allocations are design choices, not estimates from the observed treatment effects. An allocation may use less than the full budget; it may not spend more than the stated limits and retain the guarantee below.
Conditions for the Guarantee
Let \(H_i\) denote the null hypothesis for arm \(i\), and let \(p_{i,k}\) be its one-sided efficacy p-value at stage \(k\). A p-value is marginally valid under \(H_i\) if, for every \(u\) between 0 and 1,
For strong familywise error control, this validity must hold for each true \(H_i\) under every configuration of the other hypotheses, not only when all arms are null. For example, in a normal one-sided test, if \(Z_{i,k}\) has the standard normal distribution under the relevant null, \(p_{i,k}=1-\Phi(Z_{i,k})\) is marginally valid. The standard-normal assumption must actually be appropriate for the statistic being used; the formula alone does not ensure validity.
The arm statistics can be dependent, including through a shared control group. The argument also allows dependence across stages. It does not require the joint distribution of all the p-values to be known. But dependence does not excuse invalid marginal p-values: if adaptation, nuisance-parameter estimation, or a change in the data-generating process makes a true-null arm’s p-value nonuniform under some configuration, the guarantee cannot be claimed from this argument.
At an interim analysis, a rule may use observed data to decide which arms continue. For the guarantee, it must not silently change the fixed weights or give a retained arm additional error budget. Continuing an arm only permits the trial to use its already assigned future boundaries. The p-values used at those looks must still satisfy the validity condition under the design’s adaptations.
Deriving the Sequential Boundaries
Write \(\mathcal T\) for the set of indices of the true null hypotheses in a particular configuration. A false rejection occurs only if some \(i\in\mathcal T\) crosses a boundary. If \(\mathcal R_{i,k}\) is the event that arm \(i\) is selected for testing at stage \(k\) and its p-value crosses, then \(\mathcal R_{i,k}\subseteq\{p_{i,k}\leq\alpha w_iq_{i,k}\}\). Selection can only remove crossings from this event; it cannot add a crossing outside the fixed boundary. Therefore,
The first inequality is the union bound: the probability of a union of events is no greater than the sum of their probabilities. The next inequality follows from marginal validity at each stage; it does not assume independent tests. The final inequalities follow from the prespecified allocation constraints and nonnegative weights. This proves strong familywise error control: the bound holds for every configuration of true and false hypotheses.
If the stagewise statistic is a standard-normal \(Z_{i,k}\) with larger values favouring efficacy, the equivalent boundary is
This is a simple Bonferroni construction, not a joint-distribution calibration. Its conservatism can increase when there are many stages or strongly dependent looks, but its validity does not rely on modelling that dependence accurately.
Worked Example: Priority Weights Across Three Stages
Hypothetical design. A trial has three treatment arms and three planned looks, with one-sided familywise level \(\alpha=0.025\). The design assigns weights \(w_1=0.50\), \(w_2=0.30\), and \(w_3=0.20\). At each arm, stage allocations are \(q_{i,1}=0.50\), \(q_{i,2}=0.30\), and \(q_{i,3}=0.20\). Both sets of allocations sum to 1.
For arm 1, the total budget is \(\alpha w_1=0.025(0.50)=0.0125\). Its stage boundaries are \(0.0125(0.50)=0.00625\), \(0.0125(0.30)=0.00375\), and \(0.0125(0.20)=0.00250\). For arm 2, the corresponding boundaries are \(0.0075(0.50)=0.00375\), \(0.0075(0.30)=0.00225\), and \(0.0075(0.20)=0.00150\). For arm 3, they are \(0.0050(0.50)=0.00250\), \(0.0050(0.30)=0.00150\), and \(0.0050(0.20)=0.00100\).
Suppose stage 1 yields p-values \(0.007\), \(0.004\), and \(0.003\) for arms 1, 2, and 3. None crosses its stage 1 boundary: each p-value exceeds \(0.00625\), \(0.00375\), and \(0.00250\), respectively. Suppose the prespecified selection rule retains arms 1 and 3, and drops arm 2. At stage 2, suppose their p-values are \(0.0030\) for arm 1 and \(0.0014\) for arm 3. Both cross their fixed stage 2 boundaries: \(0.0030<0.00375\) and \(0.0014<0.00150\). Thus the design rejects \(H_1\) and \(H_3\) at stage 2. Arm 2’s unused budget is not reassigned. The familywise guarantee follows from the fixed allocations, not from which arms happened to continue.
Three Practical Checks
Before interpreting a weighted boundary, verify the design and the inferential inputs. These checks help identify whether the simple error bound applies.
- Check the budgets. Confirm that all weights are nonnegative, their sum is at most 1, and each arm’s stage allocations sum to at most 1. Record zero allocations as boundaries at which that arm cannot reject.
- Check the p-values under alternatives on other arms. A true-null p-value must be valid even when another treatment is effective. Validity established only under the global null is insufficient for strong control.
- Check what changes after selection. Selection may determine which fixed tests are carried out. If it changes sample sizes or the data used to construct later statistics, verify that the resulting p-values remain valid; otherwise the displayed Bonferroni calculation does not establish control.
An especially tempting shortcut is to redistribute the budget of dropped arms among the survivors. That changes the thresholds after looking at data. The proof above applies to the fixed thresholds and does not justify this reallocation. A procedure designed to update error budgets needs its own valid error-control argument; it cannot be assumed to inherit this one.
Worked Example: Comparing Priorities at One Look
Hypothetical one-stage comparison. Consider three arms tested once at familywise level \(\alpha=0.05\). Assign weights \(0.60\), \(0.30\), and \(0.10\). Since there is one look, set \(q_{i,1}=1\) for every arm. The respective p-value boundaries are \(0.05(0.60)=0.030\), \(0.05(0.30)=0.015\), and \(0.05(0.10)=0.005\).
Suppose the observed p-values are \(0.021\), \(0.014\), and \(0.006\). Arm 1 crosses because \(0.021<0.030\); arm 2 crosses because \(0.014<0.015\); arm 3 does not because \(0.006>0.005\). The priority weighting gives arm 1 a more permissive boundary and arm 3 a more stringent one. It still controls the familywise error rate at no more than 0.05 if the marginal validity assumptions hold.
For comparison, equal Bonferroni allocation would set each boundary to \(0.05/3\approx0.0167\). Under that allocation, arm 1 would not cross, arm 2 would cross, and arm 3 would not cross. The difference is a design trade-off in where the error budget is placed, not a difference in the overall bound.
Worked Example: Why Renormalizing After Selection Can Fail
Hypothetical global-null calculation. Suppose three independent, valid p-values are observed at an interim look when all three null hypotheses are true. A rule selects the arm with the smallest p-value, then tests that selected arm at 0.05, as though it had been the only arm considered. Under the global null, the probability of rejecting is
The actual familywise error rate is about 0.143, not 0.05. The problem is that selection used all three p-values, but the test treated the selected one as if no selection had occurred. By contrast, fixed weights \(0.50,0.30,0.20\) give boundaries \(0.025,0.015,0.010\). A rejection of the selected arm must still meet its own fixed boundary; the probability of a rejection is bounded above by the probability of any of the three fixed-boundary events, which is at most \(0.025+0.015+0.010=0.050\).
How This Approach Differs and Where It Is Limited
Equal Bonferroni allocation is the special case \(w_i=1/m\) with equal stage allocations. Weighted Bonferroni instead expresses priorities through fixed, unequal shares, while retaining a transparent bound that tolerates arbitrary dependence. A joint calibration using the dependence structure can sometimes yield less conservative boundaries, but it needs stronger modelling or calibration assumptions. The weighted construction is attractive when robustness and a clear audit trail matter more than extracting every possible increment in power.
The method’s main limitation is that its guarantee is only as good as the marginal p-values and the fixed allocation rules. It does not automatically correct for a p-value whose null distribution changes because of adaptive design choices. Nor does the union-bound proof show that the chosen weights are scientifically appropriate or optimal; it shows only that the stated allocations control error if their conditions are met.
Common mistake: “Only two arms remain, so I can divide the full familywise level between them now.” This reasoning ignores that the choice of those two arms used interim data. The correction is to keep using the weights and boundaries specified for the original family, or use a separately justified procedure that accounts for the adaptation. Dropping an arm does not, by itself, make its error budget available to the survivors.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- At \(\alpha=0.04\), an arm has weight \(0.25\) and two stage allocations of \(0.60\) and \(0.40\). What is its stage 2 p-value boundary?
- Why can the union-bound proof apply when treatment-arm statistics share a control group?
- Three weights are \(0.50\), \(0.35\), and \(0.20\). Do they satisfy the stated constraint? If not, give one simple correction.
- At stage 1, an arm is dropped. What happens to its unused error budget under fixed weighted Bonferroni boundaries?
Solutions. (1) The arm’s total budget is \(0.04(0.25)=0.010\); its stage 2 boundary is \(0.010(0.40)=0.004\). (2) The proof bounds the probability of a union by the sum of event probabilities and uses each p-value’s marginal validity; it does not assume independence. (3) The weights sum to \(1.05\), so they violate the constraint. One correction is to scale them by \(1/1.05\), giving weights approximately \(0.4762\), \(0.3333\), and \(0.1905\), which sum to 1. (4) The unused budget remains unused; fixed boundaries do not authorize reallocating it to retained arms.