Why Cumulative Alpha Needs Its Own Interpretation
In a group sequential design, a study may examine accumulating data at several interim looks and stop early if the evidence is sufficiently strong. The overall type I error is the probability of making at least one false efficacy rejection under the null hypothesis \(H_0\). That probability concerns the whole sequence of looks, not a single test considered in isolation.
A cumulative alpha-spending function describes how much of this overall error probability has been used by a given point on the study’s information scale. Its value at a look is not automatically the error allocated only to that look. The distinction matters when interpreting a design, comparing spending patterns, or checking whether reported look-specific probabilities are consistent with the overall error target.
Information time is often expressed as the fraction of planned total information accrued. Thus, \(t=0.5\) means half the planned information has accrued; it does not necessarily mean half the planned calendar duration has passed. The actual information measure depends on the design. For interpreting a spending function, the essential point is that \(A(t)\) is indexed by information, not simply by the number of looks or by elapsed time.
To give the definition a probability interpretation, imagine applying the design’s efficacy rule under \(H_0\). The event “a false efficacy rejection has occurred by time \(t\)” includes any rejection at or before \(t\). Consequently, these events grow as \(t\) increases, and their probabilities cannot decrease. The function describes a cumulative target; by itself, it does not specify the numerical efficacy boundary at any look.
Cumulative Error and Error at One Look
Suppose the planned looks occur at information times \(0=t_0<t_1<\cdots<t_K=1\). Let \(C_i\) be the event that efficacy has been declared by look \(i\). These events are nested: \(C_{i-1}\) is contained in \(C_i\). If the design is calibrated to the spending function, then \(P(C_i\mid H_0)=A(t_i)\).
The probability of a first false rejection at look \(i\) is the probability that a rejection has occurred by look \(i\), but not by the preceding look. Because the events are nested, this is the difference between the two cumulative probabilities:
This difference is an unconditional probability of first crossing at look \(i\), under \(H_0\). It is not the probability of crossing at look \(i\) conditional on having reached that look without rejection. Those are different quantities: a conditional probability would divide by the probability of surviving earlier looks. Nor is the cumulative value \(A(t_i)\) the probability attributable only to look \(i\).
The marginal probability that the statistic at look \(i\) alone exceeds its efficacy boundary is another quantity. It can include data paths that would already have crossed a boundary at an earlier look. Therefore, it need not equal the first-crossing probability, and marginal look-specific probabilities should not be added as though they were mutually exclusive.
Worked Example: Reading Cumulative Values Across Four Looks
Consider a hypothetical one-sided design with overall significance level \(\alpha=0.05\). Suppose its spending function has the following target values at four planned information times:
| Look \(i\) | Information time \(t_i\) | Cumulative spending \(A(t_i)\) |
|---|---|---|
| 1 | 0.25 | 0.002 |
| 2 | 0.50 | 0.008 |
| 3 | 0.75 | 0.025 |
| 4 | 1.00 | 0.050 |
At the second look, \(A(0.50)=0.008\) means the target probability under \(H_0\) of having made a false efficacy rejection by that point is 0.008, or 0.8%. It does not mean that look 2 alone has been assigned 0.8%.
The target probability of a first false rejection at look 2 is the increase since look 1:
Thus, 0.6% is the unconditional first-crossing probability at look 2 under the null, if the design achieves these cumulative targets. By the final look the cumulative target is 0.050, the full 5% overall level. The differences across the four looks are 0.002, 0.006, 0.017, and 0.025; they sum to 0.050.
Worked Example: Why a Look’s Marginal Tail Probability Is Not Its Increment
Suppose in a hypothetical design the null probability of crossing at look 1 is 0.002. At look 2, imagine that the probability of the look-2 statistic exceeding its boundary is 0.007, and that the probability of both a look-1 crossing and a look-2 exceedance is 0.001. These are stipulated probabilities for illustrating the distinction; they are not data from a real trial.
The probability of a rejection by look 2 is the probability of the union of the two crossing events. Using the union formula:
where \(B_2\) denotes the event that the look-2 statistic exceeds its boundary. The cumulative probability by look 2 is 0.008. The probability of first crossing at look 2 is \(0.008-0.002=0.006\), not 0.007. The difference arises because the look-2 marginal event includes some paths, with probability 0.001, that already crossed at look 1.
The Shape of a Spending Function
A spending function describes when the overall error probability is targeted to accumulate. Two functions can have the same starting and ending values but different intermediate values. They therefore express different early-versus-late spending patterns, even though both target the same final \(\alpha\).
Worked Example: Comparing Two Hypothetical Spending Curves
For illustration, consider two increasing functions on \(0\leq t\leq1\), each ending at \(\alpha=0.05\):
At information time \(t=0.5\), their cumulative targets are:
The first curve targets 2.5% cumulative error by halfway through the planned information, while the second targets 1.25%. At \(t=1\), both equal 0.05. These calculations compare cumulative targets; they do not provide the efficacy boundary values or imply that a particular boundary method will achieve them without suitable calibration.
If looks occur at \(t=0.25,0.50,1.00\), the increments for \(A_1\) are 0.0125, 0.0125, and 0.025. For \(A_2\), they are \(0.003125\), \(0.0125-0.003125=0.009375\), and \(0.05-0.0125=0.0375\). In each case the increments sum to 0.05, but their timing differs. This is a comparison of two hypothetical curves, not a recommendation that either curve is appropriate for every design.
What the Function Does—and Does Not—Determine
A spending function gives cumulative error targets over information time. To turn those targets into usable efficacy boundaries, a design also needs a statistical model for the sequence of test statistics under \(H_0\), a specified look schedule, and a rule for calibrating the boundary values. The joint behavior of the statistics matters because results from successive looks are related. The same cumulative target can require different numerical boundaries under different statistical models or schedules.
This is why a cumulative target should not be treated as a list of independent per-look significance levels. Adding nominal tail probabilities from separate looks generally does not give the probability of at least one false rejection: crossing events can overlap. The cumulative probability is the probability of the union of those events. The earlier material on overall error under a null path provides the central perspective: evaluate the complete sequence, not each look as if it were the only analysis.
Spending functions offer a way to state targets at information times, rather than only at a single fixed set of look numbers. But a curve alone does not guarantee that a design’s implemented boundaries achieve those targets under every timing or model circumstance. Calibration and the assumptions behind it still matter.
Checks and Common Misinterpretations
A proposed cumulative spending curve should at minimum be checked for basic coherence with its stated role. For a design intending to spend its full one-sided level by maximum information, check that \(A(0)=0\), that \(A(t)\) does not decrease as \(t\) increases, and that \(A(1)=\alpha\). At the planned looks, confirm that the reported cumulative targets are consistent with the curve and that their differences are nonnegative. These checks do not establish that the resulting boundaries are correctly calibrated; they catch simple inconsistencies before that more involved evaluation.
- Confusing cumulative and look-specific error. Saying “look 3 uses \(A(t_3)\)” treats error accumulated by look 3 as if it all occurred there. The first-crossing probability at look 3 is \(A(t_3)-A(t_2)\), when the cumulative targets are achieved.
- Assuming alpha is spent evenly. Equal increments follow only if the chosen function and look times produce equal differences. A spending function need not spend equal amounts at successive looks.
- Using calendar time as information time without justification. A look halfway through the calendar schedule need not be at \(t=0.5\). Determine information time using the design’s information measure.
- Treating the curve as the boundary. \(A(t)\) is a probability target, not a critical value for the test statistic or a nominal p-value cutoff. Boundary construction requires additional design information.
These distinctions are useful when comparing a spending-based design with fixed classical boundaries or with analyses that apply a nominal cutoff at every look. A spending function makes the cumulative error target explicit across information time; it does not remove the need to account for repeated looks or to calibrate the procedure. Its limitation is equally important: the function’s shape expresses a target pattern, not a complete analysis plan.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A design has \(A(0.4)=0.006\) and \(A(0.7)=0.021\). What is the target cumulative probability by \(t=0.7\), and what is the target first-crossing probability between these looks?
- Why is a cumulative spending value at a look not automatically the probability attributable only to that look?
- For \(A(t)=0.04t\), calculate the cumulative target at \(t=0.25\) and \(t=1\). What probability increment is targeted between those times?
- A proposed curve has \(A(0.3)=0.012\) and \(A(0.6)=0.009\). Identify the basic problem.
- State one additional ingredient needed to convert cumulative targets into numerical efficacy boundaries.
Solutions. (1) The cumulative target by \(t=0.7\) is 0.021, or 2.1%. The first-crossing increment since \(t=0.4\) is \(0.021-0.006=0.015\), or 1.5%. (2) The cumulative event includes any earlier rejection; its probability is for the union of crossing events through that look. The look-specific first-crossing probability is the increase over the preceding cumulative value. (3) \(A(0.25)=0.04(0.25)=0.010\), and \(A(1)=0.040\). The increment is \(0.040-0.010=0.030\). (4) The function decreases from 0.012 to 0.009 as information time increases, violating the nondecreasing requirement for cumulative probability. (5) One needs, for example, the null model for the joint sequence of test statistics, along with the look schedule, to calibrate boundaries that achieve the targets.