Why a Spending Schedule Needs an Audit
A futility-spending schedule assigns portions of a type II error budget to the possibility of stopping for futility at interim looks. The schedule is an input to boundary construction, not merely a list of numbers to copy into software. If its cumulative targets decrease, or if its increments use more probability than remains in the budget, no sequence of boundaries can realize those targets as stated.
The audit begins with a precise interpretation of the numbers. A cumulative target is the probability, under a specified design alternative, of having stopped for futility by a given look. An increment is the probability assigned to stopping for the first time at that look. These are unconditional probabilities over the trial’s possible paths; they are not automatically conditional probabilities among participants or trials that reach a look.
The schedule must refer to one consistent set of looks, one design alternative, and one stopping-rule convention. In a design with a total type II error target \(\beta\), futility stopping under the design alternative contributes to failure to reject, so the cumulative futility probability cannot exceed the available type II error budget. A design may reserve only part of that budget for futility; call that planned allocation \(\beta_F\), where \(0\leq\beta_F\leq\beta\). Then the schedule must stay within \(\beta_F\). If the schedule is intended to use the full futility allocation, its final cumulative target equals \(\beta_F\); otherwise it may be less.
Necessary Probability Checks
The basic checks follow from the meaning of the events, rather than from any particular formula for calculating boundaries. The events \(F_1,\ldots,F_K\) are mutually exclusive: a trial cannot first stop for futility at two different looks. Their cumulative unions can only grow as additional looks are included. Consequently, cumulative targets cannot decrease, and each increment must be nonnegative.
The last upper bound says that the increment at look \(k\) cannot exceed the budget remaining just before that look. Equivalently, adding the increment must not push cumulative spending above \(\beta_F\). These checks are necessary for a coherent schedule. They do not, by themselves, prove that the particular increments can be obtained from boundaries under the model for the sequential test.
For a given sequence of cumulative targets, compute the increments by differencing adjacent targets, including the baseline \(B_0=0\). For a given sequence of increments, compute running totals and compare each total with the budget. Doing both checks is a useful way to catch transcription errors: the cumulative and incremental columns should reproduce each other exactly, apart from explicitly stated rounding.
Worked Example: Finding a Decreasing Cumulative Target
Suppose a hypothetical design uses four interim looks and has a futility allocation \(\beta_F=0.20\). Its proposed cumulative targets are \(B_1=0.05\), \(B_2=0.12\), \(B_3=0.10\), and \(B_4=0.18\). Determine whether the schedule is coherent.
Calculate the increments from the cumulative targets:
The third increment is negative. That would require a probability of first stopping at look 3 equal to \(-0.02\), which is impossible. The fact that the final target, \(0.18\), is below the budget does not repair the decrease at look 3.
The targets need to be reviewed against the design intent and source data. For example, if the intended third target was \(0.15\), the resulting increments would be \(0.05, 0.07, 0.03, 0.03\), all nonnegative, and the final total would remain \(0.18\). That correction is appropriate only if \(0.15\) is the intended target; silently changing a target to make the arithmetic work changes the design.
Checking the Remaining Budget
A schedule can be nondecreasing and still overspend. At each look, record both the budget available immediately before the look and the increment proposed for that look. The remaining budget is \(\beta_F-B_{k-1}\). If the increment is larger than this amount, the new cumulative target exceeds \(\beta_F\).
Worked Example: An Increment Exceeds the Available Budget
Consider a hypothetical design with \(\beta_F=0.25\) and proposed increments \(b_1=0.08\), \(b_2=0.09\), and \(b_3=0.10\). Check the schedule look by look.
At look 1, the available budget is \(0.25-0=0.25\), so \(0.08\) is within budget. The cumulative target is \(B_1=0.08\).
At look 2, the available budget is \(0.25-0.08=0.17\), so \(0.09\) is within budget. The cumulative target is \(B_2=0.08+0.09=0.17\).
At look 3, only \(0.25-0.17=0.08\) remains, but the proposed increment is \(0.10\). Thus,
The schedule is invalid as specified. Reducing the third increment to \(0.08\) would make the arithmetic fit exactly, but that adjustment must be justified by the design plan; it cannot be treated as a harmless computational fix.
A useful audit table makes the check explicit. The “remaining before look” entry is calculated from the cumulative target at the previous look, not from the current target. The table also makes clear whether a schedule uses the full allocation or leaves some unused.
| Quantity | Calculation | Required check |
|---|---|---|
| Increment at look \(k\) | \(b_k=B_k-B_{k-1}\) | \(b_k\geq 0\) |
| Remaining before look \(k\) | \(\beta_F-B_{k-1}\) | Must be nonnegative |
| Target after look \(k\) | \(B_k=B_{k-1}+b_k\) | \(B_k\leq\beta_F\) |
| Final allocation used | \(B_K=\sum_{k=1}^{K}b_k\) | \(B_K\leq\beta_F\); equality only if full spending is intended |
Coherent Arithmetic Does Not Guarantee Attainability
Passing the monotonicity and budget checks means that the probability accounting is coherent. It does not establish that boundaries exist that produce exactly those increments. The distribution of test statistics across looks, the information times, the design alternative, and the stopping rules determine which first-passage probabilities can be achieved. Statistics across repeated looks are generally dependent, so a set of marginal probabilities alone is not enough to calculate the probability of first crossing at each look.
Worked Example: A Budget-Valid Schedule Still Needs Model Checks
Suppose a hypothetical three-look design has \(\beta_F=0.25\) and cumulative targets \(B_1=0.04\), \(B_2=0.12\), and \(B_3=0.23\). Check the schedule’s probability accounting.
The increments are
Each increment is nonnegative, and the final cumulative target \(0.23\) is below \(0.25\). The remaining budgets before the three looks are \(0.25\), \(0.21\), and \(0.13\), respectively; each proposed increment fits its available amount. The schedule is therefore coherent as a probability allocation, with \(0.02\) of the futility allocation unused.
This arithmetic does not show that boundaries under the design’s joint sampling model will attain \(0.04\), \(0.08\), and \(0.11\) as first-stop probabilities. That requires checking the model and the boundary calculation. If no boundary can deliver a requested target under the stated constraints, the schedule may be budget-valid but unattainable as specified.
It is also important to distinguish unconditional increments from conditional stopping probabilities. If \(q_k\) is the probability of stopping for futility at look \(k\), conditional on reaching that look while still eligible for the decision, then in a simple representation \(b_k=r_kq_k\), where \(r_k\) is the probability of being in that risk set. Conditional probabilities \(q_k\) generally cannot be added directly to obtain cumulative spending. They must first be translated into unconditional first-stop probabilities using the appropriate risk-set probabilities and stopping-rule convention.
A Practical Diagnostic Sequence
A disciplined audit catches both arithmetic errors and mismatches in what the schedule numbers mean. Check the definitions and inputs before trying to repair a failed schedule.
Verify that every entry is a probability on the same scale, refers to the same design alternative \(\theta^*\), and uses the same futility allocation \(\beta_F\). Do not combine targets from different alternatives or stopping-rule conventions.
Confirm that entries are indexed by the intended analyses in increasing information order. A row shift or a target assigned to the wrong look can create an apparent decrease or overspend.
Set \(B_0=0\) and calculate \(b_k=B_k-B_{k-1}\). Any negative increment signals a decreasing cumulative target or an error in the data, indexing, or rounding.
At each look calculate \(\beta_F-B_{k-1}\). Confirm that the increment is no larger than this amount and that every running total stays within the allocation.
Only after the accounting passes, assess whether the joint model and boundary procedure can attain the requested first-stop probabilities. Do not label this model-based check as a simple budget check.
Schedules that pass these checks can still differ in shape. A front-loaded allocation assigns more probability to earlier looks; a back-loaded allocation assigns more to later looks. Either shape may be coherent if its cumulative targets are nondecreasing and within the allocation. The schedule’s shape is a design choice, whereas monotonicity and the budget limit are probability requirements.
Common Mistakes and Limitations
- Checking only the final total. A final total below \(\beta_F\) does not excuse a negative intermediate increment. Every look must pass the checks.
- Treating an increment as a cumulative target. Increments are first-stop probabilities at individual looks; cumulative targets are running sums.
- Adding conditional stopping probabilities as if they were increments. Conditional probabilities apply to a risk set. Convert them to unconditional probabilities before comparing their sum with the cumulative budget.
- Changing numbers to force validity. Truncating a target or reallocating an overshoot changes the planned design. Confirm the intended schedule before making a correction.
- Assuming arithmetic validity proves a boundary exists. Budget checks are necessary but do not establish attainability under the joint distribution and design constraints.
Rounding can create small apparent decreases or overshoots when values have been printed to limited precision. Recheck the unrounded values when available. If only rounded entries are available, report that the schedule cannot be certified from the displayed precision alone rather than treating a negative increment as a real probability or silently removing it. The central diagnostic remains simple: a valid cumulative probability cannot go down, and spending cannot exceed the probability budget assigned to it.
Check Your Understanding
Use the definitions and checks above. Solutions follow the questions.
- With \(\beta_F=0.18\), cumulative targets are \(0.03, 0.09, 0.16\). Calculate the increments and state whether the schedule passes the basic accounting checks.
- With \(\beta_F=0.20\), increments are \(0.06, 0.08, 0.07\). At which look does the schedule first fail, and by how much?
- Why does a decrease from \(B_2=0.11\) to \(B_3=0.10\) imply an impossible increment, even if \(B_3\) is below the total budget?
- A schedule lists conditional futility probabilities at each look. Can they be summed directly and treated as cumulative spending? Explain.
Solutions. (1) The increments are \(0.03\), \(0.09-0.03=0.06\), and \(0.16-0.09=0.07\). They are all nonnegative and sum to \(0.16\leq0.18\), so the basic accounting checks pass. (2) The running totals are \(0.06\), \(0.14\), and \(0.21\). The third increment exceeds the remaining \(0.20-0.14=0.06\) by \(0.01\), so the schedule first fails at look 3. (3) The implied increment is \(0.10-0.11=-0.01\), but a first-stop probability cannot be negative. A budget-compliant final value does not fix that contradiction. (4) No. Conditional probabilities are defined among trials in a risk set and are not unconditional first-stop probabilities. Translate them using the relevant probabilities of reaching each look before summing or comparing with the cumulative budget.