Why the Final-Look Ledger Matters
In a multi-endpoint group sequential design, the final analysis does not automatically receive the endpoint’s original alpha allocation. Earlier looks may have used part of that allocation, and interim rejections may have changed how the remaining budget is distributed across endpoints. The final-look question is therefore a bookkeeping question with an inferential consequence: what error budget is still available to each endpoint, given the observed path through the design?
A ledger makes that path explicit. It records each endpoint’s allocation, the error allowance spent at each look, and any alpha transferred after an endpoint is rejected. The ledger does not by itself establish that a test boundary is valid. Its entries must correspond to a sequential testing procedure whose conditional error probabilities are properly controlled.
Here \(D_{i,k}\) is an error allowance for that sequential testing opportunity, not necessarily the ordinary nominal significance level printed beside a test statistic. In particular, a group sequential boundary is generally calibrated using the joint distribution across looks. If a design uses conditional error control, the debit must bound the conditional probability of a false rejection at that look. If it uses another valid spending construction, the ledger must represent the corresponding allocated error budget consistently.
Updating the Ledger
Suppose there are \(m\) endpoints and \(K\) planned looks. Initial balances \(A_{i,1}\) are allocated before interim results are observed, with \(\sum_{i=1}^{m}A_{i,1}\leq\alpha\), where \(\alpha\) is the familywise error-rate target. At each look, the debit is determined before the endpoint’s current p-value or test statistic is used to decide rejection. Debits may depend on information from previous looks, so they may be history-dependent.
After the current tests, an endpoint that remains active carries its post-debit balance forward. A rejected endpoint is no longer tested for that hypothesis; its post-debit balance may be transferred to eligible endpoints under the prespecified graph rule. If \(g_{ij,k}\) is the fraction of endpoint \(i\)’s post-debit balance routed to endpoint \(j\) after look \(k\), then the next balance for an active endpoint \(j\) is
where \(\mathcal R_k\) is the set of endpoints rejected at look \(k\), and only eligible active endpoints may receive transfers. The graph rule must specify what happens to any amount not routed to an eligible endpoint. It can be retired or reserved; it cannot be silently counted as available to an active endpoint.
If the design requires conservation of all post-debit balances among active endpoints, the outgoing transfer weights from each rejected endpoint must sum to one over recipients that remain eligible after that look. When some nominal recipients are also rejected, the rule must say whether weights are renormalized over the remaining eligible recipients. Alternatively, amounts directed to ineligible recipients may be explicitly retired. Simply omitting those transfers while claiming full conservation is incorrect.
Why Expected Debits Control Familywise Error
The key justification uses conditional error control and the union bound. Let \(\mathcal F_{k-1}\) denote the information available just before the decisions at look \(k\), including all previous results and the ledger state. Under a true \(H_i\), require that whenever endpoint \(i\) is still eligible for testing,
This condition requires a test calibrated for the information and sequential structure of the design. A p-value that is valid marginally at a single look need not have this conditional property after earlier data have been observed. The debit must also be chosen before the current p-value is observed; otherwise the allowance could be selected in response to the very result it is meant to bound.
Let \(E_{i,k}\) be the event that a true \(H_i\) is falsely rejected at look \(k\). Conditional validity gives \(P(E_{i,k})=E[P(E_{i,k}\mid\mathcal F_{k-1})]\leq E[D_{i,k}]\). Applying the union bound over true hypotheses and looks gives a bound on the probability of at least one false rejection. If the total debit is no more than \(\alpha\) on every possible path, then the expected total debit is also no more than \(\alpha\). Thus:
The second inequality holds because all debits are nonnegative, so including debits for false hypotheses cannot reduce the sum. The final inequality follows from the pathwise budget constraint. This argument does not require the endpoints or their test statistics to be independent, although the tests must meet their stated conditional validity requirements.
Worked Example: One Endpoint With Two Looks
Consider a hypothetical one-sided test for a single endpoint. Its initial alpha balance is \(0.025\). The design assigns a conditional error debit of \(0.006\) at the interim look. Suppose the endpoint is not rejected, and no transfer changes its balance. The post-interim balance is
Thus \(0.019\) remains available for the final look. If the final test is calibrated so that its conditional false-rejection probability is at most \(0.019\) on each relevant interim history, then the total false-rejection probability is bounded by \(0.006+0.019=0.025\). For example, if the final-stage conditional p-value is valid and the prespecified rule rejects at \(p\leq0.019\), that cutoff uses the remaining local alpha.
The conclusion is not that every group sequential design can subtract a nominal interim threshold from \(0.025\) and use the remainder as a final p-value cutoff. The subtraction is justified here because the stated debit is a valid conditional error allowance and the final test is calibrated to that allowance.
Worked Endpoint Ledger With Recycling
Worked Example: Three Endpoints and an Interim Transfer
Consider a hypothetical design with familywise target \(\alpha=0.050\) and three endpoint balances allocated as follows: \(A_{1,1}=0.025\), \(A_{2,1}=0.015\), and \(A_{3,1}=0.010\). At an interim look the prespecified conditional error debits are \(0.008\), \(0.006\), and \(0.004\), respectively. Endpoint 1 is rejected; endpoints 2 and 3 are not. All three debits are within their respective balances.
The post-debit balances are \(R_{1,1}=0.025-0.008=0.017\), \(R_{2,1}=0.015-0.006=0.009\), and \(R_{3,1}=0.010-0.004=0.006\). Suppose the graph routes endpoint 1’s post-debit balance to the still-active endpoints: 75% to endpoint 2 and 25% to endpoint 3. The transfers are
The final-look balances are therefore
Together, the active endpoints have \(0.02175+0.01025=0.03200\) available. This matches the original \(0.050\) less the interim debits \(0.008+0.006+0.004=0.018\). Endpoint 1 is no longer eligible for testing, and its entire post-debit balance was routed to active endpoints. The final local allowances still require suitable endpoint-specific tests; they are not automatically valid merely because the arithmetic balances.
Eligible Recipients and Retired Alpha
Worked Example: Two Rejections and One Eligible Recipient
In a second hypothetical design, endpoints A, B, and C begin with balances \(0.020\), \(0.015\), and \(0.015\), for a total of \(0.050\). At an interim look their conditional error debits are \(0.004\), \(0.003\), and \(0.002\). Both A and B are rejected, while C remains active. Their post-debit balances are \(0.016\), \(0.012\), and \(0.013\).
Suppose the graph’s eligible-recipient rule routes all post-debit alpha from rejected endpoints to active endpoints after the look. Since C is the only eligible recipient, C receives \(0.016+0.012=0.028\). Its final-look balance is
The result agrees with the total budget check: \(0.050-(0.004+0.003+0.002)=0.041\). If, instead, the original graph sent some of A’s or B’s balance to the other endpoint that was also rejected, that portion could not be added to C unless the prespecified rule renormalized the weights over eligible recipients. If it were simply omitted, it would have to be recorded as retired. The active balance would then be smaller, not conserved at \(0.041\).
Interpreting and Checking the Final-Look Alpha
For an endpoint still active at the final analysis, its ledger balance is the remaining budget available to that endpoint. It can be used as a final conditional error allowance, or mapped to a final-stage testing rule using the design’s prespecified calibration. If the final test uses a valid p-value at that stage, a rule of the form \(p_i\leq A_{i,\mathrm{final}}\) may be appropriate. If the final rule is based on a group sequential boundary or a conditional-error function, the boundary must be calibrated to the remaining allowance rather than inferred from the balance alone.
Useful checks include:
- Every debit is nonnegative and no larger than the balance available before the test.
- The debit is specified before the current look’s result is used, even if it depends on earlier history.
- Each transfer is made only after the triggering rejection and only to an eligible endpoint.
- The outgoing weights and any retirement rule account for the entire post-debit balance.
- The ledger total never exceeds the initial familywise budget; any reduction is explained by debits or explicitly retired alpha.
- The endpoint-specific boundaries or p-value rules have been calibrated to their sequential and multiplicity requirements.
A mismatch in the total is a diagnostic, not a harmless formatting issue. A total above the budget can indicate that alpha was transferred without deducting a debit or that an amount was counted twice. A total below the amount expected after spending can indicate omitted transfers, an unintended retirement, or a bookkeeping error. Even a perfectly balanced ledger cannot fix an invalid boundary or an invalid conditional p-value.
How Ledger Accounting Differs From Common Alternatives
| Approach | What it provides | Main limitation for final-look accounting |
|---|---|---|
| Endpoint alpha ledger | Tracks the current balance by endpoint after spending and allowed transfers. | Needs valid sequential calibration and explicit eligibility and retirement rules. |
| Fixed weighted Bonferroni allocation | Assigns fixed endpoint levels whose sum is controlled by the familywise target. | Does not by itself represent interim spending or dynamic transfers. |
| Group sequential spending without endpoint transfers | Allocates error over looks for a specified endpoint test. | Does not by itself determine how one endpoint’s remaining budget changes another endpoint’s final level. |
| Subtracting nominal look thresholds | Produces a quick arithmetic remainder. | Can be invalid when nominal thresholds are not conditional error debits or are not on a common additive budget scale. |
These approaches are not mutually exclusive: a graph can allocate familywise alpha across endpoints while a group sequential method distributes each endpoint’s error allowance across looks. The ledger connects those allocations. It does not replace either multiplicity control or sequential boundary calibration.
Common Mistakes and Limits
- Bounding a fixed probability by a random realized path sum. If debits depend on interim history, their realized total is random. Use conditional validity, take expectations, and bound the expected total by the pathwise budget.
- Calling every debit a nominal p-value threshold. A debit must have the stated error-control meaning. A group sequential boundary may need joint or conditional calibration, not simple subtraction of printed nominal levels.
- Transferring alpha to a rejected endpoint. Transfers are available only to recipients that remain eligible. Amounts not routed to eligible endpoints must be explicitly retired or reserved.
- Assuming graph weights guarantee conservation automatically. Weights must sum to one over eligible recipients if conservation among active endpoints is intended; weights to ineligible endpoints cannot be silently ignored.
- Reallocating based on the current test result without a prespecified rule. The debit is chosen before the current p-value is observed. Outcome-dependent transfers must follow a valid, prespecified procedure.
Ledger calculations are a transparent way to report what remains for final testing, but they do not settle every design question. Their validity depends on the correctness of the conditional testing rules, the graph’s update logic, and the handling of endpoints whose data are unavailable or whose hypotheses are no longer eligible. The final local alpha is therefore a design quantity interpreted together with the rule that turns it into a final test.
Check Your Understanding
Work through each question, then compare your reasoning with the solutions below.
- An endpoint has balance \(0.030\) before an interim look and receives a valid debit of \(0.007\). No transfer occurs. What balance remains for the next look?
- Why is a path-dependent debit total not, by itself, a fixed upper bound for the familywise error probability?
- Two endpoints are rejected at the same look, and a third remains active. What must a transfer rule specify if all post-debit balances are to be conserved among active endpoints?
- A final endpoint balance is \(0.018\). Does this fact alone prove that a final p-value cutoff of \(0.018\) is valid? Explain.
Solutions. (1) The remaining balance is \(0.030-0.007=0.023\). (2) The debit total is random when it depends on the observed history, whereas the familywise error probability is fixed. Conditional error bounds are averaged to obtain an expectation bound, and a pathwise budget of at most \(\alpha\) then implies that the expected total debit is at most \(\alpha\). (3) The rule must route the balances only to endpoints eligible after the look, renormalizing over eligible recipients if that is the intended policy, or explicitly retire any unrouted amount. (4) No. The final test must be calibrated to the remaining allowance and satisfy the required sequential and multiplicity validity conditions; the balance alone is bookkeeping, not proof of a valid p-value rule.