Why Track Probability Mass?
A recursive group sequential calculation carries forward the probability of paths that have not yet stopped. At each look, some of that probability crosses a stopping boundary and the remainder continues. The recursion may produce plausible-looking boundary probabilities and power estimates even if a small amount of probability disappears or is counted twice at an intermediate step. A stagewise mass balance makes that failure visible: the probability entering a look must equal the probability stopping there plus the probability surviving it.
This is an accounting check, not a new method for choosing boundaries. It applies whether the recursion is evaluated by quadrature, a grid, or another numerical method. It is especially useful when the overall result seems plausible but small implementation errors may accumulate over several looks.
What the Recursion Is Conserving
For a standard canonical group sequential statistic under the null, the look statistics have standard normal marginal distributions and satisfy \(\operatorname{Corr}(Z_i,Z_j)=\sqrt{t_i/t_j}\) for \(i<j\), where \(t_j\) is the information fraction at look \(j\). Equivalently, the transition from look \(j-1\) to look \(j\) is normal with conditional mean \(\sqrt{t_{j-1}/t_j}\,x\) and conditional variance \(1-t_{j-1}/t_j\), given \(Z_{j-1}=x\). This is the model underlying the recursion below; the transition distribution and the specified stopping regions must match the analysis being implemented.
Write \(f_{j-1}(x)\) for the survivor density at look \(j-1\). It is a sub-probability density: its integral is \(S_{j-1}\), not necessarily 1, because paths that stopped earlier have already been removed. If \(K_j(y\mid x)\) is the conditional transition density, the density propagated to the next look before applying that look’s stopping rule is $$ g_j(y)=\int f_{j-1}(x)K_j(y\mid x)\,dx. $$ For lower and upper boundaries \(a_j\) and \(b_j\), with continuation region \(C_j=(a_j,b_j)\), the stage probabilities are $$ q_j^-=\int_{y\le a_j}g_j(y)\,dy,\qquad q_j^+=\int_{y\ge b_j}g_j(y)\,dy,\qquad S_j=\int_{a_j<y<b_j}g_j(y)\,dy. $$ The regions must cover the whole possible range and must not overlap in a way that assigns positive probability twice. With continuous distributions, assigning a boundary point to one adjacent region rather than the other does not change the exact probability. On a discrete grid, however, a boundary node may carry positive quadrature weight, so the implementation must assign it consistently.
Why the Conservation Identity Holds
The identity follows directly from partitioning the transition integral. The lower crossing region, continuation region, and upper crossing region are disjoint and exhaustive, apart from boundary points whose probability is zero under a continuous density. Therefore, for every incoming value \(x\), integrating \(K_j(y\mid x)\) over these three regions gives 1. Multiplying by the incoming survivor density and integrating over \(x\) gives the total incoming mass.
The last equality uses the fact that each conditional transition density integrates to 1. This justification also clarifies the assumptions behind the check: the recursion must propagate the full incoming survivor density, use a properly normalized transition distribution, and partition the outcome space into the intended stop and continuation regions. If the design includes futility stopping or another non-rejection stop, include its probability as a separate stopping term; do not quietly classify it as efficacy rejection or continuation.
Worked Example: One-Look Mass Balance
Consider a hypothetical one-look, two-sided calculation under \(H_0\), with \(Z_1\sim N(0,1)\) and stopping boundaries \(-1.96\) and \(1.96\). The lower and upper crossing probabilities are
The continuation probability is
The total is \(0.0249979+0.0249979+0.9500042=1.0000000\), up to rounding. The two crossing probabilities sum to approximately \(0.0499958\). This check verifies that the two tails and the central continuation interval cover the standard normal distribution without a gap or overlap.
Reading the Stagewise Ledger
At every look, record the incoming mass, each stopping probability, the outgoing survivor mass, and the residual. Also check the cumulative ledger: the probability of having stopped by look \(j\), plus \(S_j\), should equal 1. In a design with separate stopping categories, sum all categories that remove a path from the recursion.
| Pattern in the ledger | Possible explanation | First check |
|---|---|---|
| Positive residual: incoming mass exceeds reported stops plus survivors | Mass was omitted from an integration range or dropped at a boundary | Compare the integration limits with the full continuation and stopping regions |
| Negative residual: reported stops plus survivors exceed incoming mass | Some mass may have been counted twice | Check for overlapping regions or a boundary node included in two sums |
| Small residuals of mixed sign | Ordinary quadrature, interpolation, or rounding error may be present | Compare residual size with refinement and the numerical error budget |
| Residual grows across looks | A local error may be propagating through later survivor densities | Find the earliest look with a material residual |
A residual is a diagnostic, not a correction factor. Renormalizing a survivor density to force its integral to match a target can conceal an omitted interval or duplicated boundary contribution. First locate the discrepancy, then correct the numerical calculation and recompute all downstream quantities.
Worked Example: Locate an Overcount in a Two-Look Ledger
Suppose a hypothetical two-look calculation reports, at look 1, \(q_1^-=0.020\), \(q_1^+=0.030\), and \(S_1=0.950\). The first ledger balances: \(0.020+0.030+0.950=1.000\). At look 2, the reported values are \(q_2^-=0.012\), \(q_2^+=0.018\), and \(S_2=0.925\).
The incoming mass to look 2 is \(S_1=0.950\), so the residual is
The reported terms exceed the incoming mass by 0.005, indicating an overcount rather than leakage. The cumulative ledger also exposes it: the stopping probability through look 2 is \(0.020+0.030+0.012+0.018=0.080\), and the reported final survivor mass is 0.925, giving \(0.080+0.925=1.005\), not 1.
The negative residual does not by itself identify the faulty line of code. A sensible audit checks whether the upper and lower crossing regions overlap, whether a boundary grid point contributes to both a stop and the survivor calculation, and whether a weighted grid value was added twice. If the stopping probabilities are independently verified, the survivor update is the natural next place to inspect.
Tracing Discrepancies to Integration Limits
Integration limits can create leakage even when the transition density itself is correct. The integration range must include the whole region assigned to each outcome. In particular, a finite grid used to approximate an unbounded normal tail must account for the probability beyond its endpoints; simply discarding values outside the grid removes mass.
Worked Example: Continuation Limits Too Narrow
Return to the hypothetical one-look \(N(0,1)\) calculation with stopping boundaries \(-1.96\) and \(1.96\). The true continuation region is \((-1.96,1.96)\), but suppose a numerical routine integrates the survivor only over \((-1.90,1.90)\) while still treating values beyond \(\pm1.96\) as crossings. The omitted continuation slices have total probability
The routine has lost about 0.0074373 of probability: those values lie inside the true continuation region but are absent from both its reported survivor mass and its crossing probabilities. The stagewise residual will be positive by approximately that amount, subject to numerical integration error. The correction is to integrate the survivor over the specified continuation limits, not to increase a tail probability or renormalize the result.
A different failure occurs if a code uses an integration interval that stops at a finite endpoint but treats all values outside it as if they had crossed a boundary, even when the stopping boundary lies farther out. That can move probability into the wrong outcome category. Thus, checking only that the total is near 1 is not enough: the category-specific integration limits must match the design.
Practical Audit and Alternatives
When a residual is larger than expected, proceed from the earliest problematic look. Verify the incoming survivor mass; check that the transition density is normalized; inspect the lower-stop, continuation, and upper-stop limits; and confirm that grid weights are included once and only once. Then refine the grid or integration tolerance. A correctly specified calculation should show the residual decreasing or stabilizing at a size consistent with the numerical error budget as accuracy is increased.
For each look, tabulate incoming mass, every stopping category, outgoing survivor mass, and the residual.
Later discrepancies may simply inherit an error in an earlier survivor density.
Check for gaps, overlaps, truncated tails, and inconsistent handling of boundary nodes.
Refine numerical settings and, where useful, compare with simulation or a direct multivariate normal calculation.
Simulation and direct multivariate normal probabilities are useful independent comparisons, but they answer a different diagnostic question. Simulation has sampling variability and may not reveal a small deterministic integration error without many replicates. A direct probability calculation can provide a benchmark, but it does not replace a stagewise ledger for locating where a recursive implementation went wrong. Conversely, conservation alone does not prove the probabilities are correct: an error can transfer mass between categories while leaving the total unchanged.
The check also has limits. A small residual can coexist with incorrect boundaries, an incorrect transition model, or a coding error that preserves total mass. Discretization can produce residuals even when the formulas are right, and finite-precision arithmetic can affect very small probabilities. Interpret the residual alongside grid-refinement results, integration diagnostics, and the numerical error budget, rather than demanding exact zero from a numerical calculation.
Common Mistakes
- Normalizing away the evidence. Rescaling the survivor density to make its integral match the expected mass may hide a missing tail or a duplicated contribution. Diagnose the cause before any rescaling.
- Checking only the final total. Errors at separate looks can offset one another. Inspect each stage and find the first material residual.
- Calling every discrepancy “lost mass.” A negative residual means the reported outcomes exceed incoming mass, which is an overcount. A positive residual indicates mass is missing from the reported partition.
- Assuming conservation validates the model. A perfectly balanced ledger can still use the wrong transition distribution or stopping boundaries.
Check Your Understanding
Answer each question using the stagewise ledger and the stated hypothetical values.
- At a look, incoming mass is 0.72, lower-stop mass is 0.08, upper-stop mass is 0.11, and survivor mass is 0.52. Calculate the residual and identify the discrepancy direction.
- A program assigns a grid node exactly at the upper boundary to both the upper-stop sum and the survivor sum. What sign of residual would this tend to produce, and why?
- Why can a small overall residual fail to detect a wrong stagewise calculation?
- In the standard normal example with boundaries \(\pm1.96\), why does integrating the survivor only over \((-1.90,1.90)\) fail even if both crossing tails are calculated correctly?
Solutions. (1) \(r=0.72-(0.08+0.11+0.52)=0.01\), so 0.01 of incoming mass is unaccounted for: the residual is positive. (2) It tends to produce a negative residual because the boundary node’s positive grid weight is counted twice, making reported outcomes exceed incoming mass. (3) Errors at different looks or in different outcome categories can offset, and a calculation can conserve total mass while assigning it to the wrong categories. Check each stage and category, not only the grand total. (4) The omitted slices from 1.90 to 1.96 on both sides are inside the continuation region, not in either crossing tail. Their total probability is approximately 0.0074373, so the survivor is understated and the stagewise ledger loses that mass.