Why Solver Stability Matters
A calibrated efficacy boundary is defined by a probability equation, but that equation usually has no closed-form solution once earlier looks and their boundaries are included. The numerical solver is therefore part of the design calculation: a converged-looking answer is not automatically the intended boundary. Poor starting values, very small derivatives, and numerical integration error can all produce unstable results.
The earlier material on recursive calibration of efficacy boundaries sets a look-specific target after earlier boundaries have been fixed. Here the concern is how reliably to solve that target equation. We consider an upper-sided group sequential test under the null, with fixed information times and the canonical joint normal model. At look \(i\), the standardized statistic is \(Z_i\), the upper efficacy boundary is \(b_i\), and a crossing occurs when \(Z_i\ge b_i\). Earlier boundaries \(b_1,\ldots,b_{i-1}\) are treated as fixed.
Under the canonical joint normal model, \(Z_j\) has a standard normal marginal distribution under \(H_0\), and the correlation between looks \(j<k\) is \(\sqrt{I_j/I_k}\), where \(I_j\) and \(I_k\) are their information levels. This model, the specified information times, the fixed earlier boundaries, and the numerical evaluation of the joint crossing probability must be adequate for the design. The root-finding discussion does not repair misspecified information times or an inaccurate probability calculation.
Existence, Shape, and the Derivative
For fixed earlier boundaries, \(q_i(b)\) is nonincreasing in \(b\): raising the current boundary can only remove paths that would otherwise cross at look \(i\). Under a nonsingular joint normal model with positive probability of reaching look \(i\) without an earlier crossing, the function is continuous and strictly decreasing at finite \(b\). Thus there is at most one finite boundary for a specified target. The target must also be attainable. At \(b\to+\infty\), \(q_i(b)\to0\); at \(b\to-\infty\), it approaches the probability of no earlier crossing. A target strictly between these limiting values has a finite root.
A useful derivative follows by conditioning on the current statistic. Write \(S_{i-1}\) for the event that no earlier boundary was crossed. Then \(q_i(b)=\int_b^\infty \phi(z)P(S_{i-1}\mid Z_i=z,H_0)\,dz\), where \(\phi\) is the standard normal density. Differentiating the integral's lower limit gives \(q_i'(b)=-\phi(b)P(S_{i-1}\mid Z_i=b,H_0)\).
This derivative explains two important stability problems. In the far tails, \(\phi(b)\) can be extremely small. Also, conditional on a value at the current look, the probability of having survived earlier boundaries can be small. Either factor can make the curve nearly flat, so a small probability error corresponds to a large boundary error. The derivative may be approximated numerically, but a noisy derivative can be less reliable than the probability itself.
Worked Example: Bracketing a One-Look Boundary
Suppose a hypothetical one-look upper-sided test needs a null crossing probability of \(0.025\). With one look, \(q(b)=P(Z\ge b\mid H_0)=1-\Phi(b)\), where \(Z\sim N(0,1)\). Define \(f(b)=q(b)-0.025\); the desired boundary solves \(f(b)=0\).
At \(b=1\), \(q(1)\approx0.1587\), so \(f(1)\approx0.1337>0\). At \(b=3\), \(q(3)\approx0.001350\), so \(f(3)\approx-0.02365<0\). Continuity therefore places a root in \([1,3]\). Since \(f\) is decreasing, this bracket contains the unique root.
Bisection takes the midpoint \(2\). Since \(q(2)\approx0.02275<0.025\), the root lies in \([1,2]\). The next midpoint is \(1.5\), where \(q(1.5)\approx0.06681>0.025\), leaving \([1.5,2]\). Continuing in this way narrows the bracket without requiring a derivative. The solution is \(b\approx1.960\), because \(1-\Phi(1.960)\approx0.0250\).
This simple case checks the sign convention and illustrates the basic safeguard: preserve endpoints with opposite residual signs. In a multi-look calibration, \(q_i\) is obtained from the joint null distribution rather than a single normal tail, but the bracketing logic is the same.
Bracketing and Newton-Type Methods
A bracketing method starts with \(L\) and \(U\) such that \(f(L)\) and \(f(U)\) have opposite signs. For a continuous function, a root lies between them. Bisection repeatedly evaluates the midpoint and keeps the half-interval with opposite endpoint signs. It converges even when the curve is not well approximated by a straight line, provided the function evaluation is sufficiently accurate and the bracket is genuine. Its main cost is that it can require many evaluations for high boundary precision.
Newton's method uses a local linear approximation. From a current value \(b_m\), it proposes \(b_{m+1}=b_m-f(b_m)/f'(b_m)\). When the starting point is close to a simple root and the derivative is accurate and not near zero, convergence can be very fast. But Newton's update is not constrained to remain near the root. A poor start or a small derivative can produce a huge step, move to a region with a different scale, or make subsequent probability calculations unreliable. A solver reporting a small step is not enough: the residual \(f(b)\) must also be acceptably small.
A practical compromise is safeguarded Newton: maintain a valid bracket, try a Newton update, and accept it only if it is finite and stays safely inside the bracket. Otherwise take a bisection step. Brent-type methods similarly combine interpolation with bracketing safeguards. These methods usually retain the robustness of a bracket while reducing the number of evaluations when the curve is well behaved.
Worked Example: A Poor Newton Starting Value
Consider the same one-look normal-tail equation, now with target \(0.001\): \(f(b)=1-\Phi(b)-0.001\). Its derivative is \(f'(b)=-\phi(b)\), and the root is near \(3.090\). Suppose, purely to illustrate solver behavior, that Newton's method is started at \(b_0=-3.234\), far from that root.
At this starting value, \(1-\Phi(-3.234)=\Phi(3.234)\approx0.999389\), so \(f(b_0)\approx0.998389\). The density is \(\phi(-3.234)\approx0.00213708\), and hence \(f'(b_0)\approx-0.00213708\). The Newton update is
The enormous proposed value is not a useful boundary; it is a warning that the local tangent at the starting point is a poor representation of the function near its root. Rounding intermediate tail probabilities or densities can change the last digits of this extreme update, so the displayed values are approximate. A bracket avoids this failure: for example, \(f(3)>0\) because the tail at 3 is about \(0.001350\), whereas \(f(4)<0\) because the tail at 4 is about \(0.00003167\). The root is safely contained in \([3,4]\).
Worked Example: A Nearly Flat Crossing-Probability Curve
To isolate the flat-curve problem, take a hypothetical one-look normal-tail target of \(10^{-8}\). The equation is \(f(b)=1-\Phi(b)-10^{-8}=0\). Its solution is approximately \(b=5.612\). Starting instead at \(b_0=8\), the tail is about \(6.221\times10^{-16}\), and the density is \(\phi(8)\approx5.052\times10^{-15}\).
Thus \(f(8)\approx-9.9999994\times10^{-9}\) and \(f'(8)\approx-5.052\times10^{-15}\). The Newton proposal is approximately \(8-(-9.9999994\times10^{-9})/(-5.052\times10^{-15})\), or about \(-2.0\times10^6\). The tiny slope makes a probability residual of only about \(10^{-8}\) generate a fantastically large step. This is numerical instability, not evidence that the calibrated boundary is extreme in that direction.
A bracket such as \([5,6]\) is informative: the tail at 5 is about \(2.867\times10^{-7}\), above the target, and the tail at 6 is about \(9.866\times10^{-10}\), below it. Bisection or a safeguarded method stays within this interval. In a multi-look design, the conditional-survival factor in \(q_i'(b)\) can flatten the curve further, so derivative checks are important even if the marginal normal density is not exceptionally small.
Practical Diagnostics and Failure Modes
Before trusting a computed boundary, assess the equation and its numerical evaluation, not merely the solver's status message. The following checks are useful during calibration:
- Check attainability. Compare the target \(\Delta_i\) with the limiting probabilities of \(q_i\). A target outside the attainable range has no finite root; changing solvers will not fix that design or setup problem.
- Check monotonicity and signs. Evaluate the crossing probability at candidate lower and upper boundaries. It should decrease as the boundary rises. A sign change in \(f(b)=q_i(b)-\Delta_i\) gives a bracket; an apparent increase may indicate a coding, integration, or precision problem.
- Check the residual. Report or inspect \(|q_i(b_i)-\Delta_i|\), not only the change in \(b\) between iterations. A small change can occur because of a stalled iteration, while the target error remains material.
- Check derivative scale. A very small \(|q_i'(b)|\) means boundary precision is sensitive to probability error. If the probability has absolute error \(\varepsilon\), a local approximation gives boundary error about \(\varepsilon/|q_i'(b)|\).
- Check integration stability. If crossing probabilities are computed by numerical integration, tighten the integration tolerance and compare results. Newton updates based on a derivative or residual dominated by integration noise can jump or falsely appear converged.
- Check the final result independently. Re-evaluate the crossing probability at the proposed boundary with tighter numerical settings, and verify that it is close to the target. Do not treat a boundary as validated merely because the iterative routine terminated.
Probability and boundary tolerances should be chosen together. For example, when a local derivative is about \(10^{-4}\), a probability error of \(10^{-7}\) corresponds approximately to a boundary error of \(10^{-3}\). If that boundary precision is unacceptable, the integration and root tolerances must be tightened. Conversely, demanding an extremely small probability residual when the integration routine is less accurate than that residual creates false precision.
Choosing a Solver and Knowing Its Limits
| Approach | Strength | Main risk or cost |
|---|---|---|
| Bisection with a valid bracket | Reliable progress for a continuous function; no derivative required | May need many probability evaluations |
| Newton's method | Very fast near a simple root with an accurate, nonzero derivative | Can take huge or invalid steps from poor starts or flat regions |
| Safeguarded Newton or Brent-type method | Combines local speed with bracket-based protection | Still depends on accurate function evaluations and a valid initial bracket |
A bracket is not infallible if the probability calculation is noisy enough to give incorrect endpoint signs, and a root solver cannot correct a wrong correlation model, incorrect information fractions, or an incorrectly specified spending target. Multiple roots are not expected for this one-sided crossing equation under the stated conditions because it is decreasing; observed nonmonotonicity is a diagnostic signal rather than a reason to select whichever root looks plausible. Numerical calibration also gives no guarantee by itself that the complete design has the intended cumulative error rate; that is a separate verification task.
The central practical rule is to use a bracket whenever feasible, then allow Newton-type acceleration only when its proposed step is sensible and its derivative is trustworthy. In a flat region, prefer bracket progress and stronger probability evaluation over increasingly aggressive Newton updates. State the numerical residual and the boundary precision, so the computed result can be judged in terms of the design target rather than the solver's internal stopping rule.
Check Your Understanding
Answer each question, then compare with the brief solutions below.
- For \(f(b)=q_i(b)-\Delta_i\), what does an endpoint sign change on \([L,U]\) establish, and what additional property gives uniqueness?
- Why can Newton's method produce a huge step when the crossing-probability curve is nearly flat?
- A solver reports a very small change in successive boundaries, but the probability residual is still larger than the design tolerance. Is calibration complete?
- For a one-look normal-tail target of \(0.025\), a proposed bracket is \([1,3]\). Do the endpoint values bracket a root?
- Give two checks that can reveal whether a computed multi-look boundary is unreliable.
Solutions. (1) Continuity and opposite endpoint signs establish at least one root in the interval; strict decrease establishes that there is at most one. (2) Newton divides the residual by the derivative, so a small derivative magnifies the update. (3) No. A small step can indicate stalling; the residual must meet the probability tolerance. (4) Yes: \(f(1)\approx0.1337>0\) and \(f(3)\approx-0.02365<0\). (5) Examples include checking that \(q_i(b)\) decreases with \(b\), re-evaluating the residual with tighter integration settings, confirming a valid endpoint sign change, and checking derivative size.