Recalibrating Stage-Two Critical Values

Why Recalibrate After Re-Estimation?

A sample size re-estimation can change how much information the final statistic receives from the interim data and how much comes from the second stage. If the original terminal critical value is simply retained, the probability of rejection conditional on the observed interim result can change. Recalibration instead asks a precise question: what stage-two threshold makes the revised design’s conditional rejection probability equal the original design’s conditional error?

The distinction between a stage-two cutoff and a terminal cutoff matters. The stage-two cutoff applies to the standardized, independent second-stage statistic. The terminal cutoff applies to the revised weighted combination of the interim and stage-two statistics. When the information weights change, that terminal cutoff generally changes even when the required standardized stage-two cutoff does not.

Definition: In a one-sided canonical design, the conditional error at an observed interim statistic $Z_1=x$ is the probability, under $H_0$ and conditional on $Z_1=x$, that the original design will reject $H_0$. Recalibration chooses a rejection threshold for the revised design so that its conditional rejection probability at the same interim result equals this conditional error.

Canonical Model and Recalibration Formula

Assume a one-sided upper-tail test and a canonical normal model under $H_0$. The original design has first-look information fraction $t$, where $0<t<1$, interim statistic $Z_1$, and terminal statistic

Z_{\mathrm{old}}=\sqrt{t}\,Z_1+\sqrt{1-t}\,Z_2,

where $Z_2\sim N(0,1)$ is independent of $Z_1$ under $H_0$. The original design rejects when $Z_{\mathrm{old}}>c$, with fixed upper-tail critical value $c$. These assumptions express the independent-increments canonical model: after conditioning on the interim data, the standardized second-stage increment remains standard normal and independent.

Given $Z_1=x$, the original design rejects exactly when $Z_2$ exceeds a particular threshold. Its conditional error is therefore

\gamma(x) =P_0(Z_{\mathrm{old}}>c\mid Z_1=x) =1-\Phi\!\left(\frac{c-\sqrt{t}\,x}{\sqrt{1-t}}\right), \qquad q_{\mathrm{old}}(x)=\frac{c-\sqrt{t}\,x}{\sqrt{1-t}}.

Here $\Phi$ is the standard normal distribution function, and $q_{\mathrm{old}}(x)$ is the original standardized stage-two cutoff. Now suppose the revised design uses information fraction $r$, with $0<r<1$, and the same canonical form:

Z_{\mathrm{new}}=\sqrt{r}\,Z_1+\sqrt{1-r}\,Z_2.

The value $r$ may be chosen using the interim data, provided it is determined by the time of re-estimation and, conditional on those data, the stated canonical model remains valid. Write $C_{\mathrm{new}}(x)$ for the revised terminal critical value. To match the original conditional error, require

P_0(Z_{\mathrm{new}}>C_{\mathrm{new}}(x)\mid Z_1=x)=\gamma(x).

Since the revised statistic is conditionally normal with mean $\sqrt{r}\,x$ and variance $1-r$, the equation can be solved directly.

Result: The unique revised standardized stage-two cutoff is $q_{\mathrm{new}}(x)=q_{\mathrm{old}}(x)$. The corresponding revised terminal critical value is $$ C_{\mathrm{new}}(x)=\sqrt{r}\,x+\sqrt{1-r}\,q_{\mathrm{old}}(x) =\sqrt{r}\,x+\sqrt{1-r}\,\frac{c-\sqrt{t}\,x}{\sqrt{1-t}}. $$

This result does not say that the revised terminal boundary stays equal to $c$. It says that the revised design must allocate the same conditional error to its independent stage-two evidence. The terminal boundary is then the value that implements that stage-two threshold under the revised weights.

Derivation and Uniqueness

For a proposed revised boundary $C$, the conditional rejection probability is

P_0(Z_{\mathrm{new}}>C\mid Z_1=x) =1-\Phi\!\left(\frac{C-\sqrt{r}\,x}{\sqrt{1-r}}\right).

Set this equal to the original conditional error $1-\Phi(q_{\mathrm{old}}(x))$. The function $\Phi$ is strictly increasing, so equality of the upper-tail probabilities implies equality of their standardized thresholds:

\frac{C-\sqrt{r}\,x}{\sqrt{1-r}}=q_{\mathrm{old}}(x).

Solving for $C$ gives $C=\sqrt{r}\,x+\sqrt{1-r}\,q_{\mathrm{old}}(x)$. The same argument proves uniqueness: with $0<r<1$, increasing $C$ strictly decreases the conditional rejection probability, so exactly one finite boundary matches any conditional error strictly between zero and one. Equivalently, the revised test rejects when $Z_2>q_{\mathrm{old}}(x)$, making the equality of conditional errors explicit.

Worked Example: Lower Revised Information Fraction

Suppose the original conditional calculation at an observed interim value $x=-0.50$ gives the standardized stage-two cutoff $q_{\mathrm{old}}(x)=3.271859$. Its conditional error is $1-\Phi(3.271859)$, approximately $0.000535$. A hypothetical re-estimation selects revised information fraction $r=0.20$. Find the revised terminal critical value.

Use the recalibration formula, retaining the original conditional stage-two cutoff:

\begin{aligned} C_{\mathrm{new}}(-0.50) &=\sqrt{0.20}(-0.50)+\sqrt{0.80}(3.271859)\\ &\approx -0.223607+2.926440\\ &\approx 2.702833. \end{aligned}

Thus a terminal statistic $Z_{\mathrm{new}}=\sqrt{0.20}Z_1+\sqrt{0.80}Z_2$ is compared with a boundary of about $2.70283$ at this interim result. To check the calculation, convert that boundary back to a stage-two threshold:

\frac{2.70283-\sqrt{0.20}(-0.50)}{\sqrt{0.80}} \approx \frac{2.92644}{0.894427} \approx 3.27186.

The back-converted cutoff agrees, to rounding, with $3.271859$. Therefore the revised conditional rejection probability remains approximately $0.000535$, the original conditional error at this interim result.

Worked Example: Recalibrating at a Positive Interim Result

At a different hypothetical interim result, suppose $x=0.80$ and the original conditional calculation gives $q_{\mathrm{old}}(x)=1.9288$. The revised information fraction is $r=0.65$. Compute the revised terminal boundary and check its implied second-stage threshold.

Substitute the interim value, revised fraction, and original conditional cutoff:

\begin{aligned} C_{\mathrm{new}}(0.80) &=\sqrt{0.65}(0.80)+\sqrt{0.35}(1.9288)\\ &\approx 0.644981+1.14109\\ &\approx 1.78607. \end{aligned}

The revised terminal critical value is approximately $1.7861$. Back-conversion gives

\frac{1.7861-\sqrt{0.65}(0.80)}{\sqrt{0.35}} \approx \frac{1.14112}{0.591608} \approx 1.9288.

Consequently, the conditional rejection probability remains $1-\Phi(1.9288)$, approximately $0.0269$. The positive interim value contributes $0.644981$ to the revised terminal statistic’s conditional mean; it does not change the required standardized second-stage cutoff.

Worked Example: Starting From the Original Design Boundary

Consider a hypothetical original design with $t=0.50$, terminal boundary $c=2.00$, and observed interim statistic $x=0$. Suppose re-estimation changes the information fraction to $r=0.80$. Find the original conditional error and the revised boundary.

First calculate the original standardized stage-two cutoff:

q_{\mathrm{old}}(0) =\frac{2.00-\sqrt{0.50}(0)}{\sqrt{0.50}} =\frac{2.00}{0.707107} \approx 2.828427.

Thus the original conditional error is $1-\Phi(2.828427)$, approximately $0.002339$. The revised terminal boundary is

\begin{aligned} C_{\mathrm{new}}(0) &=\sqrt{0.80}(0)+\sqrt{0.20}(2.828427)\\ &=0+0.447214(2.828427)\\ &\approx 1.264911. \end{aligned}

Check by standardizing the revised threshold:

\frac{1.264911-\sqrt{0.80}(0)}{\sqrt{0.20}} \approx \frac{1.264911}{0.447214} \approx 2.828427.

The revised and original conditional rejection probabilities are therefore equal, up to rounding. This example also shows why the revised terminal boundary need not equal the original boundary $2.00$.

Implementation Checks and Failure Modes

In an actual re-estimation rule, the new information fraction can depend on $x$, so the terminal boundary is also a function of $x$. At each possible interim result where the rule changes the second-stage information, the same calculation applies using the selected $r(x)$. A practical check is to calculate the implied stage-two cutoff, $(C_{\mathrm{new}}(x)-\sqrt{r(x)}x)/\sqrt{1-r(x)}$, and confirm that it equals $q_{\mathrm{old}}(x)$ within numerical tolerance. One can also compare the two conditional upper-tail probabilities directly.

The calculation relies on the model and test staying aligned. In particular, under $H_0$, the revised standardized second-stage statistic must have the stated standard normal distribution conditional on the interim data, and it must be independent of the interim statistic. The information weights must correctly describe the revised canonical statistic. If re-estimation changes the test statistic or its null distribution, or if the second-stage increment is not conditionally independent, this formula may not calibrate the actual conditional rejection probability. In those cases, the conditional distribution must be derived for the revised procedure rather than assumed to be canonical.

Conditional-error recalibration is not the only way to retain error control. A combination test specifies a rule for combining stagewise evidence and calibrates its rejection threshold accordingly; it can be useful when the design is framed in terms of stagewise statistics or p-values. By contrast, the formula here directly matches the original design’s conditional rejection probability in the canonical normal model. Simply retaining the original fixed terminal boundary is a different approach: after the weights change, it will generally produce a different conditional error.

Common Mistakes and Limitations

Reusing the old terminal boundary automatically. The same numerical boundary under different information weights generally gives a different conditional rejection probability. Recalculate the terminal boundary from the conditional stage-two cutoff.
Changing the standardized stage-two cutoff without changing the target error. Under the stated canonical assumptions, matching the original conditional error means retaining $q_{\mathrm{old}}(x)$. The revised terminal boundary changes to reflect $r$.
Mixing up the weights. The interim contribution uses $\sqrt{r}\,x$, and the stage-two contribution uses $\sqrt{1-r}\,Z_2$. Reversing them changes the conditional distribution and gives the wrong boundary.
Ignoring the conditioning value. The original threshold $q_{\mathrm{old}}(x)$ depends on the observed interim statistic. A single unconditional critical value cannot generally represent all conditional-error targets.
Using the formula outside its assumptions. A dependent stage-two statistic or a changed null reference distribution requires a new conditional-probability calculation. The normal formula is not a universal correction for every adaptation.

For $0<r<1$ and a finite original cutoff, the conditional error lies strictly between zero and one, so the matching cutoff is finite and unique. At $r=1$, there is no random stage-two increment; a deterministic terminal decision cannot generally reproduce an arbitrary conditional error strictly between zero and one. Extreme interim values can also yield very small or very large conditional errors, which may make numerical tail calculations sensitive to rounding.

Check Your Understanding

Use the one-sided canonical model and show the key calculation for each answer.

An original design has $t=0.40$, $c=2.10$, and observed $x=0.50$. Find the original standardized stage-two cutoff.
At an interim result, the original stage-two cutoff is $1.50$. If $x=0.40$ and the revised fraction is $r=0.60$, find the revised terminal boundary.
Why does preserving the original terminal boundary generally fail to preserve conditional error after the information fraction changes?
Name two assumptions needed for the canonical recalibration formula to apply.

Solutions. (1) $q_{\mathrm{old}}=(2.10-\sqrt{0.40}(0.50))/\sqrt{0.60}=(2.10-0.316228)/0.774597\approx 2.303$. (2) $C_{\mathrm{new}}=\sqrt{0.60}(0.40)+\sqrt{0.40}(1.50)\approx 0.309839+0.948683=1.25852$. Its implied stage-two cutoff is $(1.25852-0.309839)/0.632456\approx1.50$. (3) The conditional mean and variance of the weighted terminal statistic depend on the information fraction; using an unchanged boundary therefore usually changes its standardized conditional threshold and rejection probability. (4) Under $H_0$, the second-stage standardized statistic must be standard normal conditional on the interim data and independent of the interim statistic, with weights that correctly represent the revised canonical statistic.