Intervals for Seamless Phase 2/3 Estimates

Why the Phase-Transition Rule Matters

In a seamless phase 2/3 trial, phase 2 participants may contribute to the final treatment-effect estimate, but phase 3 begins only if an interim rule is met. The final estimate therefore combines data collected under a design whose later stage depended on earlier results. A conventional interval centered on the combined estimate can ignore that dependence: among trials that pass an efficacy threshold, early results tend to be unusually favorable. The resulting interval need not have its advertised coverage among trials that continue.

This tutorial treats a single treatment-versus-control contrast and a prespecified rule to continue when the phase 2 statistic exceeds a threshold. The target is one common treatment effect across both stages. The interval will have a stated conditional guarantee: its coverage is assessed among repetitions that satisfy the transition rule. That is distinct from coverage averaged over both continuation and non-continuation outcomes.

Definition: A seamless phase 2/3 estimate combines stage 1 (phase 2) and stage 2 (phase 3) data for the same treatment-effect estimand, with stage 2 observed only if a prespecified phase-transition rule is met. A conditional confidence interval is calibrated using the sampling distribution given that the rule was met. Its coverage statement is conditional on that event, not automatically a statement about all randomized trials.

Model and Conditional Interval

Suppose the stage-specific estimates are independent and normally distributed, with a common treatment effect \(\theta\) and known information \(I_1\) and \(I_2\). Information is the inverse of the estimate's variance. Write the stagewise standardized statistics as \(Z_j=\sqrt{I_j}\hat{\theta}_j\), for \(j=1,2\). Under the model, \(Z_j\sim N(\theta\sqrt{I_j},1)\). Thus the combined estimate and its unconditional standard error, before accounting for the transition, are

\hat{\theta}=\frac{I_1\hat{\theta}_1+I_2\hat{\theta}_2}{I_1+I_2} =\frac{\sqrt{I_1}Z_1+\sqrt{I_2}Z_2}{I_1+I_2}, \qquad SE(\hat{\theta})=\frac{1}{\sqrt{I_1+I_2}}.

Assume stage 2 is independent of stage 1, the two stages measure the same endpoint in the same target population, the treatment effect is common across stages, and \(I_1,I_2\) are known and fixed. Let continuation occur when \(A=\{Z_1>c\}\), for a prespecified threshold \(c\). The conditional distribution needed for inference is

F_\theta(t\mid A) =P_\theta(\hat{\theta}\le t\mid Z_1>c) =\frac{P_\theta\!\left(\sqrt{I_1}Z_1+\sqrt{I_2}Z_2\le (I_1+I_2)t,\ Z_1>c\right)} {1-\Phi(c-\theta\sqrt{I_1})}.

Here \(\Phi\) is the standard normal cumulative distribution function. Its denominator is the probability of passing the transition rule under the candidate value \(\theta\). The numerator accounts jointly for the combined estimate and the transition event; replacing it by an unconditional distribution discards the design feature that matters.

For observed combined estimate \(\hat{\theta}_{obs}\), a central \(100(1-\alpha)\%\) conditional interval is obtained by solving

F_{\theta_L}(\hat{\theta}_{obs}\mid A)=1-\frac{\alpha}{2}, \qquad F_{\theta_U}(\hat{\theta}_{obs}\mid A)=\frac{\alpha}{2}.

The direction of the equations reflects that, for this normal location model, the conditional distribution shifts upward as \(\theta\) increases. Inverting the conditional CDF gives \(P_\theta(\theta_L\le\theta\le\theta_U\mid A)=1-\alpha\), under the stated model and a continuous, invertible CDF. This is a repeated-sampling coverage statement for trials that pass the rule; it does not assign a probability to the fixed parameter after observing the interval.

Deriving the Conditional CDF

A two-stage special case shows exactly where the adaptation enters. Let \(Z_1,Z_2\) be independent \(N(\theta,1)\) variables, continue if \(Z_1>c\), and use \(S=Z_1+Z_2\) as the combined statistic. For a candidate parameter value \(\theta\), set \(U_j=Z_j-\theta\), so \(U_1,U_2\) are independent standard normal variables. The event \(Z_1>c\) becomes \(U_1>c-\theta\); the event \(S\le s\) becomes \(U_1+U_2\le s-2\theta\). Conditioning and integrating over \(U_1\) therefore gives

P_\theta(S\le s\mid Z_1>c) =\frac{\displaystyle\int_{c-\theta}^{\infty} \phi(u)\Phi(s-2\theta-u)\,du} {1-\Phi(c-\theta)}.

This expression is a conditional normal probability, not a standard normal CDF. Inverting it generally requires numerical integration and a one-dimensional root search. A useful check is to compute the conditional tail probability directly as well: the lower endpoint must leave an upper tail of \(\alpha/2\), and the upper endpoint must leave an upper tail of \(1-\alpha/2\).

Worked Example: Inverting the Transition-Conditional Distribution

Suppose \(Z_1,Z_2\) are independent \(N(\theta,1)\) stage statistics. The trial continues if \(Z_1>0\), and the observed sum is \(S=1.2\), so \(\hat{\theta}_{obs}=S/2=0.600\). Find a central 90% interval for \(\theta\), conditional on continuation.

Here \(c=0\) and \(\alpha=0.10\). The conditional CDF at the observed sum is

F_\theta(1.2\mid Z_1>0) =\frac{\displaystyle\int_{-\theta}^{\infty} \phi(u)\Phi(1.2-2\theta-u)\,du}{\Phi(\theta)}.

The denominator is \(\Phi(\theta)\), because \(P_\theta(Z_1>0)=\Phi(\theta)\). Numerical integration and root-finding give the lower endpoint from \(F_\theta=0.95\) and the upper endpoint from \(F_\theta=0.05\):

\theta_L\approx-1.12,\qquad \theta_U\approx1.63.

For a numerical check on the lower endpoint, at \(\theta=-1.20\) the transformed continuation event is \(U_1>1.20\), and \(S>1.2\) becomes \(U_1+U_2>3.6\). The conditional upper-tail probability is approximately \(P(U_1+U_2>3.6,\ U_1>1.2)/P(U_1>1.2)\approx0.0416\), so the conditional CDF is about \(0.9584\), not \(0.95\). The lower root is therefore near \(-1.12\), not \(-1.20\). At the reported endpoints, numerical evaluation gives conditional CDF values approximately \(0.950\) and \(0.050\), respectively.

The resulting interval, in the standardized units used for \(\theta\), is approximately \([-1.12,1.63]\). It is asymmetric around the observed estimate \(0.600\), as expected: the interval accounts for conditioning on \(Z_1>0\), whereas a symmetric Wald interval does not.

Combining Estimates With Unequal Stage Information

When stage information differs, the combined estimate is an information-weighted average, not generally the simple average of the two stage estimates. The transition statistic and the combined estimate also have different roles: the former determines whether phase 3 begins, while the latter uses both stages to estimate the effect.

Worked Example: Weighted Estimate After a Phase 2 Gate

Suppose the phase 2 and phase 3 information values are \(I_1=25\) and \(I_2=75\). The observed stage statistics are \(Z_1=1.70\) and \(Z_2=1.10\); the prespecified continuation threshold is \(c=1.50\). Assume the common-effect normal model and known information values.

The phase 2 estimate is \(\hat{\theta}_1=Z_1/\sqrt{I_1}=1.70/5=0.340\). The phase 3 estimate is \(\hat{\theta}_2=Z_2/\sqrt{I_2}=1.10/\sqrt{75}\approx0.1270\). Since \(1.70>1.50\), the transition rule is met. The combined estimate is

\hat{\theta} =\frac{25(0.340)+75(0.1270)}{100} \approx0.1803.

Its unadjusted standard error under the model is \(1/\sqrt{100}=0.1000\), in the treatment-effect units used for the stage estimates. These calculations identify the estimate and its ordinary standard error, but they do not by themselves give a conditional interval. To make the interval conditional on continuation, its endpoints must be obtained from the joint normal distribution of the weighted statistic and the event \(Z_1>1.50\), using the conditional-CDF inversion above.

Why the Transition Rule Must Be Reconstructed

The denominator and numerator of the conditional distribution depend on the actual prespecified rule. If continuation requires \(Z_1>c\), use that event. If the protocol uses a different cutoff, a futility boundary as well as an efficacy boundary, or a more complicated transition condition, the conditioning event must represent that rule exactly. Treating every adaptation as a simple efficacy threshold can produce an interval for the wrong conditional distribution.

The rule must also be distinguished from the observed interim value. Conditioning on the event \(Z_1>c\) describes coverage among all trials that pass the gate. Conditioning on the exact observed \(Z_1\) is a different inferential target: it treats more interim information as fixed and requires a distribution conditional on that value. The interval derived here makes only the first guarantee.

Worked Example: Checking Which Rule Was Applied

Suppose a protocol specifies phase 3 continuation only if \(Z_1>1.00\). A hypothetical trial records \(Z_1=1.20\) and \(Z_2=0.80\), with \(I_1=4\) and \(I_2=5\). First check the gate: \(1.20>1.00\), so phase 3 continuation is consistent with the rule.

The stage estimates are \(\hat{\theta}_1=1.20/\sqrt{4}=0.600\) and \(\hat{\theta}_2=0.80/\sqrt{5}\approx0.3578\). Thus

\hat{\theta} =\frac{4(0.600)+5(0.3578)}{9} \approx0.4654, \qquad SE(\hat{\theta})=\frac{1}{\sqrt{9}}=0.3333.

The conditional CDF must use \(P_\theta(Z_1>1.00)=1-\Phi(1.00-\theta\sqrt{4})\) in its denominator and the joint event involving the weighted statistic in its numerator. If the recorded interim statistic had instead been \(0.90\), phase 3 continuation would conflict with the stated rule; it would be unsafe to calculate an interval as if the rule had been followed without resolving the discrepancy.

Diagnostics, Alternatives, and Limitations

Before reporting a conditional interval, check the design and analysis inputs:

Rule and path: Verify the cutoff, which statistic was used, and that the observed transition decision follows the recorded rule.
Common estimand: Confirm that both stages target the same treatment contrast, outcome definition, and population. If the effect changes between stages or the target population shifts, a single common \(\theta\) may not describe both stages.
Independence and information: Check that stagewise samples are independent as assumed and that the information values used in the calculation are justified. Reuse of participants or unaccounted covariance changes the joint distribution.
Numerical inversion: Check that the conditional CDF is evaluated using the actual gate and that the endpoint equations are satisfied to the reported precision. A plausible-looking interval is not a substitute for checking the tail probabilities.

A naive Wald interval uses \(\hat{\theta}\pm z_{1-\alpha/2}SE\) and is straightforward, but it does not generally account for conditioning on continuation. A conditional interval is designed for coverage given the specified transition event and may be asymmetric or longer. An unconditional interval instead targets coverage averaged over the adaptation outcomes, including trials that do not continue; it answers a different question. The introductory material on confidence interval coverage and its conditioning distinguishes these coverage targets.

Conditional intervals have limitations. They rely on a correctly specified sampling model and an accurately recorded transition rule; if the assumed normality, known information, independence, or common-effect model is poor, the nominal conditional coverage may not hold. Conditioning on a rare continuation event can also make the denominator small, increasing numerical sensitivity and often producing wide intervals. The interval is not a remedy for unrecorded adaptations, changing estimands, or an unsuitable model.

Common mistake: “The combined estimate is a weighted average, so its ordinary standard error gives a valid interval after the phase 2 gate.” The weighted average and its unconditional standard error are useful summaries, but they do not account for selection into phase 3. For conditional coverage, calculate the distribution of the combined statistic given the prespecified transition event and invert that distribution.

Key takeaway: A seamless phase 2/3 interval must match both the estimand and the phase-transition rule. Conditional limits come from inverting the combined statistic's distribution given that the rule was met, with coverage interpreted among trials satisfying that rule.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

In the normal model, why does the conditional CDF denominator depend on \(\theta\)?
A 90% conditional interval uses which two CDF values at the observed combined estimate?
With \(I_1=16\), \(I_2=84\), and \(Z_1=2.00\), \(Z_2=1.50\), calculate the combined estimate and its unadjusted standard error.
What coverage population does an interval conditional on continuation describe?
Name two assumptions whose failure can invalidate the stated conditional coverage.

Solutions. (1) The probability of passing the gate is parameter-dependent; for \(Z_1>c\), it is \(1-\Phi(c-\theta\sqrt{I_1})\). (2) The lower endpoint solves \(F_\theta(\hat{\theta}_{obs}\mid A)=0.95\), and the upper solves it at \(0.05\). (3) The estimate is \((\sqrt{16}(2.00)+\sqrt{84}(1.50))/100=(8+13.748)/100\approx0.2175\); the unadjusted standard error is \(1/\sqrt{100}=0.1000\). (4) It describes repeated trials that meet the specified continuation rule. (5) Examples include a failure of stagewise independence, a changing treatment-effect estimand across stages, misspecified normal sampling distributions, or an incorrectly recorded transition rule.