Worked Design Workshop for a Seamless Phase 2/3 Trial

From Clinical Objective to a Two-Stage Design

A seamless Phase 2/3 trial must answer two connected questions: which candidate treatment should continue, and whether the selected treatment improves the outcome enough to support a confirmatory conclusion. The transition rule links those questions. If it is chosen after examining the interim results, the final analysis must account for that selection; an ordinary pooled test does not do so.

This workshop develops one complete, hypothetical design. The endpoint is change from baseline in a symptom score at 12 weeks, with larger values representing greater improvement. Stage 1 compares two candidate treatments, A and B, with a shared control. A pre-specified rule selects at most one candidate. Stage 2 randomizes new participants between the selected treatment and a new control group. Stage 1 results contribute to the final test through a specified combination procedure.

Define the Estimand and Assumptions

The objective is to test whether the selected treatment improves the mean 12-week change score relative to control. Define \(\theta_k\) as the mean change score under assignment to candidate \(k\) minus the mean change score under assignment to control, for \(k\in\{A,B\}\), in the trial's eligible population. The treatment-policy strategy is used: outcomes are analyzed according to randomized assignment, regardless of treatment discontinuation or additional care. The estimand is measured in symptom-score points; positive \(\theta_k\) favors the candidate.

For the numerical design and examples below, assume randomized participants are independent, the outcome is approximately normal within each arm, and the standard deviation is 10 score points in every arm. Treat this standard deviation as known for the worked \(Z\)-tests. The two stages enroll different participants, and Stage 2 data are independent of Stage 1 data conditional on the selected treatment. The shared Stage 1 control can make the two Stage 1 treatment comparisons dependent; the multiplicity adjustment below does not require those comparisons to be independent. These assumptions are planning simplifications. A real design should use the planned analysis model and examine departures from its assumptions.

Write the Selection and Testing Rules

At the end of Stage 1, calculate one-sided \(Z\)-statistics for each candidate against the shared control. Select the candidate with the larger Stage 1 \(Z\)-statistic if that statistic is positive and the candidate satisfies the pre-specified safety and feasibility criteria. Otherwise, stop for lack of a suitable candidate. No safety threshold is invented from the efficacy results: the safety criteria and the information used to assess them must be written in the protocol.

For the selected candidate, obtain its Stage 1 one-sided p-value, \(p_{1,\mathrm{raw}}\), and adjust it for the two candidates using Bonferroni: \(p_1=\min(1,2p_{1,\mathrm{raw}})\). Obtain the one-sided Stage 2 p-value, \(p_2\), by comparing the selected treatment with the new Stage 2 control. Combine the two values using fixed inverse-normal weights:

Here \(\Phi\) is the standard normal cumulative distribution function. Reject the selected treatment's null hypothesis if \(Z_{\mathrm{comb}}>z_{0.975}=1.960\), giving a one-sided significance level of \(\alpha=0.025\). If no treatment is selected, there is no confirmatory rejection. The weights are fixed from planned information, not chosen after seeing outcomes. For independent Stage 1 and Stage 2 comparisons with equal variance, use \(w_j=\sqrt{I_j/(I_1+I_2)}\), where \(I_j\) is the planned information in stage \(j\).

Why the Test Controls Error

A brief justification shows what the adjustment accomplishes. Under the global null, each Stage 1 one-sided p-value is valid. Bonferroni gives \(P(\min(p_{A,1},p_{B,1})\leq u/2)\leq u\), because the probability of either event is no greater than the sum of their probabilities. Thus, twice the smaller raw p-value, capped at 1, is a valid p-value for the selected Stage 1 result. Under a partial null, when the selected candidate is truly null, the event that its adjusted p-value is at most \(u\) implies its own raw p-value is at most \(u/2\); that event has probability no greater than \(u/2\). This remains true even when the Stage 1 comparisons share a control.

Stage 2 uses new participants, so under a selected candidate's null, \(p_2\) is uniform or conservative conditional on all Stage 1 data and the selection. Under the global null, the selected Stage 1 Bonferroni value (set to 1 if the rule stops) is valid; for a mathematical bound, extend \(p_2\) by an independent uniform value when the rule stops. The resulting independent valid p-values give a level-0.025 inverse-normal test, and the actual rule cannot reject when it stops. Under a partial null with exactly one null candidate, selection requires a positive Stage 1 statistic, so for every \(u\in[0,1]\), \(P(\text{null candidate selected},p_1\leq u)\leq P(p_{\mathrm{null,raw}}\leq u/2)\leq u/2\). Combining this sub-CDF bound with the conditionally valid independent \(p_2\) bounds rejection on that selected-null event by \(\alpha/2=0.0125\). Thus the false-positive probability for any null candidate is at most 0.025. This argument relies on the stated stage independence, valid stagewise p-values, fixed weights, and pre-specified rule; it is not a license to choose the combination method after looking at results.

Plan Sample Sizes and Check Operating Characteristics

For the example design, use 60 evaluable participants per arm in Stage 1 and 120 per arm in Stage 2. These sizes give standard errors of 1.826 and 1.291 points, respectively, under the assumed standard deviation of 10. If 10% of outcomes are expected to be missing, the corresponding arithmetic inflation is \(60/0.90=66.7\) expected recruits per Stage 1 arm and \(120/0.90=133.3\) expected recruits per Stage 2 arm. These are inflation calculations, not claims that a fractional number of people can be enrolled; the protocol must specify integer recruitment targets and reassess the operating characteristics using those targets.

As a simple benchmark, if the true effect for the selected treatment is 5 points, the Stage 2-only noncentrality is \(5/1.291=3.873\). A one-sided Stage 2-only test at 0.025 would have approximate power \(\Phi(3.873-1.960)=\Phi(1.913)\approx0.972\). This benchmark is not the power of the complete adaptive design. The selection step, adjusted Stage 1 p-value, and final combination rule all affect that power.

Before finalizing the design, evaluate the complete algorithm by simulation or an appropriate joint probability calculation. Simulate outcomes under clinically relevant scenarios, including the global null, one effective candidate and one null candidate, both effective candidates, and effects near the selection boundary. Record the probability of selecting each candidate, stopping, rejecting each true null, and reaching the target power; also record expected sample size. Under each null scenario, check that the simulated false-positive rate is compatible with 0.025, allowing for Monte Carlo error. Repeat checks when the standard deviation, missingness, or Stage 1 and Stage 2 outcome distributions depart from planning assumptions. Earlier material on simulating Type I error and stress-testing dependence provides relevant diagnostic context.

Alternatives, Diagnostics, and Limitations

The combination test is one defensible way to reuse Stage 1 evidence while accounting for selection. A Stage 2-only test is simpler and remains valid after Stage 1 selection because its data are independent, but it discards confirmatory information from Stage 1. A conditional-error approach can permit other adaptations while preserving the original test's error rate, but its conditional error calculation and adaptation rules must be specified in advance. By contrast, an ordinary pooled test treating the selected Stage 1 comparison as if it had been chosen independently can inflate Type I error.

Warning signs include choosing the candidate using outcomes or thresholds not specified in the algorithm, using a Stage 2 control group whose data overlap with Stage 1, changing weights after seeing treatment effects, or observing simulated Type I error materially above 0.025. Also inspect whether selection probabilities are stable across plausible effect and variance scenarios. A design can control Type I error yet have poor power or frequently select the less effective candidate.

The numerical design relies on normal approximations, a common known standard deviation, and a particular missingness allowance. In practice, variance estimation, baseline adjustment, non-normal outcomes, delayed outcomes, and the exact handling of missing data can change the stagewise p-values and information weights. The final protocol should define the analysis model and validate the full decision algorithm under those features. The combination procedure controls testing error under its assumptions; it does not eliminate selection bias in a naïvely reported pooled treatment-effect estimate.