Binary Endpoint Example With Unequal Stage Sizes

Why the Stagewise Analysis Needs a Plan

In a seamless phase 2/3 trial, the first stage can serve two purposes: it helps identify a treatment to continue and may contribute evidence to that treatment’s confirmatory test. Those purposes create a statistical challenge. The treatment is chosen using first-stage outcomes, so the selected treatment’s first-stage result is not an ordinary prespecified result. A confirmatory analysis must account for that selection rather than treat the winning result as if it had been chosen in advance.

This tutorial works through a two-stage design with a binary endpoint, such as whether a participant achieves a defined clinical response. It uses separate, non-overlapping participant groups at the two stages and a prespecified inverse-normal combination test. Unequal stage sizes are handled by fixed weights based on planned information. A Bonferroni adjustment protects the familywise type I error across candidate treatments.

Design and model: There are \(K\) candidate treatments and a concurrent control. For candidate \(k\), let \(p_{Tk,s}\) and \(p_{C,s}\) be the response probabilities for treatment and control in stage \(s\), where \(s=1,2\). Higher response is better. The one-sided hypotheses are \(H_{0k}: p_{Tk,s} \leq p_{C,s}\) against \(H_{ak}: p_{Tk,s} > p_{C,s}\). Within each stage, outcomes are Bernoulli, participants are randomized, and the stage 1 and stage 2 participant groups are disjoint. The endpoint definition and direction of benefit remain the same across stages.

The score tests below are large-sample tests. They require enough expected successes and failures for their normal approximations to be reasonable. Stage 1 selection and the rule for choosing a treatment must be set in advance; for example, continue the candidate with the largest stage 1 score statistic, breaking ties by a fixed rule. The stage 2 sample sizes and combination weights must also be fixed before observing stage 1 results. Stagewise p-values are independent across stages for a given candidate under the null because they use outcomes from disjoint randomized groups.

Stagewise Tests and Fixed Combination Weights

For candidate \(k\) in stage \(s\), let \(x_{Tk,s}\) and \(x_{C,s}\) be the numbers of responses among \(n_{Tk,s}\) treated participants and \(n_{C,s}\) control participants. The observed response proportions are \(\hat p_{Tk,s}=x_{Tk,s}/n_{Tk,s}\) and \(\hat p_{C,s}=x_{C,s}/n_{C,s}\). The pooled proportion under the null of equal response probabilities is \(\hat p_{k,s}=(x_{Tk,s}+x_{C,s})/(n_{Tk,s}+n_{C,s})\).

Z_{k,s} = \frac{\hat p_{Tk,s}-\hat p_{C,s}} {\sqrt{\hat p_{k,s}(1-\hat p_{k,s}) \left(\frac{1}{n_{Tk,s}}+\frac{1}{n_{C,s}}\right)}}, \qquad p_{k,s}=1-\Phi(Z_{k,s}).

A large positive \(Z_{k,s}\) supports a higher response probability on treatment. The one-sided p-value \(p_{k,s}\) is the upper-tail probability from the standard normal approximation. Under the boundary null of equal probabilities it is approximately uniform; under the composite null of no treatment benefit it is conservative, subject to the score-test approximation.

For unequal stage sizes, assign fixed nonnegative weights \(w_1,w_2\) satisfying \(w_1^2+w_2^2=1\). A standard information-based choice uses a prespecified planning response probability \(p_0\) under the null. Write \(v_0=p_0(1-p_0)\) and define the planned information for a treatment-control comparison in stage \(s\) by \(I_s=1/\{v_0(1/n_{T,s}+1/n_{C,s})\}\). Then \(w_s=\sqrt{I_s/(I_1+I_2)}\). The common variance factor cancels when the same \(p_0\) is used at both stages, so the relative weights depend on the treatment and control sample sizes. These are design weights, not weights recalculated from observed stage results.

Combination test: For a candidate with stagewise p-values \(p_{k,1}\) and \(p_{k,2}\), define \(Z_{k,\mathrm{comb}}=w_1\Phi^{-1}(1-p_{k,1})+w_2\Phi^{-1}(1-p_{k,2})\). With independent, continuous uniform stagewise p-values under \(H_{0k}\), this statistic is standard normal under that null. For \(K\) candidate treatments and overall one-sided level \(\alpha\), a Bonferroni rule rejects \(H_{0k}\) only if \(Z_{k,\mathrm{comb}}>\Phi^{-1}(1-\alpha/K)\).

Worked Selection and Combination

Worked Example: Selecting Between Two Treatments

Suppose a hypothetical trial has two candidates, A and B. In stage 1, the control group has 30 responses among 100 participants. A has 40 among 80, and B has 36 among 80. Selection is based on the larger one-sided stage 1 score statistic.

For A, \(\hat p_{A,1}=40/80=0.5000\), \(\hat p_{C,1}=30/100=0.3000\), and the pooled proportion is \(70/180=0.3889\). Thus

Z_{A,1} = \frac{0.5000-0.3000} {\sqrt{0.3889(1-0.3889)(1/80+1/100)}} = \frac{0.2000}{0.07312} =2.735.

For B, the pooled proportion is \(66/180=0.3667\), and

Z_{B,1} = \frac{36/80-30/100} {\sqrt{0.3667(1-0.3667)(1/80+1/100)}} = \frac{0.1500}{0.07228} =2.075.

Because \(2.735>2.075\), the prespecified rule selects A. The approximate stage 1 one-sided p-values are \(p_{A,1}\approx0.0031\) and \(p_{B,1}\approx0.0190\). The winning stage 1 p-value is not, by itself, a confirmatory p-value: selection favored the larger observed statistic.

Worked Example: Combining Evidence With Unequal Stage Sizes

Continue the hypothetical trial above. Stage 2 enrolls 160 participants on selected treatment A and 200 on control; 76 on A and 54 on control respond. For the weight calculation, use the prespecified planning value \(p_0=0.30\). The treatment-control sample sizes are 80 and 100 in stage 1, then 160 and 200 in stage 2.

The stage 2 pooled response proportion is \(130/360=0.3611\). Its score statistic and p-value are

Z_{A,2} = \frac{76/160-54/200} {\sqrt{0.3611(1-0.3611)(1/160+1/200)}} = \frac{0.2050}{0.05095} =4.024, \qquad p_{A,2}\approx 0.0000286.

Since \(p_0(1-p_0)=0.30(0.70)=0.21\), the planned information values are

I_1=\frac{1}{0.21(1/80+1/100)}=211.6, \qquad I_2=\frac{1}{0.21(1/160+1/200)}=423.3.

Therefore \(w_1=\sqrt{211.6/(211.6+423.3)}=0.5774\) and \(w_2=\sqrt{423.3/(211.6+423.3)}=0.8165\). Transforming the stagewise p-values back to normal scores gives approximately 2.735 and 4.024. The combined statistic is

Z_{A,\mathrm{comb}} = 0.5774(2.735)+0.8165(4.024) =4.865.

With \(K=2\) and overall one-sided \(\alpha=0.05\), the Bonferroni cutoff is \(\Phi^{-1}(0.975)=1.960\). Since \(4.865>1.960\), reject \(H_{0A}\). The combined p-value is approximately \(5.7\times10^{-7}\), below the candidate-level threshold \(0.025\). Under the stated model and approximations, the data provide strong evidence that A increases the response probability relative to control. This conclusion concerns the randomized comparison; it does not establish benefit outside the trial population or endpoint definition.

Why the Combination Calibration Works

The central justification is that the normal-score transform of a continuous uniform p-value is standard normal. For a fixed candidate under its null, independent stagewise p-values therefore give independent \(U_{k,s}=\Phi^{-1}(1-p_{k,s})\sim N(0,1)\). A weighted sum of independent normal variables is normal, with variance equal to the sum of the squared weights. The choice \(w_1^2+w_2^2=1\) makes that variance one:

w_1U_{k,1}+w_2U_{k,2} \sim N(0,w_1^2+w_2^2)=N(0,1).

Thus the fixed-candidate combination test has level \(\alpha/K\) at its specified cutoff. To see why selection among candidates does not invalidate the overall bound, consider the event of falsely rejecting any true null. It is contained in the union of the candidate-specific false-rejection events. The union bound gives

P(\text{at least one false rejection}) \leq \sum_{k\text{ true null}} P(\text{reject }H_{0k}) \leq K\frac{\alpha}{K} =\alpha.

This bound does not require independent candidate tests. It remains valid when stage 1 results are correlated, as they may be when candidates share a control group. The argument relies on valid candidate-specific stagewise tests, independent stage data for the combination, fixed weights, and the prespecified multiplicity rule. Discrete or approximate stagewise p-values may require conservative or design-specific calibration rather than assuming exact continuous uniformity.

Worked Example: A Larger Second Stage Gets More Weight

Consider a separate hypothetical design with stage 1 treatment and control sizes 60 and 75, and stage 2 sizes 240 and 300. Suppose \(p_0=0.28\), so \(v_0=0.28(0.72)=0.2016\). The information values are

I_1=\frac{1}{0.2016(1/60+1/75)}=165.3, \qquad I_2=\frac{1}{0.2016(1/240+1/300)}=661.4.

The second stage has four times the information, so \(w_1=\sqrt{1/5}=0.4472\) and \(w_2=\sqrt{4/5}=0.8944\). Suppose the selected candidate has stagewise statistics \(Z_1=2.427\) and \(Z_2=3.198\). Its combined statistic is

Z_{\mathrm{comb}}=0.4472(2.427)+0.8944(3.198)=3.946.

The candidate-specific one-sided p-value is approximately \(0.000040\), which is below \(0.025\) for two candidates at overall \(\alpha=0.05\). The larger stage contributes more because its planned comparison has more information, not simply because it occurs later.

Diagnostics, Alternatives, and Common Mistakes

Check the analysis plan and data before interpreting the combined statistic. The endpoint, response direction, eligibility criteria, and outcome assessment should be comparable across stages. Confirm the actual stage-specific denominators, response counts, and randomization groups; a change in those inputs changes each score test. Ensure that the stage 2 comparison is based on the selected treatment and concurrent stage 2 control, and that the combination weights and selection rule were fixed before stage 1 outcomes were reviewed.

The inverse-normal calculation is not automatically reliable with very sparse binary outcomes. If expected successes or failures are small, a normal score approximation can be inaccurate. A prespecified exact or other calibrated stagewise test may be preferable, followed by a combination procedure whose null calibration is justified for those p-values. Likewise, unplanned interim looks, outcome-dependent changes to weights, or a change in endpoint definition require an analysis that accounts for those design choices; the simple two-stage formula does not do so.

A valid alternative is to base confirmation solely on the independent stage 2 comparison, using a multiplicity-adjusted level. That approach avoids reusing selected stage 1 evidence but may discard useful information. The combination test uses both stages under a specified rule. More tailored conditional-error or closed-testing procedures can also control error, but they require their own prespecified calibration and are not interchangeable with an unadjusted combination test.

Common mistake: test the selected treatment at the ordinary level. The winner was chosen for a high stage 1 result, so treating its result as prespecified and testing at \(\alpha\) can inflate the chance of a false claim. Use a selection-valid procedure, such as the fixed candidate-level combination tests with the stated multiplicity adjustment.
Common mistake: use weights based on observed effects. Choosing weights after seeing which stage looks more favorable changes the test’s calibration. Use fixed weights based on planned information or another prespecified valid rule.
Common mistake: pool the two stages and apply an ordinary two-sample test. Selection, changing stage sizes, and the adaptive treatment allocation mean the pooled statistic need not have the assumed reference distribution. Use a prespecified stagewise method that accounts for the adaptation.

Key takeaway: Unequal stage sizes are handled by planned information weights, but those weights alone do not correct for selecting a treatment. Valid confirmation also requires calibrated stagewise tests, independent stage data, a prespecified selection rule, and multiplicity control across candidate treatments.

Check Your Understanding

Answer each question using the assumptions and procedure stated above. Solutions follow the questions.

Why does the combination statistic have unit variance when the stagewise normal scores are independent and the weights satisfy \(w_1^2+w_2^2=1\)?
For \(K=3\) candidate treatments and one-sided overall \(\alpha=0.05\), what candidate-level significance threshold does the Bonferroni rule use?
Suppose stage 2 has four times the planned information of stage 1. What are the information-based weights?
Why can a valid analysis not simply select the treatment with the largest stage 1 statistic and test it at the unadjusted overall level?

Solutions. (1) The variance of the weighted sum of independent unit-variance scores is \(w_1^2+w_2^2=1\). (2) The threshold is \(0.05/3=0.01667\) for each candidate; the corresponding one-sided normal cutoff is approximately \(\Phi^{-1}(0.98333)=2.128\). (3) The information proportions are \(1/5\) and \(4/5\), so \(w_1=\sqrt{1/5}=0.4472\) and \(w_2=\sqrt{4/5}=0.8944\). (4) Selection favors a high stage 1 result, so the selected result is not distributed like the result for a treatment fixed in advance. A selection-valid combination procedure and multiplicity control are needed.