Worked Example: Combining Two Stages After Selection

Why Combining Stages Needs Care

When two treatment arms are compared at an interim analysis, selecting the arm with the larger estimated effect also selects, in part, the arm with a favourable random error. If that same interim estimate is carried forward into the final estimate, its error does not disappear merely because new data are collected. The stage-two data are not selected in the same way, but the combined estimate still gives some weight to the interim result.

This tutorial works through that issue in a two-arm normal model. The target is the mean effect of the arm selected at the interim analysis, relative to a common control or reference. The selected arm is random, so its true effect is also a random quantity across repetitions of the design. A model-based adjustment targets the expected upward selection error under specified treatment means and variances; it does not reveal the true effect in a particular trial.

Model and Selection-Bias Result

Assume that the stage-one estimates are independent and satisfy \(X_k=\theta_k+\varepsilon_k\), where \(\varepsilon_A,\varepsilon_B\sim N(0,\tau^2)\). Assume also that stage two is independent of stage one and, conditional on the selected arm, \(Y_J\sim N(\theta_J,\sigma^2/I_2)\). At stage one, the variance can be written \(\tau^2=\sigma^2/I_1\). These assumptions give the information-weighted combination above its usual normal-model interpretation. The selection rule must be the stated rule, and the means and variances used to calculate an adjustment must describe that rule and model adequately.

Write \(\Delta=\theta_A-\theta_B\). Under this model, the expected error in the selected stage-one estimate is

where \(\phi\) is the standard normal density. The result is nonnegative whether arm A or arm B has the larger true mean: selection favours an estimate with a positive error. The stage-two estimate has mean \(\theta_J\) under the independence assumption, so the expected bias of the combined estimate for the selected-arm target is

This is an expectation over repetitions of the full selection-and-estimation procedure under fixed model parameters. It is not a claim that the observed estimate in one trial must exceed its target by this amount.

Why the Expected Error Has This Form

A short derivation makes clear where selection enters. Write \(X_A=\theta_A+\varepsilon_A\) and \(X_B=\theta_B+\varepsilon_B\), and define \(M=(\varepsilon_A+\varepsilon_B)/2\) and \(Q=(\varepsilon_A-\varepsilon_B)/2\). Under independent, equal-variance normal errors, \(M\) and \(Q\) are independent, \(E(M)=0\), and \(Q\sim N(0,\tau^2/2)\). The selection event for A is \(\Delta+2Q\ge0\). The selected error is \(M+Q\) on this event and \(M-Q\) otherwise. Therefore its expectation is \(E[Q\,\mathrm{sign}(\Delta+2Q)]\). Since \(E(Q)=0\), this equals \(2E[Q\,1\{Q\ge-\Delta/2\}]\). For a centred normal variable with standard deviation \(s\), \(E[Q\,1\{Q\ge c\}]=s\phi(c/s)\). Taking \(s=\tau/\sqrt{2}\) and \(c=-\Delta/2\), and using the symmetry of \(\phi\), gives the stated result.

The combination is a fixed weighted average once \(I_1\) and \(I_2\) are specified. The stage-two contribution has zero expected error relative to \(\theta_J\), while the stage-one contribution has the selection error just derived. Multiplying that error by \(I_1/(I_1+I_2)\) gives the combined-estimate bias formula.

Worked Example: A Moderate Difference Between Arms

Worked Example: A Larger Difference Reduces the Correction

Worked Example: More Stage-Two Information

Diagnostics, Alternatives, and Limitations

Before applying the formula, check the assumptions that determine it. Confirm that the interim rule really selects the larger of the two estimates, that the stage-one errors are approximately normal with the stated common standard error, and that stage-two data are independent of the selection process. Confirm also that the information weights correspond to the analysis actually used. If allocation, variances, or selection rules differ between arms, this simple symmetric formula need not apply.

Useful warning signs include an interim rule that uses a transformed or adjusted statistic rather than \(X_A-X_B\), a stage-two sample affected by the interim outcomes, substantial variance inequality, or an estimate of the mean difference that is inconsistent with the model used for the correction. In those situations, the calculated adjustment may not target the actual selection bias. The formula is also sensitive to the assumed \(\Delta\): if it is estimated from the same selected data, treating it as known understates uncertainty in the adjustment. A reported adjusted estimate should identify the model and inputs used; the subtraction alone does not account for uncertainty in estimated nuisance parameters.

The main alternative is to estimate the selected arm using stage-two data alone. Under the model, \(Y_J\) is unbiased for \(\theta_J\) because stage two is independent of selection, but it discards the interim information and may be less precise. The naive combined estimate uses both stages but retains the expected selection error. The model-adjusted combined estimate retains both stages and subtracts the modelled expected error, but depends on the model and adjustment inputs being adequate.

The normal result is a benchmark for this two-arm, two-stage setting, not a universal correction for all adaptive designs. It assumes a fixed selection rule, independent equal-variance stage-one estimates, and independent stage-two data. If those assumptions fail, the expected selected error must be derived under the actual design rather than borrowed from this formula. The adjustment targets mean bias under the assumed model; it does not by itself give a confidence interval with correct coverage.