Sample Size Planning With Intermediate Selection

Why selection changes a sample size calculation

When an experimental arm must pass an intermediate-outcome threshold to continue, final power is not just the probability that the final test rejects under the alternative. The arm must first be retained. If the intermediate outcome only imperfectly predicts the final outcome, some arms with a beneficial final effect will be dropped, and some retained arms will not have a beneficial final result. Planning must therefore account for both stages and their dependence.

The relevant target depends on the question. The probability of retaining an arm describes how often the interim rule allows it to continue. The probability of final success and retention measures the chance of success from the start of the trial. These are different quantities. In the terminology of the earlier material on power definitions after intermediate selection, the latter is selection-adjusted, or unconditional, power.

Definition: For an experimental arm, retention probability is the probability that its intermediate statistic passes the prespecified selection rule. Selection-adjusted power is the probability, under a specified alternative, that the arm is retained and its final statistic passes the prespecified final efficacy rule. Conditional power after selection is the probability of passing the final rule given that the arm was retained. These definitions refer to a specified model, treatment effect, and trial design.

A planning model for retention and final success

Consider one experimental arm compared with a control arm. Let \(Z_I\) be a standardized intermediate-outcome statistic, with larger values favoring the experimental arm, and let \(Z_F\) be a standardized final-outcome statistic oriented in the same direction. Suppose the arm is retained when \(Z_I\geq c\) and is declared efficacious at the final analysis when \(Z_F>z_F\). The value \(z_F\) is the critical value from the prespecified final testing rule; in a multi-arm design it must reflect the applicable multiplicity procedure.

For planning, assume the two statistics are approximately jointly normal under the alternative, each with variance 1, with means \(\delta_I\) and \(\delta_F\) and correlation \(\rho\). Thus the model specifies \(Z_I\sim N(\delta_I,1)\), \(Z_F\sim N(\delta_F,1)\), and correlation \(\rho\), where \(-1<\rho<1\). The means and correlation must represent the anticipated treatment effects, analysis, and timing. This approximation is most credible when the statistics are based on adequate information and their joint distribution is reasonably approximated by a bivariate normal distribution.

P(\text{retained})=P(Z_I\geq c)=\Phi(\delta_I-c)

Here \(\Phi\) is the standard normal cumulative distribution function. By conditioning on \(Z_I=x\), the final statistic has conditional distribution \(Z_F\mid Z_I=x\sim N\bigl(\delta_F+\rho(x-\delta_I),1-\rho^2\bigr)\). Consequently, the probability needed for selection-adjusted power is

P(Z_I\geq c,\ Z_F>z_F) = \int_c^\infty \frac{1}{\sqrt{2\pi}}\exp\left\{-\frac{(x-\delta_I)^2}{2}\right\} \Phi\left( \frac{\delta_F+\rho(x-\delta_I)-z_F}{\sqrt{1-\rho^2}} \right)\,dx.

This integral can be evaluated numerically. Conditional power after selection is the joint probability in the integral divided by \(\Phi(\delta_I-c)\), provided the retention probability is positive. The division is important: a high conditional power among retained arms does not, by itself, mean a high chance of success from the start of the trial.

Connecting the model to enrollment and event targets

The normal model turns a design target into a calculation. For example, under a proportional-hazards model with a beneficial hazard ratio \(HR<1\), define the favorable final effect as \(\theta=-\log(HR)>0\). With balanced randomization and \(D\) total events in the two groups, the usual large-sample log-rank approximation gives final information \(D/4\), so that

\delta_F\approx\theta\sqrt{\frac{D}{4}}=\frac{\theta\sqrt{D}}{2}.

This relationship assumes proportional hazards, a suitable large-sample approximation, and balanced allocation for the comparison. Other allocation ratios or analysis methods require their corresponding information formula. The intermediate mean \(\delta_I\) likewise depends on the planned interim sample size and the intermediate effect. The correlation \(\rho\) is a planning input: it should be justified for the chosen statistics and data-generating assumptions, not treated as automatically equal to a correlation between raw outcomes.

A practical calculation searches over candidate interim sample sizes and event targets. At each candidate, compute \(\delta_I\) and \(\delta_F\), evaluate retention probability and the joint probability, and select targets that meet the prespecified objectives. An event target is not itself an enrollment target. Converting between them requires assumptions about event incidence, follow-up, censoring, and accrual.

Specify the design rules.
Fix the intermediate threshold, final testing rule, allocation, timing, and the power target. Use the final critical value appropriate to the design.

State planning assumptions.
Specify the intermediate and final effects, their joint dependence, event and censoring assumptions, and the intended follow-up horizon.

Find candidate targets.
For each interim sample size and event target, calculate the model means and the retention and joint-success probabilities.

Translate and check.
Convert the event target to approximate enrollment under the stated event model, then verify that interim enrollment, follow-up, and accrual are feasible.

Worked Example: Finding an Event Target for Selection-Adjusted Power

Suppose a hypothetical balanced comparison uses \(Z_I\geq 0.50\) for retention and \(Z_F>1.96\) for final efficacy. Under the planning alternative, let \(\delta_I=1.50\), \(\rho=0.40\), and \(HR=0.819\), giving \(\theta=-\log(0.819)\approx 0.200\). The target is selection-adjusted power of at least 0.80.

First, retention probability is \(\Phi(1.50-0.50)=\Phi(1.00)=0.8413\). For each candidate event target, use \(\delta_F=0.200\sqrt{D}/2\) in the joint-probability integral. At \(D=1{,}191\), \(\delta_F=0.100\sqrt{1{,}191}=3.451\). Substitution gives

\int_{0.50}^{\infty}\phi(x-1.50) \Phi\left(\frac{3.451+0.40(x-1.50)-1.96}{\sqrt{1-0.40^2}}\right)\,dx \approx 0.801,

where \(\phi\) is the standard normal density. Numerical integration therefore gives approximately 0.801 selection-adjusted power, just above the 0.80 target, with a retention probability of 0.841. The corresponding conditional power among retained arms is approximately \(0.801/0.8413=0.952\). The event target is based on the stated effect, threshold, critical value, and correlation; changing any of these inputs can change the required target.

Choosing an interim sample size for a retention objective

An interim sample size can also be planned directly against a desired probability of retaining an arm under a specified intermediate effect. For a one-sided threshold \(c\), requiring retention probability at least \(r\) means \(\Phi(\delta_I-c)\geq r\), or equivalently \(\delta_I\geq c+\Phi^{-1}(r)\). This is a retention objective, not a final-power guarantee: it says nothing by itself about the chance of meeting the final criterion.

Worked Example: Meeting a Retention Probability Target

Suppose the intermediate outcome is continuous, the treatment effect is a standardized mean difference \(d_I=0.25\), allocation is balanced, and there are \(n_I\) participants per group at the interim analysis. Under equal variances and the usual normal approximation, \(Z_I\) has variance approximately 1 and mean \(\delta_I=d_I\sqrt{n_I/2}\). The retention threshold is \(c=0.50\), and the target retention probability under this alternative is 0.80.

The required mean is \(\delta_I\geq 0.50+\Phi^{-1}(0.80)=0.50+0.8416=1.342\). Hence

n_I\geq 2\left(\frac{1.342}{0.25}\right)^2=57.6.

Rounding up gives a minimum of 58 participants per group, or 116 across this two-group comparison, at the interim analysis. Checking the rounded value, \(\delta_I=0.25\sqrt{58/2}=1.346\), so \(P(\text{retained})=\Phi(1.346-0.50)=\Phi(0.846)\approx0.801\). The target is met under the assumed effect and approximation. It is not a guarantee that the arm will be retained in any particular trial.

From events to approximate enrollment

To translate an event target to enrollment, define \(q\) as the probability that a randomized participant experiences the event by the planned follow-up horizon. If participants have a common \(q\) and outcomes are independent, the expected number of events among \(N\) randomized participants is \(Nq\). Dividing the target number of events by \(q\) gives an approximate enrollment, not a guarantee of observing that many events by a fixed date. If event probabilities differ by arm, use allocation-weighted event probabilities or a more detailed model.

Worked Example: Translating an Event Target Under Exponential Hazards

Suppose the comparison needs 1,191 total events and uses balanced allocation. For illustration, assume each participant has an event hazard of \(\lambda=0.08\) per year, an independent dropout hazard of \(\mu=0.02\) per year, and three years of follow-up. With constant hazards, the probability of an event before dropout and before the horizon is

q=\int_0^3 \lambda e^{-(\lambda+\mu)t}\,dt =\frac{\lambda}{\lambda+\mu}\left(1-e^{-(\lambda+\mu)3}\right) =0.8(1-e^{-0.30})\approx0.2073.

The approximate enrollment needed for 1,191 expected events is \(N=1{,}191/0.2073\approx5{,}745\) participants in total, or about 2,873 per group with balanced allocation (rounding group sizes to sum to the total). The conversion assumes the same event probability in both groups; if the experimental effect changes event incidence, arm-specific probabilities should be used. Under a fixed three-year horizon, this enrollment yields about the target number in expectation, not with certainty. An event-driven trial would instead continue follow-up until the prespecified event count is reached, subject to its operational rules.

Why the joint calculation matters

The derivation of the joint probability shows why treating selection and final testing as separate power calculations can be misleading. Given an intermediate statistic \(Z_I=x\), the conditional mean of \(Z_F\) shifts by \(\rho(x-\delta_I)\). When \(\rho>0\), arms that pass the threshold with stronger-than-average intermediate evidence tend also to have higher final statistics. The joint probability therefore depends on \(\rho\), not just on the two marginal means. If \(\rho=0\), the statistics are independent under the model and the joint probability reduces to \(P(Z_I\geq c)P(Z_F>z_F)\); that shortcut is not justified when the planned correlation is nonzero.

There are several legitimate planning summaries, but they answer different questions. Planning to meet a target for \(P(Z_F>z_F\mid Z_I\geq c)\) describes final success among the arms that survive selection. Planning to meet a target for \(P(Z_I\geq c,Z_F>z_F)\) describes success from trial entry. A design may report both, but should identify which one governs the sample size. A design that ignores selection and calculates ordinary final power for every arm can understate the enrollment or event target needed for a specified unconditional probability of success.

The calculation is also different from a rule that assumes the intermediate outcome perfectly predicts the final effect. That assumption would effectively remove the uncertainty represented by the joint distribution and could give overconfident retention and power projections. At the other extreme, planning only for retention can yield an adequate interim sample while leaving final power too low. A complete plan states the retention objective and the final success objective separately, then evaluates the joint probability.

Assumptions, diagnostics, and limitations

Before accepting a target, check that the model describes the planned design. In particular, verify that the intermediate and final statistics are oriented consistently, the thresholds and final critical value match the protocol, and the assumed \(\delta_I\), \(\delta_F\), and \(\rho\) correspond to the same treatment scenario. Confirm that the event-information relationship matches the final analysis and allocation. For enrollment conversion, check that the event, dropout, and follow-up assumptions correspond to the operational horizon.

Impossible or unstable inputs. A correlation outside \([-1,1]\), or a calculation that changes sharply for tiny numerical changes near a boundary, signals an input or numerical problem. Check the correlation and integration method.
Mismatch between target and calculation. If the protocol seeks success from randomization but the calculation reports only conditional power among retained arms, the reported quantity answers the wrong question. Calculate the joint probability instead.
Insufficient information for the normal approximation. Sparse events, small interim samples, or discrete intermediate outcomes may make the bivariate normal approximation poor. Consider a design-specific method that represents the actual data-generating and analysis procedures.
Uncertain planning effects or dependence. A target based on a single assumed effect and correlation can be fragile. The result is conditional on those inputs; it should not be presented as a guarantee of power in the realized trial.

In a multi-arm trial, arm-specific effects and selection probabilities can differ, and arms share a control group. The one-arm calculation illustrates the planning principle but does not by itself determine an overall design target for all arms. Apply the selection and final testing rules actually specified for the design, including their multiplicity structure. Nor does an expected enrollment calculation account for how the event process changes after arms are dropped; that requires a separate operational calculation.

Key takeaway: Plan against the probability that an arm both survives intermediate selection and meets the final efficacy rule when the objective is success from trial entry. Use the joint distribution of the two statistics, and report retention probability and conditional power separately so their meanings remain clear.

Check Your Understanding

Answer each question using the definitions and assumptions in this tutorial. Short solutions follow the questions.

If retention probability is 0.75 and selection-adjusted power is 0.60, what is conditional power after selection?
With \(c=0.40\), \(\delta_I=1.20\), and a standard normal approximation, calculate the retention probability.
Why is it generally incorrect to multiply retention probability by final marginal power when \(\rho\neq0\)?
Under balanced allocation, \(\theta=0.15\) and \(D=400\), calculate the approximate mean \(\delta_F\) from the log-rank information formula.
Suppose the target is 800 events and the estimated event probability by the fixed horizon is \(q=0.25\). What approximate enrollment gives 800 expected events, and what does that calculation not guarantee?

Solutions. (1) Conditional power is \(0.60/0.75=0.80\). (2) \(P(\text{retained})=\Phi(1.20-0.40)=\Phi(0.80)=0.7881\). (3) The intermediate and final statistics are dependent, so the probability of passing the final rule conditional on passing the interim rule can differ from the final marginal probability; use the joint model. (4) \(\delta_F=0.15\sqrt{400}/2=0.15(20)/2=1.50\). (5) \(N=800/0.25=3{,}200\) participants. This gives 800 expected events under the assumed event probability, not certainty of observing at least 800 by the fixed horizon.