Tutorials › Adaptive Clinical Trial Design › FOCUS4 Arm Closure After Interim Futility

Case Studies · Tutorial 775 of 799

FOCUS4 Arm Closure After Interim Futility

Learn how a prespecified futility assessment can stop one FOCUS4 comparison while leaving other eligible molecular groups and platform activity governed by their own rules.

Advanced 13 min read

What You'll Learn

  • Distinguish closure of one randomized comparison from closure of an entire platform trial.
  • Interpret a futility rule as a prospective decision criterion, not proof that a treatment has no effect.
  • Calculate conditional power using interim information and a prespecified target effect.
  • Identify assumptions and diagnostics relevant to interim futility decisions.
  • Compare conditional-power stopping with predictive-probability and unadjusted interim-p-value approaches.
  • Explain how governance and operational safeguards protect ongoing molecularly stratified recruitment.

Why an Interim Futility Decision Can Be Local

In a platform trial, treatment comparisons can be organized around distinct patient groups, eligibility rules, and questions. An interim decision that one comparison is unlikely to answer its question need not mean that the wider platform has failed. FOCUS4 illustrates this distinction: the platform used molecular stratification in metastatic colorectal cancer, and a treatment comparison was closed after an interim futility assessment while other platform activity could continue under its own eligibility and decision rules.

The introductory material on FOCUS4 molecularly stratified recruitment establishes why molecular groups matter: a patient’s molecular profile determines which comparison may be relevant. This tutorial focuses on what happens when evidence within one comparison triggers a futility decision. The statistical decision is about the remaining prospects of that comparison under its protocol; it is not a conclusion that every treatment strategy, molecular group, or future platform question is futile.

Definition: An interim futility rule is a prespecified criterion for stopping a treatment comparison early when the accumulating evidence indicates that continuing is unlikely to meet a stated goal, such as demonstrating a clinically relevant treatment effect at the planned final analysis. Arm closure means that new participants are no longer randomized to the closed comparison according to the protocol. It does not, by itself, mean that ongoing follow-up stops, that all platform recruitment stops, or that the treatment has been proved ineffective.

These distinctions are important in FOCUS4. The trial’s molecularly stratified structure allows one comparison to be evaluated within its eligible group. A futility decision for that comparison is therefore a local adaptation: it changes the availability of that treatment comparison, while recruitment to other eligible comparisons may continue if their protocols and governance permit. The actual decision must follow the trial’s prespecified criteria and oversight process; it cannot be reconstructed from a headline such as “closed for futility” alone.

What Futility Means Statistically

Consider a comparison whose primary endpoint is progression-free survival, summarized by a hazard ratio. Let \(\theta=\log(HR)\), where \(HR\) is the treatment hazard divided by the control hazard. Under this convention, \(\theta=0\) corresponds to \(HR=1\), and a treatment benefit corresponds to \(\theta<0\). A trial might plan to test \(H_0:\theta\geq 0\) against \(H_a:\theta<0\), using a prespecified one-sided significance level \(\alpha\). A futility assessment asks a different question from the final efficacy test: given what has been observed so far, how likely is the planned analysis to meet its success criterion if the comparison continues?

One common measure is conditional power. It is the probability of meeting the final success criterion, conditional on the interim data, under a specified assumption about the treatment effect that will hold during the remainder of the trial. The assumption matters. Conditional power can be calculated under a target effect considered clinically worthwhile, under the effect estimated at interim, or under another protocol-specified value. These are not interchangeable calculations.

Definition: In a one-sided test favoring \(\theta<0\), conditional power is the probability, given the interim evidence, that the final test statistic will be below its prespecified critical value, under a stated assumed value of \(\theta\) for future information. A futility boundary might, for example, recommend stopping if this probability is below a protocol-defined threshold. The threshold and the assumed effect must be specified in advance; there is no universal cutoff that applies to every trial.

A Conditional-Power Calculation

A simple normal-approximation calculation makes the logic explicit. Suppose the interim and future score contributions are independent, with information \(I_I\) observed at interim and \(I_F\) expected afterward. Let \(U_I\) be the observed interim score, and suppose future information is planned so that \(I=I_I+I_F\). Under an assumed effect \(\theta^*\), the future score \(U_F\) has approximate distribution \(N(\theta^* I_F,I_F)\). The final statistic is \(Z_{\mathrm{final}}=(U_I+U_F)/\sqrt{I}\). For a one-sided test at level \(\alpha\), success requires \(Z_{\mathrm{final}}<-z_{1-\alpha}\), where \(z_{1-\alpha}\) is the corresponding standard-normal quantile.

Conditional on the interim data, \(U_I\) is fixed. Standardizing the random future score gives the calculation below. This is a model-based approximation: it depends on the assumed effect, information plan, and independent-increments normal model.

$$ \begin{aligned} P(Z_{\mathrm{final}}<-z_{1-\alpha}\mid U_I,\theta^*) &=P\left(U_F<-z_{1-\alpha}\sqrt{I}-U_I\mid\theta^*\right)\\ &=\Phi\left(\frac{-z_{1-\alpha}\sqrt{I}-U_I-\theta^* I_F}{\sqrt{I_F}}\right). \end{aligned} $$

Here \(\Phi\) is the standard normal cumulative distribution function. The numerator compares the score needed for final success with the interim score already observed and the mean future score expected under \(\theta^*\). If the interim score is unfavorable, or the assumed future effect is small, the resulting conditional power can be low. A low value does not establish that the true effect is exactly zero; it says that success under the stated assumptions has become unlikely.

Worked Example: Conditional Power Under a Target Effect

Hypothetical calculation. Suppose a comparison uses a one-sided final test with \(\alpha=0.05\), and the interim analysis has observed \(I_I=40\) units of information with \(Z_I=2.00\). Another \(I_F=60\) units are planned. Assume a clinically worthwhile target hazard ratio of \(0.75\), so \(\theta^*=\log(0.75)=-0.2877\). For this illustration, a prespecified futility recommendation is made if conditional power is below \(0.20\).

The total information is \(I=40+60=100\). Since \(Z_I=U_I/\sqrt{I_I}\), the observed interim score is \(U_I=2.00\sqrt{40}=12.65\). Also, \(z_{0.95}=1.645\). Substituting into the conditional-power formula gives:

$$ \begin{aligned} CP &=\Phi\left(\frac{-1.645\sqrt{100}-12.65-(-0.2877)(60)}{\sqrt{60}}\right)\\ &=\Phi\left(\frac{-16.45-12.65+17.26}{7.746}\right)\\ &=\Phi(-1.529)\approx 0.063. \end{aligned} $$

Under the target effect and this information plan, the estimated conditional probability of meeting the final criterion is about \(6.3\%\), below the illustrative \(20\%\) boundary. The rule would therefore recommend stopping for futility. The conclusion is conditional on the assumed target effect and model; it is not a statement that the treatment has a \(6.3\%\) chance of working in all possible circumstances, or that its true effect is known.

What Closing a FOCUS4 Comparison Does—and Does Not—Do

Applied to FOCUS4, the central governance question is how to make a decision about the comparison that generated the interim evidence without confusing it with the platform as a whole. Once a comparison is closed to new randomization, the protocol and operational plan must identify which patients remain eligible for other comparisons and how those comparisons continue. Molecular screening or other platform processes may remain active, but eligibility for a different comparison is not automatic: it depends on that comparison’s criteria and on the participant’s clinical situation.

Closure also does not erase data already collected. Participants already randomized may need continued outcome follow-up, safety monitoring, and analysis according to the protocol, even when no additional participants are assigned to the comparison. The DMC and trial leadership must communicate the recommendation and implement it through the agreed governance process. Sites need a clear effective date and instructions for screening, consent, randomization, and participants who are between steps. The aim is a controlled change, not an informal shift in treatment allocation.

The distinction matters for interpretation as well. A futility recommendation addresses the expected value of continuing to gather data under a specific design and decision target. It does not prove equivalence, establish absence of benefit for every patient subgroup, or answer a treatment question that the comparison was not designed to address. In a molecularly stratified platform, conclusions are especially bounded by the eligible molecular group, treatment comparison, endpoint, and analysis plan.

Worked Example: Keeping a Platform Open After One Comparison Closes

Hypothetical operational scenario. Imagine a platform has three active comparisons, each with its own eligible molecular group. An interim rule recommends closing comparison A for futility. At that point, 18 participants have been randomized to A, 42 remain in the platform’s screening pathway, and the protocol permits comparisons B and C to continue if participants meet their separate eligibility criteria.

The immediate change is that new randomizations to A stop on the authorized date. The 18 already randomized to A remain accounted for in its follow-up and analysis plan. The 42 people in screening are not automatically assigned to B or C: the team checks their molecular results, clinical eligibility, consent status, and the current status of those comparisons. Suppose 15 meet B’s eligibility criteria, 10 meet C’s, and 17 meet neither. The totals reconcile: \(15+10+17=42\). Only the 25 who are eligible for B or C can be considered for those comparisons, and only under their relevant protocol rules.

This example shows why “one comparison closed” and “platform closed” are different operational statements. It also shows why eligibility cannot be inferred merely from the availability of another comparison. The closure changes one assignment option; it does not remove the need for molecular classification, informed consent, or protocol-based eligibility checks.

Assumptions, Diagnostics, and Alternatives

A futility calculation is only as dependable as its inputs. Before acting on conditional power, reviewers should check whether interim data are sufficiently complete and mature, whether endpoint definitions and censoring are being applied consistently, and whether the planned information at the final analysis remains realistic. Delayed or missing outcomes, unexpectedly slow recruitment, or changes in treatment exposure can make a calculation based on the original information plan misleading. The assumed target effect must also be clinically and scientifically defensible rather than chosen after seeing the interim result.

For a hazard-ratio analysis, the normal approximation and information-based calculation should be checked against the actual analysis plan. Non-proportional hazards, sparse events, changes in eligibility or treatment delivery, and major deviations from planned follow-up can undermine a simple model. These are reasons to investigate and report limitations, not automatic reasons to ignore a prespecified rule. The DMC should review the quality and context of the evidence before recommending action.

Conditional power is not the only possible approach. A predictive-probability approach averages over uncertainty about the effect, often using a prespecified probability model for that effect. It may represent uncertainty more directly than conditional power calculated at one fixed target, but its result depends on the probability model and prior assumptions. A simple interim \(p\)-value, by contrast, measures compatibility of observed data with a null model; it does not by itself estimate the chance of final success. Stopping merely because an interim \(p\)-value is not significant can discard a promising comparison when much information remains.

ApproachWhat it summarizesKey caution
Conditional powerChance of final success given interim data under a specified future effectCan change substantially with the assumed effect and future information
Predictive probabilityChance of final success averaged over uncertainty in the effectDepends on the probability model and prior assumptions
Interim \(p\)-value aloneCompatibility of interim data with a null hypothesisDoes not directly quantify the chance of eventual success

The approaches should not be mixed informally. The protocol should state which quantity informs the decision, what assumptions are used, what threshold applies, and who reviews the result. An alternative calculation can be a useful sensitivity analysis, but it does not replace the prespecified rule or the committee’s responsibility to consider data quality and clinical context.

Worked Example: Why a Non-Significant Interim Result Is Not Enough

Hypothetical calculation. At an interim review, a treatment comparison has a one-sided \(p\)-value of \(0.28\), so it has not crossed a conventional \(0.05\) significance threshold. Suppose the protocol’s conditional-power calculation, under its target effect and planned information, is \(0.34\), while its futility threshold is \(0.10\).

The interim test result is not statistically significant, but \(0.34>0.10\), so the stated futility rule is not met. The comparison should not be closed solely because the interim \(p\)-value exceeds \(0.05\). The trial team should follow the protocol’s decision rule and consider the quality and maturity of the interim data. Conversely, if conditional power falls below the specified boundary, that recommendation is based on the prospective futility rule—not on relabeling a non-significant interim test as proof of no treatment effect.

Common Mistake and Key Takeaway

  • Wrong reasoning: “FOCUS4 closed one comparison for futility, so the platform had shown that the treatment was ineffective and should stop recruiting altogether.”
  • Correction: A futility decision is a protocol-defined judgment that continuing one comparison is unlikely to meet its stated objective under specified assumptions. The comparison may stop recruiting while other eligible comparisons continue under their own rules. The decision does not prove a zero effect or settle questions outside that comparison.

For a platform such as FOCUS4, the statistical and operational decisions must align: use the prespecified interim rule, establish that its inputs are reliable, close only the comparison the decision covers, and preserve the eligibility and governance safeguards for ongoing platform activity. This is how an interim futility decision can conserve resources without treating one comparison’s evidence as a verdict on the entire platform.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

  1. In a one-sided test favoring a hazard ratio below 1, what does a low conditional power under the target effect mean? What does it not prove?
  2. A comparison closes to new randomization after a futility decision. Does that imply that participants already randomized stop follow-up? Explain.
  3. Why is an interim \(p\)-value above \(0.05\) not, by itself, a sufficient futility rule?
  4. A platform has four comparisons. One is closed under its prespecified futility rule, and the other three remain active. What should determine whether a person previously being screened can enter one of the remaining comparisons?

Solutions. (1) It means the model-based probability of meeting the final success criterion is low, conditional on the interim data and the specified future effect and information. It does not prove the true effect is zero or that the treatment cannot benefit anyone. (2) No. Follow-up for participants already randomized is governed by the protocol and may continue for outcome and safety assessment even though new randomization has stopped. (3) A non-significant interim result is not the same as a low probability of final success; substantial information may remain, and the \(p\)-value alone does not incorporate the planned future data or a target effect. (4) The person must meet the remaining comparison’s molecular and clinical eligibility criteria and applicable consent and protocol requirements. Closure of one comparison does not automatically transfer a person to another.

Clinical Biostats