From Clinical Objectives to an Executable Rule
At a seamless Phase 2/3 boundary, a clinical team may need to decide whether to continue a treatment, discontinue it, or select one treatment from several. The clinical objective might be to retain treatments with meaningful benefit while avoiding unacceptable toxicity. That objective is not yet a protocol rule: terms such as “meaningful,” “unacceptable,” and “best” need operational definitions, including what to do at a boundary or when the data do not clearly favor an action.
Prespecifying the rule means documenting, before the interim data are reviewed, how the allowed data summaries map to the allowed actions. The rule should be clinically motivated, computationally reproducible, and complete: two qualified analysts using the same locked data and written rule should reach the same decision. A prespecified rule makes the decision process transparent; it does not by itself establish that the selected treatment is effective or that subsequent statistical claims have valid error rates.
Before setting thresholds, state what the interim quantities mean. For example, define the endpoint and analysis population, the treatment contrast and its direction, the safety outcomes and their observation windows, the data cutoff, and how incomplete or unavailable observations are handled. The definitions should be consistent with the estimand intended across the phase boundary. Otherwise, the rule could compare a Phase 2 summary that answers a different clinical question from the one the later stage is intended to address.
The rule also needs assumptions about the information it receives. The endpoint must be measured as specified, treatment groups must be analyzed under the prespecified assignment and analysis conventions, and safety denominators must be defined. If a rule uses a safety rate, for instance, it must say which participants count in the numerator and denominator and whether events are assessed over a common follow-up window. Thresholds cannot repair a data summary that is not defined consistently.
Specify Criteria, Precedence, and Boundaries
A useful design starts by translating each objective into an observable criterion. A clinical team might specify a minimum treatment effect worth pursuing, a maximum tolerable serious-toxicity rate, and a rule for choosing among treatments that pass both criteria. The values and summaries should be justified clinically and operationally before the interim results are known. A point-estimate threshold, for example, is a decision criterion; it is not automatically a test of a statistical hypothesis.
When several criteria apply, their order matters. A safety override can be evaluated first, followed by efficacy criteria for treatments that meet the safety requirement. If safety is neither clearly acceptable nor clearly unacceptable, the rule can assign an explicitly defined inconclusive action. The protocol should say whether equality meets or fails a threshold, and what happens when two criteria point in different directions.
A compact formal representation helps reveal missing cases. Let \(D\) denote the locked interim data and let \(A\) be the set of allowed actions. A rule is a function \(g(D)\) that assigns one element of \(A\) to each permitted \(D\). If a safety criterion overrides an efficacy criterion, the rule must encode that precedence rather than leave it to interpretation. The basic justification is simple: if the rule checks conditions in a fixed order, and its conditions cover all possible data states without overlap, exactly one action is returned. This is why explicit boundaries and fallback clauses are part of the statistical design rather than administrative decoration.
The equation is a template, not a universal rule. In a particular protocol, “safety acceptable,” “efficacy criterion,” “futility criterion,” and “inconclusive action” must each be defined numerically or operationally. The order also communicates priority: a safety stop takes precedence even if the efficacy summary is favorable.
Worked Example: A Safety Gate and an Efficacy Threshold
Suppose a hypothetical trial compares one treatment with control on a continuous outcome for which larger values are better. Define \(\hat{\Delta}=\bar{x}_{T}-\bar{x}_{C}\), in outcome points, at the interim analysis. The clinical team specifies an efficacy threshold of 5 points, an acceptable serious-toxicity rate of at most 10%, and a safety-stop rate above 15%. Rates between those boundaries are inconclusive for safety. Safety is assessed first. If the rate is at most 10%, continue when \(\hat{\Delta}\geq5\), drop when \(\hat{\Delta}\leq0\), and otherwise take the stated inconclusive action.
In hypothetical interim data, the treatment-group mean is 12.4 points and the control-group mean is 6.8 points. Among 40 treated participants, 3 have the specified serious toxicity. Thus,
The safety rate is no more than 10%, so the safety gate is passed. The estimated difference is at least 5 points, so the rule assigns continue. This conclusion is the action specified by the rule, not a claim that the treatment has demonstrated confirmatory efficacy. The calculation also depends on the protocol’s assumptions that the 40-person denominator and toxicity observation window are the intended ones and that the endpoint summaries use the defined analysis population.
Rules for Selecting Among Treatments
When more than one treatment is considered, “select the best” is not sufficiently precise. The protocol needs to say which treatments qualify for selection, what happens if none qualify, how close estimates are handled, and whether safety or another clinical consideration takes priority over a small efficacy difference. A tie-breaker should be specified in advance, even if an exact numerical tie seems unlikely.
One possible structure is a sequence: exclude treatments that fail a safety criterion; among the remainder, retain those meeting a minimum efficacy criterion; select the largest estimated benefit among those eligible; and apply a prespecified tie-breaker when estimates are sufficiently close. If no treatment qualifies, the rule must still assign an action, such as selecting no treatment for the next stage. Whether that outcome stops the program or triggers another predefined operational action is a separate clinical and protocol decision that must also be stated.
Worked Example: Selecting a Treatment With a Tie-Breaking Rule
Consider two hypothetical treatments, A and B. Define each treatment’s estimated benefit against the same control as a difference in mean outcome points, with larger values favorable. A treatment qualifies only if its estimated benefit is at least 5 points and its serious-toxicity rate is at most 10%. Among qualified treatments, select the one with the larger estimated benefit unless the estimates differ by at most 0.2 points; within that tie range, choose the treatment with the lower serious-toxicity rate. If toxicity rates are also tied, choose the treatment with the lower administration burden, as defined in the protocol.
Suppose the hypothetical summaries are:
| Treatment | Estimated benefit | Serious toxicities | Administration burden |
|---|---|---|---|
| A | 5.8 points | 2/40 = 5.0% | One infusion |
| B | 5.7 points | 1/40 = 2.5% | One infusion |
Both treatments meet the benefit threshold and the toxicity limit. Their benefit estimates differ by \(5.8-5.7=0.1\) points, which is within the prespecified 0.2-point tie range. The tie-breaker therefore applies. Treatment B has the lower toxicity rate, \(2.5\%\) versus \(5.0\%\), so the rule selects B. The larger estimate for A does not take precedence because the protocol explicitly treats differences within 0.2 points as a tie for selection purposes.
The tie range must be defined before examining these results, and comparisons should use the unrounded values specified for the decision. Rounding estimates before checking the tie rule can change which branch applies. The final administration-burden criterion also matters: without it, a tie in both benefit and safety would leave the decision unresolved.
Make Inconclusive Results Actionable
“Inconclusive” describes a decision state, not a complete protocol action. It may arise because an efficacy summary lies between the continuation and futility thresholds, safety lies between acceptable and unacceptable limits, too few outcomes have matured, or data quality prevents a reliable calculation. For each source of uncertainty, the protocol should identify the permitted next step and its limit. For example, it may specify that the team waits for the already planned follow-up to mature, performs a defined data query, or makes no selection if the uncertainty remains at the final cutoff.
Avoid discretionary language such as “review the totality of the evidence” unless the protocol also defines who reviews which information and how that review determines the action. Otherwise, identical results may lead to different decisions, and the stage transition is not reproducible. If an exception is needed for an unforeseen safety concern, state the governance pathway and document that it is an exception rather than silently changing the threshold rule.
Worked Example: An Intermediate Result and a Prespecified Fallback
Use the single-treatment criteria from the first example. Suppose the safety rate is 4/50, and the estimated benefit is 3.7 points. The safety rate is \(4/50=0.08=8.0\%\), which is acceptable. The estimated benefit is greater than 0 but less than 5 points, so it meets neither the continuation nor the futility criterion. The rule therefore classifies the result as inconclusive.
Suppose the protocol specifies the following action for this state: complete the follow-up already scheduled for the interim cohort and recalculate once at the stated final cutoff; do not add participants solely because the first interim result was inconclusive. If the recalculated result still falls between 0 and 5 points, do not select the treatment for the next stage. This specifies both the immediate action and the eventual fallback. It does not permit the team to choose an unplanned cutoff after seeing which one gives the more favorable estimate.
This example assumes that the endpoint and safety data can be updated at the specified cutoff and that the rule’s thresholds apply to the recalculated summaries in the same way. If those assumptions do not hold—for example, if later follow-up changes the endpoint definition—the protocol needs a separately defined rule rather than an informal reinterpretation.
Audit the Rule Before It Is Used
A practical review asks whether the rule is executable without inventing decisions on the day of the transition. Useful diagnostics include checking every equality boundary, testing combinations that conflict (such as favorable efficacy with excessive toxicity), and confirming that missing or immature data have an assigned outcome. A simple decision table or flow diagram can expose gaps before the protocol is finalized.
The choice of approach depends on the decision objective. A threshold-and-gate rule is easy to communicate and can make clinical priorities explicit. A ranked rule selects the highest summary among eligible treatments but needs a tie policy and a no-eligible-treatment action. A structured inconclusive pathway prevents ambiguous cases from defaulting to informal judgment. These are ways to express decision logic, not substitutes for choosing clinically defensible inputs or evaluating the statistical properties required for later confirmatory claims.
The key limitation is that prespecification alone does not make a threshold clinically meaningful, ensure adequate precision, or guarantee valid inference after treatment selection. Thresholds based only on point estimates can be sensitive to sampling variability. Conversely, adding uncertainty summaries or more criteria can make the rule harder to interpret and may require additional statistical justification. Keep the decision rule as simple as the clinical objective permits, define every input and boundary, and distinguish the action it assigns from the evidentiary claims that may later be made.
Check Your Understanding
For each question, apply the stated rule literally. Solutions follow the questions.
- A protocol continues a treatment if its estimated benefit is at least 4 points and its safety rate is at most 8%; it drops the treatment if the safety rate is above 12%; otherwise, results are inconclusive. The estimated benefit is 5.1 points and the safety rate is 10%. What is the action?
- Two eligible treatments have estimated benefits of 6.0 and 5.6 points. The tie range is a difference of at most 0.3 points, with lower toxicity deciding ties. Are they tied under the rule?
- Why should a rule specify the action when no treatment meets the eligibility criteria?
- What does “inconclusive” fail to specify on its own?
Solutions. (1) The safety rate is above 8% but not above 12%, so the safety result is neither acceptable nor a safety-stop result. The action is inconclusive, despite the favorable estimated benefit, because the rule’s safety condition has not been met. (2) The difference is \(6.0-5.6=0.4\) points, which exceeds 0.3 points. They are not tied under this rule, so the treatment with the 6.0-point estimate is selected, assuming both remain eligible. (3) Without a no-eligible-treatment action, the rule leaves a possible data state without a decision; the protocol should state whether no treatment is selected or another predefined action applies. (4) “Inconclusive” names a result category but does not say what happens next, such as whether to await specified data, perform a defined review, or make no selection at the final cutoff.