From Protocol Language to a Decision Algorithm
A protocol may say that an interim review occurs “when sufficient information is available,” that treatment “may be adapted,” or that the trial stops “if safety is concerning.” Those phrases may communicate intent, but they do not determine a unique action for every possible trial history. An executable algorithm does: given the trial’s current information, it specifies which conditions are checked, in what order, and what happens next.
The task here is to formalize decisions, not to choose the scientific design or to assess its operating characteristics. The algorithm must preserve the protocol’s intended rules while making their implementation reproducible. In particular, it must define the data available at each decision, the exact trigger for each review, how competing rules are prioritized, and what analysis is performed at the end.
For the algorithm to be executable, each condition needs an observable definition. “At the interim” needs a trigger; “eligible” needs criteria and a time at which they are assessed; and “stop for safety” needs a threshold and a priority relative to other stopping rules. A simulation can only implement the rule that has been specified. If the protocol leaves a decision to discretion, a simulation requires an additional rule representing that discretion, and the added rule should be documented rather than silently improvised.
Define the Inputs and Decision Order
Before writing branches, identify the minimum state information the decision-maker is allowed to use. For example, a rule might use only outcomes that have completed a specified follow-up window. It should not accidentally use outcomes that would not yet be available at the review date. Similarly, the algorithm should distinguish a participant who has not yet completed follow-up from one whose outcome is missing after the follow-up window. The protocol must state how each case affects the decision.
A useful specification states, for every rule, (1) its trigger, (2) its data inputs, (3) its condition, (4) the action if the condition is met, and (5) what happens if it is not met. When more than one condition may hold, an explicit priority order is essential. The following sequence is a general way to organize the rules; the protocol’s actual clinical decisions determine the content of each step.
Apply the prespecified inclusion and exclusion criteria at the stated assessment time. Record whether the person may enter, and how an ineligible person is handled.
Evaluate the interim trigger using the stated count, calendar time, or outcome-maturity condition. If it is not met, continue the current procedure.
Evaluate safety, futility, efficacy, or adaptation rules in the order specified. Stop at the first applicable action if the rules are designed to be mutually exclusive by priority.
Apply the maximum-enrollment, maximum-duration, or other terminal condition. State how it interacts with any interim decision due at the same time.
When the trial ends, state which data are included, how outcomes not yet observed are handled, and which prespecified analysis is reported.
This ordering is not a universal clinical priority. It is a prompt to make the order explicit. For example, a safety rule may take precedence over every other decision, but that must be specified. If “stop for safety” and “stop for futility” are both satisfied at one review, an algorithm needs to return a single action and, if relevant, a single recorded stop reason.
Why Explicit Branches Produce One Decision
The core justification is a simple property of ordered decision rules. At a decision point, let \(C_1,\ldots,C_k\) be the conditions checked in order. The algorithm takes the action associated with the first condition that is true. If none is true, it takes a stated default action. Thus, even when several conditions are simultaneously true, the order selects exactly one action. The default is necessary because otherwise some possible histories have no defined next step.
This construction makes the decision single-valued: the same state and the same available data yield the same action, provided any random component (such as a treatment-allocation draw) is also specified. It does not guarantee that the rules are clinically appropriate. It guarantees only that their implications are defined. The protocol team must still justify the thresholds, endpoints, and priorities on scientific and ethical grounds.
A practical check is to list all plausible combinations of triggers, including simultaneous triggers, and verify that each has one outcome. If conditions are intended to be mutually exclusive, check that their definitions really prevent overlap. If they are not exclusive, provide a priority. Also specify equality: a rule using “greater than” differs from one using “greater than or equal to” at the threshold.
Worked Examples
Worked Example: Defining an Interim Trigger With Incomplete Follow-Up
Hypothetical protocol rule. The first interim review is scheduled once 12 participants have been randomized. A participant is evaluable for the review only after completing 28 days of follow-up. The review uses outcomes from all 12 randomized participants. Enrollment is paused at 12 until those outcomes are available. On day 28 after the twelfth participant’s randomization, 9 participants have completed follow-up; 3 have not.
The randomized-count trigger has been reached, because the count is 12. But the data condition for conducting the review has not been reached: only 9 of the 12 participants have completed the required 28-day window. Under the stated rule, the review does not occur yet. Enrollment remains paused, and the algorithm waits for the three pending follow-up windows to finish.
When all 12 have completed the window, the review data set contains 12 outcomes. The trigger can be written as \(N_{\mathrm{rand}}\geq 12\) and \(N_{\mathrm{complete}}\geq 12\), where \(N_{\mathrm{rand}}\) is the number randomized and \(N_{\mathrm{complete}}\) is the number with the required follow-up complete. At the stated review, \(N_{\mathrm{rand}}=12\) but \(N_{\mathrm{complete}}=9\), so the second condition fails. This rule avoids treating “12 randomized” as if it meant “12 outcomes observed.”
The protocol would still need to specify what happens if an outcome remains unavailable indefinitely or is missing after the follow-up window. The algorithm cannot infer whether that participant is excluded, counted as a failure, or handled by another prespecified method.
Worked Example: Applying a Dose-Adaptation Rule
Hypothetical protocol rule. At an interim review, consider three doses. A dose is eligible for selection only if the posterior probability that its toxicity rate exceeds 0.30 is less than 0.20 and the posterior probability of efficacy is at least 0.50. Among eligible doses, select the one with the largest posterior probability of efficacy. If none is eligible, stop for lack of an acceptable dose. At this review, the model has produced the following probabilities:
| Dose | Posterior probability toxicity rate exceeds 0.30 | Posterior probability of efficacy |
|---|---|---|
| 1 | 0.08 | 0.35 |
| 2 | 0.16 | 0.62 |
| 3 | 0.24 | 0.71 |
Dose 1 meets the toxicity requirement because \(0.08<0.20\), but it fails the efficacy requirement because \(0.35<0.50\). Dose 2 meets both requirements: \(0.16<0.20\) and \(0.62\geq 0.50\). Dose 3 fails the toxicity requirement because \(0.24\not<0.20\), although its efficacy probability is the largest.
Therefore dose 2 is the only eligible dose and is selected. The rule does not select dose 3 merely because it has the largest efficacy probability; the toxicity condition is a gate applied first. The comparison must also say what to do in a tie. This hypothetical rule has no tie-break because no tie occurs in the supplied values; a complete protocol should add one, such as selecting the lower dose among tied eligible doses.
Worked Example: Resolving Simultaneous Stopping Triggers
Hypothetical protocol rule. At each scheduled review, stop first if at least 5 of the first 24 participants experience the defined dose-limiting toxicity. If that safety condition is not met, stop for futility if the prespecified posterior probability of eventual success is below 0.10. Otherwise continue unless the maximum sample size of 40 has been reached. At the review, 5 of 24 participants have dose-limiting toxicity, the futility probability is 0.06, and 40 participants have been randomized.
All three terminal conditions appear relevant: the safety threshold is met, the futility probability satisfies \(0.06<0.10\), and the maximum sample size has been reached. The stated priority resolves the overlap. The algorithm checks safety first, observes \(5\geq5\), and stops for safety. It does not proceed to assign futility as the stop reason or to continue checking for an enrollment action.
The final-analysis instruction must then be applied. Suppose the hypothetical protocol specifies that the primary analysis includes all 40 randomized participants according to their randomized groups, using the defined endpoint data available under the final data-lock rules. The algorithm records the safety stop and proceeds to that analysis; it does not discard participants enrolled before the stop. How outcomes that are still incomplete at data lock are handled must be specified separately in the protocol rather than guessed by the implementer.
Diagnostics, Failure Modes, and Alternatives
The most useful diagnostics for an algorithm are checks of its logic, not just checks that its software runs. Review the rule table or pseudocode against edge cases: no participant eligible; an interim count reached while outcomes are pending; two stopping criteria met together; a value exactly equal to a threshold; a tie between candidate treatments; and the maximum sample size reached on the same occasion as an interim review. For each case, confirm that exactly one next action is defined.
- Unreachable branch: A condition can never be met because another earlier condition always captures those states. Test whether each intended branch can occur.
- Uncovered state: A rule specifies actions for “yes” but not “no,” or omits missing and delayed outcomes. Add an explicit alternative or default.
- Order-dependent result: Two conditions can both be true, but no priority is stated. Specify which action wins and which reason is recorded.
- Hidden discretion: Terms such as “clinically meaningful,” “adequate information,” or “acceptable toxicity” are used without operational criteria. Define them or state who decides and how that decision is represented.
- Nonterminating loop: A trial can remain paused while waiting for an outcome or repeatedly reach a review without a next action. Define a maximum wait, a missing-data rule, or another termination procedure where appropriate.
The main alternative to a fully ordered algorithm is a policy that delegates some decisions to a committee or investigator. Such a policy can be appropriate in a real trial, especially when safety information requires judgment, but it is not a fully deterministic rule. For simulation-based design assurance, the delegated decision must be represented by an explicit decision model or a set of possible policies; otherwise the simulated procedure cannot reproduce the protocol decision. Conversely, an entirely deterministic algorithm can omit valuable clinical judgment. The choice is substantive, not merely a programming preference.
A second distinction is between a flowchart and a rule table or pseudocode. A flowchart can make the decision path easy to inspect, while a table can precisely record thresholds, data inputs, and actions. Pseudocode can be closest to an implementation. These are complementary representations: whichever is used, it should encode the same ordering, boundary conventions, and default actions.
Why This Specification Matters
Ambiguity is not a cosmetic problem. Two competent implementers can translate the same vague sentence into different rules, producing different allocations, stopping times, or final analyses. Then any comparison of design performance is partly a comparison of undocumented interpretations. A precise algorithm makes those choices visible and reviewable before they are implemented.
The specification should also make clear which decisions are adaptive and which remain fixed. For example, the dose-selection rule may change assignments at an interim, while the endpoint definition, analysis population, and tie-break rule remain prespecified. Recording that distinction helps prevent a simulated procedure from acquiring unplanned flexibility. The final analysis is part of the algorithm, not an afterthought: state the analysis population, the data cutoff, and the method for handling incomplete outcomes.
Check Your Understanding
For each question, identify the rule needed to make the decision unambiguous. Solutions follow the questions.
- An interim is due after 20 participants are randomized, but only 18 have completed the outcome window. What must the algorithm specify before deciding whether to review?
- A safety threshold and a futility threshold are both met at the same review. What feature of the algorithm determines the action?
- A treatment-selection rule chooses the eligible treatment with the highest efficacy probability. What additional rule is needed if two eligible treatments tie?
- Why should the algorithm specify what happens when no treatment meets the eligibility criteria?
- At the maximum sample size, some outcomes are not yet available. Name two final-analysis details that must be specified.
Solutions. (1) It must state whether the trigger requires only 20 randomized participants or also requires a specified number, or all 20, to complete the outcome window; it must also define what happens while waiting. (2) The explicit priority order determines which action is taken and which stop reason is recorded. (3) A tie-break, such as choosing the lower dose, must be prespecified. (4) Without a no-eligible-treatment action, the algorithm has no defined next step for that possible state; it might, for example, stop or continue under a stated default. (5) The protocol must define the final analysis population and how incomplete or missing outcomes at the data cutoff are handled.