Why an Adaptation Rule Needs More Than a Threshold
An adaptation rule is a planned link between information arising during a trial and a permitted response. A statement such as “adapt if the response rate is low” is not yet an executable rule: it does not identify which data count, when the decision is made, what “low” means, or what happens if the condition is met. Those omissions leave room for different people to reach different decisions from the same trial record.
A useful way to specify a rule is to identify five parts: the information used, the decision point, the trigger, the permitted action, and the constraints on that action. The parts are related, but they answer different questions. A numerical threshold, for example, belongs to the trigger; it does not by itself define the data to which the threshold applies or authorize a particular response.
The Five Parts of a Rule
Information used specifies the observed data or summary on which the decision depends. It should identify the measure, its source, the relevant observations, and how it is calculated. For a binary outcome, for instance, a proportion is not fully defined until its numerator and denominator are clear. If observations can be missing or unresolved, the rule needs a way to handle them; otherwise, two analysts may calculate different values from the same records.
Decision point identifies when the rule is evaluated and which information is available for that evaluation. This is not the same as the trigger. “When 40 outcome assessments are available” describes a decision point; “if fewer than 20% meet the response definition” describes a trigger. The timing and conduct of interim assessments are treated in more detail in the material on interim information and adaptation timing.
Trigger is the explicitly defined condition that determines whether the rule activates an action. It may use a threshold, a range, or several conditions. The comparison operator matters: “less than 20%” and “20% or less” differ at the boundary. A complete trigger also states how equality is handled and, when several conditions are involved, whether all or any must be satisfied.
Permitted action states what may happen if the trigger is met. The action should be described precisely enough that it is not mistaken for another response. A rule should also state the outcome when the trigger is not met, often that the trial continues without the specified change.
Constraints define the boundaries within which an action can be taken. They can specify which actions are allowed, which are prohibited, who is authorized to implement or approve an action, and what safeguards apply. A constraint can be statistical or operational. For example, a rule may state that a safety response cannot be overridden by a favorable efficacy measure, or that an action does not alter eligibility criteria.
| Part | Question it answers | Example of a precise specification |
|---|---|---|
| Information used | Which data or summary are inputs? | Number meeting the defined response outcome divided by the number with an assessable outcome |
| Decision point | When is the rule evaluated? | At the prespecified review after 40 assessable outcomes |
| Trigger | What condition activates the rule? | The response proportion is less than 0.20 |
| Permitted action | What may follow if it activates? | Pause new enrollment and refer the decision for review |
| Constraints | What limits or governs that action? | No restart without the named independent committee’s authorization |
Writing a Rule as a Decision Procedure
For a simple rule, let \(I\) denote the information available at the decision point, and let \(g(I)\) denote the trigger indicator: \(g(I)=1\) when the trigger is met and \(g(I)=0\) otherwise. Let \(a_1\) be the action permitted when the trigger is met, and \(a_0\) the specified outcome when it is not met. The rule maps the observed information to one of these outcomes, subject to the stated constraints.
This notation is useful because it exposes two requirements for an unambiguous binary rule. First, each permitted value of \(I\) must fall into exactly one trigger outcome. Second, each outcome must have a specified response. For example, a trigger defined as “proportion less than 0.20” and its complement “proportion greater than or equal to 0.20” cover all possible proportions without overlap. If the trigger is instead described as “less than 0.20” and the alternative is left unstated, the rule does not say what to do at 0.20 or when the trigger fails.
The decision procedure is only as reliable as its inputs and implementation. It assumes that the measure and data source are defined, the observations can be classified consistently, the decision point is identifiable, and the person or body applying the rule follows the stated calculation. If those conditions are not met, the apparent precision of a numerical threshold can be misleading.
Worked Example: A Response-Rate Trigger
Suppose a hypothetical trial protocol specifies a review after 40 assessable outcomes. If the proportion meeting a defined response outcome is less than 0.20, the permitted action is to pause new enrollment and refer the situation for review. Otherwise, enrollment continues without that action. For this illustration, assume that all 40 outcomes have been classified using the same response definition and that the rule uses these 40 outcomes as its denominator.
The hypothetical data show 7 participants meeting the response definition out of 40 with assessable outcomes. The information used is therefore \(7\) responses and \(40\) assessable outcomes. The response proportion is:
The trigger is \(\hat{p}<0.20\). Substituting the calculated value gives \(0.175<0.20\), which is true. Thus the rule’s trigger is met, and the specified action is to pause new enrollment and refer the situation for review. This calculation does not establish why the response proportion is low; it only shows how the prespecified rule maps these data to an action.
The assumptions are important to the result. If some outcomes were unresolved and the protocol did not say whether to exclude them or count them as nonresponses, the denominator or numerator could differ. Likewise, if the response definition changed during the calculation, the 7/40 result would not be a consistent application of the rule.
Boundary Cases and Rule Completeness
Thresholds should be written with their boundary cases in mind. Suppose the trigger is “at least 8%.” A calculated value of exactly 8% meets that trigger; a rule written as “greater than 8%” would not activate at equality. Using an explicit inequality avoids an interpretive choice at the moment a decision is required.
The rule should also specify how its inputs are constructed. A rate requires a defined numerator and denominator; a mean requires a defined set of observations; and any composite information requires a clear calculation. Where the information includes missing, delayed, or disputed records, the rule needs a prespecified handling approach. Otherwise, the trigger may be reproducible in principle but not in practice.
Worked Example: A Data-Quality Trigger at the Boundary
Consider a hypothetical operational rule evaluated at a specified review point. It uses the proportion of audited records with an unresolved data query. If that proportion is at least 0.08, the permitted action is standardized retraining for the relevant trial sites. Assume that 10 of 120 audited records have an unresolved query, and that each record is counted once under the protocol’s query definition.
The information used is the unresolved-query count and the audited-record count. The calculation is:
The trigger condition is “at least 0.08,” or \(\text{proportion}\geq 0.08\). Since \(0.0833\geq 0.08\), the trigger is met. The rule therefore permits standardized retraining for the relevant sites.
The constraints make this action narrower and more consistent: retraining uses the same materials and process at every site meeting the rule’s definition, and it does not change participant eligibility or treatment assignment. If the proportion had been exactly 0.08, the trigger would still be met because the rule says “at least.” The calculation also depends on the denominator being audited records, not the number of queries; changing that denominator would define a different rule.
Constraints, Priority, and Safety
When a trial has more than one rule, their relationship needs to be clear. Two triggers may both be met, one may take priority over another, or one may restrict the action available under a second rule. A specification that lists several triggers without stating how simultaneous triggers are handled can produce competing instructions. A priority clause—such as a safety rule taking precedence over a non-safety rule—can resolve this, provided it is explicit and does not leave an action undefined.
Constraints should be operational rather than aspirational. “The action will be appropriate” does not identify who decides what is appropriate. A more usable constraint states who may authorize the action, what evidence or review is required, and what cannot be changed. In a hypothetical safety rule, for example, a clearly defined event rate might trigger a pause in enrollment, with restart allowed only after authorization by a named independent safety committee. The constraint preserves a defined safeguard; it does not guarantee that the event rate itself is measured without error.
Worked Example: A Safety Rule with an Explicit Equality Case
Suppose a hypothetical rule uses the proportion of participants with a prespecified serious safety event at a review point. If the proportion is at least 0.08, new enrollment is suspended and an independent safety committee reviews the situation. Restart is not permitted without that committee’s authorization. Assume the rule’s data consist of 2 participants with the event among 25 assessed participants, with event status defined consistently for all 25.
The event proportion is:
The trigger is \(\text{event proportion}\geq 0.08\). Here the value equals the threshold, so \(0.08\geq0.08\) and the trigger is met. The specified action is suspension of new enrollment and committee review. The constraint is that restarting requires committee authorization. It would be incorrect to report that the rate exceeds 8%; it equals 8%, and equality is sufficient under this rule.
The calculation assumes the event definition, assessment set, and denominator are fixed for this decision. If event status is unknown for some participants, the rule must say how those participants are handled before the proportion can be applied consistently.
Diagnostics, Alternatives, and Limitations
Before applying a rule, check that a reader could independently answer all five questions: What information is used? At what decision point? What exact condition is the trigger? What action follows each outcome? What constraints govern that action? A missing answer is a diagnostic warning that the rule may rely on discretionary interpretation.
- Unclear input: The rule names a rate but not its numerator, denominator, or handling of unresolved records.
- Unclear boundary: The rule says “high” or “below target” without a numerical condition, or leaves equality unspecified.
- Unclear response: The trigger is defined, but the rule does not state what happens when it is met or not met.
- Unclear authority: The action is listed, but no responsible decision-maker or authorization constraint is identified.
- Conflicting instructions: Multiple triggers can be met together, but their priority or combined action is not specified.
A prespecified rule is an alternative to ad hoc discretion: it makes the input-to-action link available in advance and helps ensure that the same condition is handled consistently. A trial with no adaptation rule instead follows its planned design without this decision procedure. Neither alternative is automatically superior in every respect. A rule can limit subjective, post hoc choices, but only if its information and action are well defined and its constraints can be followed.
Most importantly, a well-written rule is not, by itself, proof that the overall trial has acceptable statistical operating characteristics. The trigger may respond to noisy or biased information, and the action can affect interpretation of the trial’s results. The rule must be considered as part of the complete design, with appropriate evaluation of the consequences of its use. Clear wording supports that evaluation; it does not replace it.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A rule says “review when the event rate is high.” Which parts of the rule are still unclear?
- A rule triggers when a proportion is at least 0.10. The observed count is 5 out of 50. Does the trigger activate? Show the calculation.
- Why is “if the trigger is not met, continue as planned” an important part of a complete rule?
- A protocol has two triggers that may both be met, but does not say which action takes priority. What is the problem, and how could it be corrected?
Solutions. (1) “Event rate” needs a defined event, numerator, denominator, data source, and handling of unresolved observations. “High” needs an explicit threshold and boundary operator. The decision point, permitted action, and constraints also need to be stated. (2) The proportion is \(5/50=0.10\). Since \(0.10\geq0.10\), the trigger activates. (3) Without a specified outcome for a false trigger, the rule leaves the response to discretion and does not define the decision procedure’s output for every possible result. (4) The rules give potentially conflicting instructions for the same information. The protocol should specify which trigger has priority or define an explicit combined action, along with any constraints on that action.