Why Decision Data Need Precise Boundaries
An adaptation rule can state a clear numerical threshold and still leave the decision uncertain. The uncertainty may be about which participants count, which measurements are included, whether data must be verified or adjudicated, or what “available at the interim” means. If two teams apply the same threshold to different data snapshots, they can reach different decisions without either team miscalculating.
Documenting decision criteria and interim data cutoffs means specifying both what is calculated and which version of the data is used to calculate it. The aim is not to prescribe one universally correct cutoff or data-cleaning workflow. Those choices depend on the design and endpoint. The aim is to make the intended choice sufficiently definite that another qualified reader can identify the same inputs and apply the stated rule.
The earlier material on prospective adaptation rules establishes why an adaptation should be specified before the interim data are examined. Here, the focus is narrower: making the input to a prespecified decision rule operationally unambiguous. Documentation of inputs does not, by itself, establish that the rule has acceptable statistical properties or that a particular adaptation is scientifically justified.
What a Decision-Criteria Specification Must Identify
A useful specification connects the decision rule to a particular, well-defined dataset. At minimum, describe the following elements wherever they are relevant:
- Decision and rule. Identify which adaptation is being decided and state the statistic or condition used, its threshold, and whether equality meets the criterion. If there are multiple conditions, state whether all or any must be satisfied.
- Population. Define which participants are eligible for the calculation. Examples include all randomized participants, participants assigned to a specified group, or participants meeting stated endpoint-evaluability criteria. Do not rely on a population label alone if the label could be interpreted in more than one way.
- Variables and derivations. Name the endpoint, its units and assessment time, and the variables or records used to derive it. Specify relevant rules for visit windows, repeated measurements, and derived values.
- Data status. State which records count as usable: for example, whether required edit checks must be complete, whether queries may remain open, whether treatment assignments are unmasked, and whether endpoint adjudication must be final. Include a defined approach for missing or invalid values when that affects the decision quantity.
- Cutoff and timing. Give the date and time, time zone, and the event that determines eligibility when using a calendar cutoff. For an event-driven cutoff, define the event and how it is verified. Clarify whether the cutoff concerns the date of the clinical event, data entry, verification, or another milestone.
The wording should also make the denominator and unit of analysis clear. A rate might be calculated among participants with a known outcome, among all randomized participants, or among participants who completed a specified visit. These are different quantities. If the rule uses a count, state what is being counted: participants, endpoint events, completed assessments, or another unit.
Worked Specification Examples
Worked Example: Defining a Response-Rate Decision
Hypothetical setting. At an interim review, a trial team will decide whether to continue one treatment group. The prespecified rule is to continue if at least 60% of the evaluable participants in that group have a response. The draft says only, “Use available response data at interim.”
That draft leaves the denominator, response definition, evaluability, and cutoff unclear. Suppose 50 participants have been randomized to the group. By the stated interim snapshot, 42 have a response assessment that satisfies the endpoint rules; 23 meet the response definition. Five more records arrive after the proposed snapshot, of which four are responses. A reader cannot know from “available” whether these five records belong in the calculation.
Under an explicit rule defining the denominator as the 42 evaluable participants, the calculation is:
The 60% criterion is not met, since \(54.8\%<60\%\). If the five later records were included instead, the calculation would be \(27/47=0.5745\), or 57.4%, still below the threshold in this hypothetical example. The decision happens to be unchanged, but the datasets and estimates differ. A specification must not depend on whether an ambiguity happens to change the decision in one instance.
An unambiguous version would state the response definition, assessment window, evaluability rule, denominator, cutoff date and time, time zone, and data-status requirements. For example: “Calculate the proportion of evaluable participants assigned to Group A whose Week 8 response is confirmed by the endpoint rules. Include participants with a valid Week 8 assessment in the prespecified visit window and a response status recorded and verified in the database by 17:00 UTC on 30 June 2026. The denominator is all such evaluable participants in Group A; participants without a valid assessment are excluded from this proportion. Continue Group A if and only if the calculated proportion is at least 60%.”
Worked Example: Separating Event Time From Data Availability
Hypothetical setting. An adaptation decision uses the number of confirmed primary endpoint events. The rule requires at least 40 events. The specification sets a cutoff of 23:59 UTC on 31 March 2026, but does not say whether the event date or database-entry date controls.
Assume the event records show 38 confirmed events entered by the cutoff. Two additional events occurred on 29 March, but their records were entered on 2 April. Another event occurred on 1 April and was entered on 1 April. If the rule is based on events occurring by 31 March and subsequently verified, there may be 40 eligible events. If it is based on records verified in the database by the cutoff, there are only 38. The first interpretation concerns clinical event time; the second concerns information available in the data system at a fixed time.
A usable specification must select and define the intended interpretation. Suppose the intended input is the database snapshot available at the cutoff. Then the calculation is \(38\) confirmed events, and the 40-event condition is not met. The two late-entered records and the 1 April event are excluded from that snapshot. If instead the intended condition is that 40 events have occurred, the specification must say how event occurrence is established and what verification is required before the decision can be made. It should not leave those options implicit.
One precise version for the database-snapshot interpretation is: “Use the primary endpoint event records confirmed and present in the locked interim extract generated at 23:59 UTC on 31 March 2026. Count no more than one primary endpoint event per participant, using the event definition in the endpoint specification. The event-count condition is met when the extract contains at least 40 such events.” The word “locked” must itself refer to a defined process or artifact in the study documentation; it should not be used as a substitute for explaining which records are included.
Worked Example: Stating Data Status for a Continuous Endpoint
Hypothetical setting. A rule compares the mean change from baseline in a treatment group with a threshold of \(-5\) points on a symptom scale; more negative values indicate improvement. The draft says, “Use the cleaned Week 12 data.” This does not explain whether unresolved queries, out-of-window assessments, or values awaiting central review count as cleaned data.
Suppose a specified snapshot contains 30 randomized participants. Under the endpoint rules, 24 have a valid Week 12 assessment in the allowed visit window, and their changes sum to \(-132\) points. The mean among these 24 participants is:
If the decision criterion is “the mean change is at most \(-5\) points,” then this dataset meets the criterion because \(-5.50\leq -5\). This calculation is interpretable only if the specification makes clear that the 24 valid assessments, rather than all 30 randomized participants, form the analysis set for this particular decision. It must also say how a Week 12 value is selected if there are multiple measurements and what makes a value valid.
For example: “For this decision, use the change from baseline to the assessment nearest Week 12 within the defined Week 12 visit window, with ties resolved by selecting the earlier assessment. Include participants randomized to the treatment group with a valid baseline and at least one valid assessment in that window. Include only values with completed range checks and no unresolved query affecting the endpoint value in the interim extract generated at 17:00 UTC on 30 June 2026. Calculate the arithmetic mean of these participant-level changes. The criterion is met when the mean is less than or equal to \(-5\) points.” If central review can alter the endpoint value, whether it must be complete should be stated as well.
Why the Cutoff Is Part of the Decision Rule
A simple formalization shows why input definitions matter. Let \(D(t,S)\) denote the records and derived values that satisfy the stated population and data-status rules \(S\) and are eligible under cutoff rule \(t\). Let \(g\) be the prespecified calculation and \(c\) its decision threshold. The decision can be represented as:
where \(I\{\cdot\}\) equals 1 when the condition inside braces is true and 0 otherwise. If the wording allows two plausible rules, \(t_1,S_1\) and \(t_2,S_2\), it may produce different datasets and therefore different values of \(g\). Even if \(g\) and \(c\) are perfectly clear, the decision is not reproducible until the input is defined.
This is a logical dependency, not a claim that every possible cutoff choice has the same statistical consequences. A different cutoff can change the amount or composition of information available, so the intended timing and eligibility rules should be considered when the adaptation plan is developed. Documentation makes the chosen rule explicit; it does not prove that the adaptation preserves a desired error rate or other operating characteristic.
A practical justification is to trace the decision from the written rule to its inputs: identify the eligible participants, apply the endpoint derivation, apply the data-status filter, impose the cutoff, and calculate the statistic. If any step requires an undocumented judgment, the written specification does not yet identify a unique decision dataset.
Diagnostics, Alternatives, and Limitations
Several wording checks can reveal trouble before an interim decision:
- Ask two independent readers to list the included participants and records using only the specification. Differences are evidence that a definition is missing.
- Search for terms such as “available,” “clean,” “evaluable,” “complete,” and “as of interim.” Each should be defined or replaced with an operational condition.
- Check that the population, endpoint, visit window, denominator, unit of analysis, missing-data handling, and cutoff refer to compatible sets of records.
- Check whether each data-status condition can be determined at the stated cutoff. For example, “adjudicated before the decision” needs a defined point at which that status is assessed.
- Verify the boundary cases: a record exactly at the cutoff, a participant with multiple assessments, a value entered after the cutoff for an earlier event, and an unresolved query affecting the endpoint.
A calendar-based cutoff can provide a common snapshot time, but it may include different numbers of eligible events or assessments across groups. An event-driven cutoff can target a stated number of events or participants, but the event definition, counting rule, and confirmation procedure must be clear. A hybrid rule, such as reaching a target number of events by a maximum date, should state what happens if the target is not reached by that date. Neither approach is universally preferable; the chosen approach should match the intended design and be fully specified.
A single fixed snapshot is often easier to describe than a continually updated feed, because it identifies one version of the inputs. A rolling update may be appropriate in some settings, but it needs an equally clear rule for when the calculation is run and which records may enter each run. The key distinction is not “locked” versus “unlocked” as labels: it is whether the exact eligible data can be determined.
Precise wording has limits. It cannot remove uncertainty caused by missing outcomes, delayed data, or measurement error; it cannot replace suitable endpoint definitions or appropriate statistical analysis; and it does not establish that an adaptation is justified. Nor should the specification claim that later information is ignored for all trial purposes when the cutoff only defines the inputs for one decision. The scope of the cutoff should be stated: it governs the named adaptation calculation, not necessarily every subsequent analysis or data use.
Common Mistakes
- Using “data available at interim” without defining availability. Specify whether eligibility depends on event occurrence, entry, verification, adjudication, or presence in a dated extract.
- Giving a cutoff date without a time zone or endpoint. “31 March” is insufficient when records can arrive at different local times. State the time, time zone, and relevant timestamp.
- Using a population label as a complete definition. “Evaluable participants” should be accompanied by criteria for evaluability in this decision.
- Leaving the boundary of a threshold unclear. “Above 60%” excludes exactly 60%; “at least 60%” includes it. State the intended inequality.
- Treating data cleaning as self-explanatory. Identify which checks or reviews must be complete and how unresolved issues affecting the endpoint are handled.
- Assuming more information is always preferable. Adding late-arriving records may alter the interim calculation and depart from the specified snapshot. Follow the declared rule rather than changing the input opportunistically.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A rule says, “Continue if the event rate is at least 20% among available participants.” Name three details that must be clarified.
- A record describes an endpoint event on 8 May, but it is entered into the database on 12 May. A cutoff is 10 May. What must the specification say before deciding whether to include it?
- A mean-change rule uses a threshold of \(-3\) points and the defined decision dataset has 18 participant-level changes summing to \(-63\) points. Does the mean meet a criterion of “at most \(-3\) points”?
- Give one strength and one specification challenge of an event-driven cutoff compared with a calendar-based cutoff.
- Why does a clearly written threshold alone not guarantee that an interim decision is reproducible?
Solutions. (1) Clarify the event definition and assessment period, the population and denominator, and what “available” means at the cutoff. Also specify the cutoff timestamp and data-status requirements if they affect eligibility. (2) State whether eligibility follows the clinical event date or the database-entry or verification date, and define any required confirmation status. With a database-availability cutoff of 10 May, the record entered on 12 May is excluded; with an event-date rule it may be included if the specified verification conditions are met. (3) The mean is \(\bar{x}=-63/18=-3.50\) points. Since \(-3.50\leq -3\), it meets the stated criterion. (4) An event-driven cutoff can target a specified amount of event information; it must define what counts as an event and how the target is confirmed. A calendar cutoff gives a common time boundary; it may yield different numbers of events or assessments. (5) The threshold does not identify which participants, endpoint values, data statuses, or snapshot enter the calculation. Different plausible inputs can produce different statistics and decisions even when everyone applies the same threshold correctly.