Why Prespecification Matters
An adaptive trial can change course in response to accumulating data, but a decision is interpretable only if the basis for it is clear. Reviewers need to know what information could trigger a change, when that information would be assessed, and what action the design allowed. Without those details, it may be impossible to distinguish implementation of the planned design from a decision made after seeing results.
Prespecification does not mean that every operational detail must be frozen regardless of circumstances. It means that planned adaptations are described in enough detail to be applied consistently, and that departures from the plan follow a defined change-control process. The aim is to make the path from eligible data to trial action transparent and reproducible, not to eliminate all legitimate changes.
For a rule to be operational, it must specify more than a general intention such as “increase enrollment if results are promising.” At minimum, the plan should identify the decision quantity, how it is calculated, the population or data included, the boundary values, the permitted action for each outcome, and who may implement that action under the trial’s procedures. The design rationale and supporting operating-characteristic evaluations should be consistent with the written rule.
Representing an Adaptive Rule Precisely
A useful way to write a rule is as a mapping from information available at a defined decision time to an action. Let \(D_t\) denote the eligible data at assessment \(t\), and let \(g(D_t)\) be the prespecified decision quantity. The rule partitions possible values of \(g(D_t)\) into regions and assigns one action to each region. If the data, calculation, or boundary convention is ambiguous, two analysts could apply the same nominal rule and reach different actions.
This expression is a documentation principle, not a new statistical model. The function \(g\) must be defined for the chosen analysis, and \(h\) must assign an action to every possible result, including values exactly on a boundary. The plan should also say what happens if the assessment cannot be completed because a required input is unavailable or the prespecified minimum data condition is not met.
State the trigger or calendar time, eligible participants, data cutoff, and minimum data completeness required.
Name the model or summary, its inputs, relevant prior or other assumptions, and the handling of incomplete or unavailable observations.
Give numerical boundaries and state whether each boundary is inclusive or exclusive.
Specify the operational consequence of each region, any limits on that action, and what happens if no region’s data requirements are met.
The timing specification matters as much as the threshold. “After 40 patients” could mean after enrollment, treatment completion, a specified follow-up visit, or availability of the outcome. These are not interchangeable. A plan should state the event that triggers the assessment and the rule for assigning observations to its data cutoff. It should also say whether an assessment occurs once or can recur, and how later assessments relate to earlier ones.
Worked Examples of Specifying and Applying Rules
Worked Example: A Sample-Size Adaptation Based on a Defined Summary
Consider a hypothetical single-arm trial with a binary response outcome. The protocol specifies one adaptation after 40 evaluable participants have completed the response assessment. For illustration, suppose the response probability \(p\) has a Beta(2,2) prior and the planned analysis uses the beta-binomial model. The rule is: stop for futility if the posterior mean is below 0.35; retain the planned total of 60 participants if it is at least 0.45; otherwise increase the maximum total to 80. These numbers and actions are hypothetical design choices, not general recommendations.
The rule also defines an evaluable participant as one with the specified response assessment available by the cutoff. Patients enrolled but without that assessment are not counted among the 40; the assessment is delayed until 40 evaluable outcomes are available. Let \(x\) be the number of responses among those 40. The posterior distribution is Beta\((x+2, 40-x+2)\), so its mean is:
Suppose \(x=17\). Then the posterior mean is \(19/44=0.4318\). Since \(0.35 \leq 0.4318 < 0.45\), the rule assigns the intermediate region and the maximum total sample size becomes 80. If instead \(x=15\), the mean is \(17/44=0.3864\), also in the intermediate region. If \(x=13\), the mean is \(15/44=0.3409<0.35\), so the rule calls for stopping for futility.
The calculation alone is not the full specification. The plan must also state that the lower boundary is included in the intermediate region, the upper boundary is included in the retain region, and which action applies if a value equals either boundary. Here, a mean of exactly 0.35 is assigned to the intermediate region, while exactly 0.45 is assigned to the retain region. This removes boundary ambiguity.
Worked Example: Specifying a Data Cutoff and an Incomplete-Data Rule
Suppose a hypothetical randomized trial plans an interim assessment when 50 participants have completed 12 weeks of follow-up. Its rule uses the number of participants with a recorded primary outcome at that time. The document states that the data cutoff is the date on which the 50th participant completes the 12-week visit; only outcomes recorded by that cutoff enter the assessment. The prespecified calculation is a response proportion among participants with an observed outcome, and the planned action is to increase the enrollment cap from 100 to 120 if that proportion is at least 0.40.
At the cutoff, 50 participants have reached the visit, but only 46 outcomes are recorded; 20 of those 46 are responses. The stated proportion is \(20/46=0.4348\). Because \(0.4348 \geq 0.40\), the rule calls for increasing the cap to 120. It does not use \(20/50=0.4000\), because that would treat the four outcomes not recorded by the cutoff as nonresponses, a different analysis not specified in this example.
The example also reveals an important limitation: an observed-outcome proportion can be sensitive to missing outcomes. A complete design description must say whether this calculation is in fact the intended one, whether a minimum number or proportion of outcomes must be available, and what happens if the minimum is not met. For example, the rule could require at least 45 recorded outcomes and defer assessment if fewer are available. That condition must be set before the data are observed, not chosen in response to the observed result.
Worked Example: Applying Change Control to a Proposed Rule Revision
A hypothetical trial prespecifies a single assessment after 40 evaluable outcomes, with an increase in maximum enrollment if a stated posterior summary is at least 0.45. Before the assessment, the team notices that follow-up is slower than expected and proposes moving the assessment to 35 evaluable outcomes. The proposal could change which data inform the adaptation and therefore may affect the design’s behavior.
The first question is whether the original design already permits this timing change. Suppose it does not. The team should not simply use 35 outcomes and describe the result as application of the original rule. It should document the proposed change, its rationale, the planned effective date, the decision authority required by the applicable procedures, and its implications for the design. The change should be evaluated and formally handled before the new rule is applied, consistent with applicable requirements.
Now suppose the team waits until the original assessment is complete, sees a posterior summary of 0.43, and then proposes lowering the adaptation boundary from 0.45 to 0.42 so the enrollment cap will increase. This change is informed by the observed assessment result and directly alters the action that the original rule would produce. It cannot be presented as a prespecified adaptation. The record should distinguish the original rule and its result from any later proposal and its status; retrospective relabeling would obscure how the decision was made.
Justification: Why a Complete Rule Is Reproducible
A complete rule supports reproducibility because it fixes the inputs, the calculation, and the mapping to actions before the result is known. Given the same eligible dataset \(D_t\), the same calculation \(g\), and the same action mapping \(h\), independent applications yield the same action. Conversely, if a plan leaves the cutoff unspecified, one analyst might include a late outcome that another excludes. If it leaves a boundary unspecified, one might adapt at exactly 0.45 while another might not. The difference is not computational; it is an avoidable gap in the written rule.
The reasoning can be stated directly. Assume that (1) the eligible data are identified unambiguously, (2) the calculation is fully specified, and (3) the action regions cover all possible calculation results without overlap. Then each assessment produces one defined result \(g(D_t)\), and that result belongs to exactly one region. The rule therefore assigns exactly one action. If any assumption fails, the procedure may be indeterminate or allow discretionary choices not described as part of the design.
This is why a protocol or statistical analysis plan should not rely on phrases such as “as appropriate,” “if results warrant,” or “after sufficient follow-up” without operational definitions. Those phrases may be useful in explanatory prose, but they do not substitute for the rule itself.
Change Control, Alternatives, and Diagnostics
A useful change-control plan distinguishes planned flexibility from a revision to the design. Planned flexibility is built into the original rule: for example, a specified assessment window, a stated range of enrollment caps, or a defined fallback if the minimum data condition is unmet. A revision changes what the written plan permits. The change record should identify what text or calculation changes, why the change is proposed, when it becomes effective, what data were available to decision-makers, and how the change was authorized and communicated under applicable procedures.
Different approaches have different strengths. A fixed design avoids data-dependent adaptation but cannot use interim information to alter the planned course. A prespecified algorithm permits defined responses to accumulating data and allows its behavior to be assessed during design planning. An ad hoc discretionary approach can respond flexibly, but without predefined boundaries and documentation it is difficult to establish that decisions followed a consistent plan. These approaches are not interchangeable; the appropriate one depends on the trial question, the design, and applicable requirements.
| Approach | What is specified | Main concern |
|---|---|---|
| Fixed design | Planned conduct and analysis without a data-triggered adaptation | Does not use interim information to change the planned design |
| Prespecified algorithm | Assessment, data, calculation, boundaries, and actions set in advance | Ambiguity or implementation outside the stated rule |
| Ad hoc discretion | Decision made after considering accumulating information, without a complete prior mapping | Harder to distinguish planned flexibility from data-driven revision |
Diagnostics for the documentation include checking whether an independent reader can answer the following without asking the design team: What event triggers the assessment? Which participants and outcomes are included? How are late, missing, or conflicting records handled? What exact calculation is used? What happens at each boundary? Is there a single action for every possible result? Is the proposed action within the permitted range? Does the version in force predate the decision and the relevant data access?
A common mistake is to treat any change made before the final analysis as prospective and therefore harmless. The timing alone does not establish that a change was part of the original design. A rule modified after interim results were available is a revision informed by those results, even if enrollment is still ongoing. The correction is to preserve the distinction between the original prespecified rule and the revised proposal, document when each applied, and avoid describing the revised action as though it followed the original rule.
Prespecification also has limits. It cannot guarantee that data are complete or unbiased, that the model is appropriate, or that operational circumstances will never require a change. Nor does a detailed rule replace the design justification or operating-characteristic evaluation. It makes the planned procedure assessable; it does not by itself establish that the procedure is scientifically suitable.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A rule says, “At interim, expand enrollment if the posterior probability is high.” Name four details needed to make it operational.
- A prespecified rule uses outcomes recorded by the date the 30th participant completes follow-up. Only 27 outcomes are recorded then, and the rule has no minimum-completeness requirement. What is the documentation problem?
- A decision quantity is 0.50. The plan says to adapt when the value is “above 0.50” and retain the design when it is “below 0.50.” What is missing?
- After seeing an interim result, a team lowers a decision boundary before the trial’s final analysis. Is that automatically an application of the original prespecified rule? Explain.
Solutions. (1) Specify the assessment trigger and cutoff; the eligible data; the precise posterior probability and model used to calculate it; and the numerical boundary, boundary convention, and resulting action. (2) The rule does not establish whether to assess with 27 outcomes, wait for more outcomes, or use another handling procedure. The minimum data condition and fallback action should have been defined in advance. (3) Neither condition includes exactly 0.50, so the action is undefined at that value. Assign the boundary to one region or specify a separate action. (4) No. Because the boundary was changed after the interim result was available, it is a revision informed by accumulating data. The original rule and its result must remain distinguishable from the revision and any action taken under it.