Why Amendments Need a Clear Statistical Record
A Bayesian trial’s assumptions and decision rules help determine how evidence will be interpreted and what actions may follow. New operational information, a corrected scientific assumption, or a safety concern may make a change necessary while the trial is underway. The important regulatory question is not simply whether a change was made, but what changed, when, why, under whose governance, and with what effect on the interpretation of the evidence.
An amendment does not erase the analysis that was specified before the change. Preserve that original analysis as a distinct reference point, then describe and report the revised analysis separately. This distinction matters especially if anyone involved in proposing or approving a change has had access to unblinded outcome data. Access does not automatically make a scientifically justified change invalid, but it raises the need for transparent documentation and careful interpretation.
The earlier material on prespecifying adaptive rules and change control establishes why a trial needs governance for changes. Here the focus is how to document and interpret amendments that alter Bayesian assumptions or decision rules. The amendment’s exact status and required approval process depend on applicable regulations, ethics review, and trial governance; statistical documentation should make the change’s substance and timing clear regardless.
What to Record and Compare
A useful amendment record lets a reviewer reconstruct both the original plan and the revised plan without relying on memory or informal explanations. It should identify the affected protocol, statistical analysis plan, or other controlled documents and their version dates. It should also give the effective date and explain whether the change applies prospectively, to analyses of already accrued data, or to both.
- Specify the change. Identify the prior distribution, likelihood, borrowing rule, decision threshold, stopping boundary, or other affected element. State the old and new versions precisely.
- Explain the rationale. Describe the new information or scientific reasoning that prompted the change. Distinguish a correction to an error from a change in scientific judgment or trial conduct.
- Document timing and access. Record when the proposal was first considered, when it was approved, whether outcome data had been reviewed, and who had access to unblinded information.
- Identify governance. Record the responsible decision-makers, relevant independent review, approvals, and any protections used to manage access to accumulating outcome data.
- Assess the impact. Compare the original and revised analyses on the same data when possible. Describe changes in posterior summaries, decision probabilities, and resulting conclusions, as relevant to the amendment.
- Preserve reproducibility. Retain the dated versions, code or specifications needed to reproduce each analysis, and a record of which analysis supports each reported conclusion.
The comparison should be interpretable, not just a list of changed numbers. Explain whether the change affects the target question, evidence model, or action threshold. For example, changing a prior changes how the likelihood is combined with prior information; changing a decision threshold changes the action taken for a given posterior distribution. Those are distinct changes and should not be conflated.
Worked Example: Amending a Prior Distribution
Suppose a hypothetical trial estimates a treatment effect \(\theta\), measured in a clinically interpretable unit. Assume four independent observations \(Y_i\) satisfy \(Y_i\mid\theta\sim N(\theta,1)\), with known variance 1, and the observed mean is \(\bar{x}=0.60\). The original plan uses \(\theta\sim N(0,1)\); a proposed amendment uses a more skeptical prior, \(\theta\sim N(0,0.25)\). For illustration, the decision rule is to declare the posterior probability \(P(\theta>0\mid\text{data})\) above 0.85.
For a normal prior \(N(m_0,v_0)\) and this normal sampling model, the posterior variance \(v_1\) and mean \(m_1\) are
With the original prior, \(m_0=0\), \(v_0=1\), \(n=4\), and \(\sigma^2=1\). Thus \(v_1=1/(1+4)=0.20\), \(m_1=0.20(4\times0.60)=0.48\), and the posterior standard deviation is \(\sqrt{0.20}=0.4472\). Therefore
The original analysis exceeds the illustrative 0.85 threshold. Under the proposed prior, \(v_0=0.25\), so \(v_1=1/(4+4)=0.125\), \(m_1=0.125(4\times0.60)=0.30\), and the posterior standard deviation is \(\sqrt{0.125}=0.3536\). The probability becomes \(\Phi(0.30/0.3536)=\Phi(0.8485)\approx0.802\), below the threshold.
A suitable record would state why the prior changed, when the change was proposed and approved, and who had access to outcomes at those times. It would report both results on the same four observations, explicitly identifying which prior was original and which was amended. The numerical difference alone does not establish which prior is scientifically preferable.
How Changes Affect Decisions
A decision rule turns an analysis result into an action. For example, a rule may call for a particular conclusion when a posterior probability exceeds a specified cutoff. Changing that cutoff can change the action even when the data, likelihood, prior, and posterior distribution remain unchanged. The amendment record should therefore report the evidence result and the rule-based action separately.
Worked Example: Amending a Posterior Probability Threshold
In a hypothetical trial with a binary outcome, suppose \(X\) of \(n\) participants have a response, where \(X\mid p\sim\operatorname{Binomial}(n,p)\). Use a \(\operatorname{Beta}(1,1)\) prior and observe 15 responses among 20 participants. The posterior is \(\operatorname{Beta}(16,6)\). The original rule requires \(P(p>0.5\mid\text{data})>0.95\); an amendment proposes raising the cutoff to 0.99.
For this integer-parameter beta posterior, the probability can be calculated using the binomial-tail identity \(P(p>q\mid\text{data})=P\{Z\le a-1\}\), where \(Z\sim\operatorname{Binomial}(a+b-1,q)\) for \(p\sim\operatorname{Beta}(a,b)\). Here \(a=16\), \(b=6\), and \(q=0.5\), so \(Z\sim\operatorname{Binomial}(21,0.5)\). Consequently,
The posterior probability exceeds the original 0.95 cutoff, so the original rule signals its specified action. It does not exceed 0.99, so the amended rule does not signal that action. The posterior probability itself is the same under both rules; only the decision threshold changed. Documentation should say whether the amended rule was approved before or after outcome access, explain its rationale, and report the original and amended rule-based results separately. It should not relabel the amended cutoff as the trial’s original threshold.
The same discipline applies when an amendment changes a likelihood assumption. State the sampling model and its assumptions before interpreting the resulting analysis. In particular, do not treat a changed variance, endpoint model, or borrowing structure as a mere formatting update: it may change how much information the data contribute.
Worked Example: Amending a Likelihood Variance Assumption
Consider a separate hypothetical analysis with \(n=4\), \(\bar{x}=0.60\), prior \(\theta\sim N(0,1)\), and sampling model \(Y_i\mid\theta\sim N(\theta,\sigma^2)\) independently. The original analysis assumes \(\sigma^2=1\), while an amendment proposes \(\sigma^2=2.25\). For illustration, suppose the decision rule uses the same 0.80 cutoff for \(P(\theta>0\mid\text{data})\).
Under the original variance, the posterior mean is \(0.48\) and variance is \(0.20\), as calculated using the normal conjugate formulas above. Thus \(P(\theta>0\mid\text{data})=\Phi(0.48/\sqrt{0.20})\approx0.858\), above 0.80.
Under the proposed variance, the data precision is \(n/\sigma^2=4/2.25=1.7778\). The posterior variance and mean are
The posterior standard deviation is \(\sqrt{0.36}=0.60\), giving \(P(\theta>0\mid\text{data})=\Phi(0.384/0.60)=\Phi(0.640)\approx0.739\), below 0.80. The amendment therefore changes the illustrative rule-based conclusion. The numerical comparison does not justify choosing the variance assumption that produces a preferred result; the proposed model needs a scientific and statistical rationale, and both analyses must be labelled accurately.
Justifying and Evaluating an Amendment
The comparison is most informative when it isolates the change. If a prior is amended, hold the data, likelihood, and decision rule fixed for the direct prior comparison. If a threshold is amended, show the posterior result once and apply each threshold to it. If a likelihood is amended, explain why the observation model is now considered more appropriate and identify which other assumptions were held constant. If several elements change at once, report them individually where feasible and explain why their separate effects cannot be isolated otherwise.
When an amendment is prompted by information observed during the trial, distinguish a change to trial conduct from a change to the analysis. A change in enrollment or monitoring arrangements may have consequences for the analysis, but it does not automatically require a new statistical model. Conversely, new evidence about the endpoint or data-generating process may warrant revisiting a model even if trial operations do not change. For a change prompted after unblinded results were seen, preserve the dated record of access and decision-making and treat analyses under the revised plan with that context visible.
Write down the original and proposed assumptions or rules precisely, including their scope and effective date.
Record the rationale, timing, approvals, and whether relevant decision-makers had access to accumulating outcome data.
Compare the original and revised analyses on the same data where possible, separating changes in posterior evidence from changes in the action rule.
Identify which analysis was prespecified, which was amended, and which supports each conclusion; retain enough information to reproduce both.
A sensitivity analysis and an amendment are related but not interchangeable. A sensitivity analysis explores how conclusions vary under alternatives without necessarily changing the primary analysis plan. An amendment formally revises the plan or trial conduct. Results from a sensitivity analysis can inform a proposed amendment, but they do not by themselves establish that the amended analysis was prespecified.
Diagnostics for a poor amendment record include conflicting version dates, an unexplained mismatch between the protocol and reported analysis, a claim that a revised rule was prespecified when it was not, or a comparison that changes the prior, likelihood, and threshold all at once without explaining their separate roles. Another warning sign is a rationale that describes only the observed result rather than the scientific or operational reason for the change. These gaps make it difficult to judge whether the revision was justified and how it affected the evidence.
Common Pitfall and Limitations
Wrong reasoning: “The amended analysis is now the official analysis, so the original plan no longer needs to be reported.” This erases the information needed to understand how the trial’s interpretation changed. Correction: preserve the original plan and its results, label the amended analysis and its timing, and explain the rationale and impact. The original analysis does not have to be treated as the only scientifically relevant analysis, but it must not be represented as though it incorporated a later change.
An amendment record cannot, by itself, remove the interpretive concerns that arise when a change follows unblinded outcome review. Nor can a numerical comparison establish that a prior, likelihood, or decision threshold is appropriate; that requires a defensible scientific rationale and the relevant design and governance context. The comparison clarifies what changed and what conclusions are sensitive to it. It does not replace appropriate review, trial governance, or transparent reporting.
Check Your Understanding
Try each question, then compare your reasoning with the solutions.
- A sponsor changes a posterior probability cutoff after reviewing unblinded outcomes. What should the amendment record make explicit?
- If the prior changes but the data and likelihood do not, what should be held fixed to isolate the prior’s impact?
- In the beta-binomial example, does the amended cutoff change the posterior probability or the action under the decision rule?
- Why is a sensitivity analysis not automatically equivalent to a protocol amendment?
Solutions. (1) The old and new cutoffs, rationale, proposal and approval dates, data-access context, responsible decision-makers, and impact under both rules should be recorded; the revised analysis should not be described as the original prespecified rule. (2) Hold the observed data, likelihood, and decision threshold fixed, and compare the posterior summaries or probabilities under each prior. (3) The posterior probability remains approximately 0.9867; the action changes because it exceeds 0.95 but not 0.99. (4) A sensitivity analysis explores alternatives, whereas an amendment formally changes the plan or trial conduct. An alternative analysis can be informative without becoming the amended primary plan.