From a Regulatory Decision to an Evidence Question
A Bayesian analysis is most useful for regulatory review when it answers a defined question that matters for a defined decision. Reporting a posterior distribution, a credible interval, or a probability of benefit is not by itself a rationale for approval, a labeling claim, or another regulatory action. The analysis must connect what the decision-maker needs to decide with the population to whom the decision applies, the claim being proposed, and the evidence that would support that claim.
The regulatory question should be framed before choosing or presenting an analysis. For example, “Does the treatment work?” leaves important matters unspecified: work for whom, on which outcome, compared with what, over what period, and to what degree? A reviewable question identifies these elements sufficiently to show how the proposed evidence bears on the intended decision. The earlier material on regulatory evidence standards and ICH E9 principles provides relevant context; here the focus is making the question-and-evidence chain explicit.
These four elements are related but not interchangeable. The decision is the action or choice under review, such as whether a product may be used for a specified indication. The target population describes the patients to whom that action or claim would apply. The claim states what is asserted about the product in that population. The evidence question defines what must be learned from the available data to evaluate the assertion.
A useful rationale also identifies the comparator, outcome, time horizon, and relevant treatment effect. These details help define the estimand: the specific treatment effect of interest in the specified population under the stated conditions. An analysis of a different endpoint, population, or time period may still be informative, but its relevance to the proposed claim must be explained rather than assumed.
A Practical Structure for the Rationale
A strong rationale can be organized as a short chain of questions. First, name the regulatory decision and its consequences. Second, define the target population and intended use. Third, state the proposed claim in terms that can be evaluated. Fourth, specify the evidence quantity that would bear on that claim, the model used to estimate it, and how the result is intended to inform the decision. Finally, state assumptions and limitations that could weaken the link between the observed data and the proposed claim.
State which regulatory action is under consideration and what choice the evidence is intended to inform. Do not imply that a statistical result alone determines the action.
Describe who is eligible, the clinical setting, and any important boundaries on the proposed use.
Specify the outcome, comparator if relevant, time point, and clinically meaningful effect or threshold.
State the posterior quantity that addresses the claim, the data and model underlying it, and any proposed decision rule.
Explain why the analyzed data and assumptions are relevant to the population, claim, and decision, and identify material uncertainty or limitations.
The posterior evidence should match the claim. If a claim is that a response rate exceeds a clinically relevant minimum, a posterior probability that the rate exceeds that minimum may be directly relevant. If the claim concerns improvement relative to standard care, a single-arm response rate generally cannot establish the comparative effect by itself. In either case, the rationale must distinguish the probability statement from the policy or regulatory decision made using it.
Worked Applications
Worked Example: A Minimum Response-Rate Claim
Hypothetical case. A sponsor seeks a claim that a treatment produces a response in more than half of adults with a specified condition who meet the proposed eligibility criteria. In a single-arm trial, 8 of 10 participants respond. For illustration, let \(p\) be the response probability in that population, use a uniform prior \(p\sim\operatorname{Beta}(1,1)\), and use a binomial likelihood. Suppose the proposed statistical evidence criterion is \(P(p>0.50\mid\text{data})\geq 0.95\). This criterion is an illustrative prespecified rule, not a universal regulatory standard.
With \(x=8\) responses among \(n=10\) participants, beta-binomial conjugacy gives the posterior distribution
For integer beta parameters, the beta-binomial identity gives \(P(p>0.50)=P(Y\leq 8)\), where \(Y\sim\operatorname{Binomial}(11,0.50)\). Thus
Under the stated model and illustrative criterion, the posterior probability exceeds 0.95. The result supports the specified evidence question: whether the response probability exceeds 50% in the modeled population. It does not, by itself, establish comparative benefit, describe durability of response, or guarantee the regulatory outcome. The rationale should address whether the trial participants and response assessment represent the proposed population and claim.
Worked Example: A Claim About Improvement in Time
Hypothetical case. A proposed claim is that a treatment reduces time to symptom resolution by more than 3 days relative to control among adults with a defined acute condition. Let \(\theta\) denote the mean reduction in days, with positive values favoring treatment. Suppose a randomized comparison produces an estimated reduction of 5 days with standard error 2 days. For a transparent illustration, assume the estimate has likelihood \(\bar{D}\mid\theta\sim N(\theta,2^2)\), and use the prespecified prior \(\theta\sim N(0,2^2)\). Assume the sampling model is appropriate and that the estimated standard error can be treated as known.
The posterior variance is the reciprocal of the sum of prior and likelihood precisions, and the posterior mean is the precision-weighted mean:
Therefore \(\theta\mid\text{data}\sim N(2.5,2)\), where the second parameter is the variance and the units of \(\theta\) are days. The posterior probability of exceeding the proposed 3-day threshold is
This posterior probability is well below an illustrative criterion of 0.95. The observed estimate is 5 days, but the posterior evidence for an effect larger than 3 days is not strong under the stated model. A rationale should not replace the proposed claim with the easier claim that the treatment has some positive effect: those are distinct questions. It should also make clear that this analysis addresses the mean reduction, not necessarily the distribution of individual patients’ recovery times.
Worked Example: A Safety Claim With Limited Exposure
Hypothetical case. A proposed rationale asks whether the probability \(q\) of a specified adverse event during a defined treatment period is below 10% in the intended population. Thirty participants receive treatment, and no such events are observed. For illustration, assume independent binary outcomes with a common event probability \(q\), complete event ascertainment, a fixed follow-up period, and a uniform prior \(q\sim\operatorname{Beta}(1,1)\).
After zero events in 30 participants, the posterior is \(q\mid\text{data}\sim\operatorname{Beta}(1,31)\). Its cumulative distribution function at \(q=0.10\) is \(1-(1-0.10)^{31}\), so
The calculation gives a high posterior probability that the event rate is below 10% under the specified model. It does not show that the event cannot occur, and it does not settle whether 10% is an acceptable safety threshold. Thirty participants may also provide little information about uncommon harms, longer-term risks, or risks in patients unlike those enrolled. The rationale must connect the event definition and follow-up period to the claim and explain the limits of the available exposure.
Why the Posterior Calculation Supports Only Part of the Rationale
The calculations above illustrate a general distinction: Bayesian analysis quantifies uncertainty conditional on a model, its assumptions, the prior, and the observed data. A regulatory decision additionally involves the proposed use, the clinical importance of the effect, the quality and relevance of the evidence, and the consequences of acting or not acting. Thus a statement such as \(P(\theta>c\mid\text{data})=0.96\) means that, under the specified analysis, the posterior probability that \(\theta\) exceeds \(c\) is 0.96. It does not mean there is a 96% probability that the treatment will work for every patient, or that the regulator must take a particular action.
A brief derivation shows why the assumptions matter. If an estimate \(y\) has a normal likelihood with known standard error \(s\), and the prior for an effect \(\theta\) is normal with mean \(m_0\) and variance \(v_0\), then the posterior density is proportional to the product of the likelihood and prior:
Collecting the terms involving \(\theta\) and completing the square gives a normal posterior with variance \(v=(1/s^2+1/v_0)^{-1}\) and mean \(m=v(y/s^2+m_0/v_0)\). The resulting posterior probability therefore depends on the likelihood’s sampling assumptions and on the prior as well as on \(y\). This is why a rationale should identify the model and explain why it is relevant to the stated evidence question, rather than present a probability without its conditions.
Diagnostics, Alternatives, and Common Failure Modes
Before relying on a rationale, check for a mismatch at each link. Does the analyzed population correspond to the proposed target population? Does the endpoint and time point represent the claim? Is the comparator appropriate for a comparative assertion? Are the data sufficiently complete and the outcome assessed consistently? Do model assumptions such as independence, common event probability, or the stated sampling distribution make sense? If the answer to one of these questions is no, a precise posterior calculation may still answer a narrower question, but it cannot automatically support the broader claim.
The main alternatives are not merely different formulas. A frequentist analysis may summarize evidence using confidence intervals and error-rate-controlled tests; a Bayesian analysis summarizes uncertainty using a posterior distribution conditional on its model and prior. Either approach requires the scientific question, population, endpoint, and design to be aligned. Regulatory decisions may also draw on multiple evidence sources and considerations beyond a single statistical summary. The rationale should explain why the chosen Bayesian evidence is useful for this decision, not suggest that Bayesian analysis removes the need to examine design quality or clinical relevance.
Common mistake: “The posterior probability of benefit is above 0.95, so approval follows.” This skips the definition of benefit, the target population, the claim, the evidence assumptions, and the decision context. A corrected statement is: “Under the prespecified model and prior, the posterior probability that the defined effect exceeds the stated clinically relevant threshold is 0.96. This result meets the proposed statistical evidence criterion; its relevance to the requested claim also depends on the trial’s population, endpoint, follow-up, and other evidence considered in the review.”
A useful final check is whether a reviewer could identify, from the rationale alone, what is being decided, for whom, what exactly is claimed, and which evidence quantity addresses that claim. If any part remains ambiguous, the analysis may be mathematically correct but poorly connected to the intended decision.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A proposed claim concerns reduced hospitalization compared with standard care in adults over 65. Name four details that should be specified to make the evidence question reviewable.
- A single-arm trial estimates a response rate of 70%. Why does this alone not establish that the treatment is better than standard care?
- In the safety example, what assumptions allow the zero-event calculation, and name one limitation that remains even if those assumptions hold.
- A posterior probability that an effect exceeds a threshold is 0.97. State what this means and one conclusion it does not justify by itself.
- Give one example of a mismatch between a trial population and a proposed target population, and explain why the mismatch matters.
Solutions. (1) Specify the eligible population and setting, the comparator, the hospitalization outcome and follow-up time, and the effect or threshold relevant to the claim; also state the posterior evidence criterion if one is proposed. (2) A single-arm response rate contains no direct concurrent estimate of the standard-care outcome, so it does not by itself identify a comparative treatment effect. (3) The calculation assumes independent binary outcomes, a common event probability, complete ascertainment, a fixed observation period, and the stated prior. Even then, a small sample may provide limited information about rare events or longer-term risk. (4) Conditional on the model, prior, and observed data, the posterior probability that the defined effect exceeds the threshold is 0.97. It does not alone establish that the treatment is appropriate for every patient or dictate a regulatory action. (5) For example, a trial enrolling only younger adults may not represent a claim covering older adults with multiple comorbidities; differences in response, risk, or treatment exposure could limit generalization.