Preparing a Bayesian Pre-Submission Briefing Package

Purpose and Scope of a Briefing Package

A pre-submission briefing package is a structured aid for a technical discussion with a regulatory agency. Its job is to make a proposed trial design understandable and reviewable before the sponsor commits to it. A strong package shows how the clinical objective connects to the design, analysis, and decision rule; states the assumptions that support those choices; summarizes evidence from simulations; and identifies the questions on which agency feedback is sought.

The package is not a substitute for the protocol, statistical analysis plan, or complete technical documentation. It is also not evidence that an agency has agreed with the proposal. Its purpose is to help reviewers find the important choices, assess their implications, and give feedback on clearly framed issues. Presenting a large volume of technical output without identifying the choices at stake can make that task harder rather than easier.

A Reviewable Package: What to Include

The most useful organization follows the logic of the proposal rather than the order in which analyses happened to be conducted. Begin with the product or intervention, trial population, endpoint, and intended use of the evidence. Then summarize the design and analysis choices needed to interpret the proposed decision. The package should identify, at minimum, the analysis population, endpoint definition and assessment time, planned sample size, interim looks if any, model, prior distributions, decision rule, and handling of relevant missing or pending outcomes. The detail should be sufficient to understand the proposal without requiring the reviewer to infer important rules from a diagram.

For each statistical component, distinguish what is fixed in advance from what is a sensitivity analysis or an open question. State assumptions in operational terms. For example, a binomial response model for a single-arm study assumes that, conditional on a common response probability \(p\), participants’ binary outcomes are independent and have the same probability of response. If enrollment, follow-up, outcome assessment, or the analysis population makes that description implausible, explain how the design or model accounts for the difference. A formula alone does not establish that its assumptions fit the trial.

A decision rule should be written in a form that can be applied to trial data, with its threshold and relevant analysis time stated. Consider a hypothetical single-arm study in which \(X\) of \(n\) participants have a response, with \(X\mid p\sim\operatorname{Binomial}(n,p)\). Suppose the prior is \(p\sim\operatorname{Beta}(1,1)\), and the proposed rule is to declare the Bayesian criterion met if the posterior probability that \(p\) exceeds 0.50 is at least 0.95. This is a hypothetical example, not a proposed clinical standard. Its assumptions include a defined binary response, complete and consistent assessment, and a common response probability for the participants represented by the model.

A useful package also shows how the proposal will be evaluated before trial data exist. Simulation plans should say what was simulated, under which scenarios, how the planned design and decision rule were applied, and what performance summaries were calculated. These details allow reviewers to distinguish the design’s operating characteristics under specified assumptions from claims about what will happen in the actual trial.

Simulation Evidence and Its Uncertainty

A simulated operating characteristic is an estimated probability of a design outcome under a specified data-generating scenario. For example, the probability of meeting a Bayesian success criterion can be estimated by simulating many hypothetical trials under a stated response rate and recording how many meet that criterion. Interpret the estimate only alongside its scenario: it is conditional on the assumed data-generating model, parameter values, sample size, follow-up, and implementation of the decision rule.

If \(M\) independent simulated trials produce a particular outcome in \(K\) trials, the estimated probability is \(\hat{q}=K/M\). Treating the simulation indicators as independent Bernoulli variables with probability \(q\), their sample proportion has estimated Monte Carlo standard error

This formula describes numerical variation from using a finite number of simulation replicates. It does not measure uncertainty about whether the assumed scenario is clinically realistic, whether the model is appropriate, or whether the design will perform as predicted in practice.

Show enough scenario detail to make the evidence interpretable. A compact table can include the assumed response rate or treatment effect, sample size, simulation count, estimated operating characteristic, and Monte Carlo standard error. Include scenarios that reveal meaningful trade-offs, such as performance when the assumed effect is absent, modest, or clinically important. Explain why each scenario matters; a table of numbers without that context can obscure what the simulations test.

These example rows are interpretable only if the package defines what the scenarios represent and what “meeting criterion” means. For instance, if Scenario A represents no clinically relevant effect, its 0.104 estimate is the simulated probability of meeting the success criterion under that scenario—not the probability that the treatment is ineffective. Similarly, Monte Carlo precision does not establish that the assumed no-effect scenario is the only relevant one.

Quality Checks, Failure Modes, and Alternatives

Before sending the package, check consistency across the narrative, protocol synopsis, analysis description, decision rule, and simulation code or outputs. The endpoint definition and analysis population should match; planned interim timing should match the rules applied in simulations; and the reported decision should follow the stated threshold. The package should identify the software or implementation sufficiently for a reviewer to understand how the analyses were produced. If a rule has edge cases—such as a boundary tie, an incomplete assessment, or a pending outcome—state how it is handled rather than leaving it implicit.

Diagnostics for simulation work include checking that replicate counts match the stated total, that the reported estimates equal event counts divided by replicates, and that Monte Carlo standard errors use those same estimates and counts. Compare independent runs or use more replicates if estimates are too noisy to support a design comparison. Inspect whether simulated data reproduce the intended assumptions, including outcome definitions and timing. A simulation that implements a different rule from the one described in the package is not evidence for the proposed design.

Common alternatives are a slide-only overview and a full technical report. Slides can support a meeting but often omit assumptions and implementation details needed for review. A complete technical report can document these details but may make the central questions hard to locate. A focused briefing package combines a short decision-oriented narrative with tables or appendices that preserve technical traceability. Neither alternative removes the need for a clear, internally consistent account.

A frequent mistake is to treat a precise simulation estimate as strong evidence that the design is reliable. For example, a Monte Carlo standard error of 0.005 may show that finite simulation noise is small, but it says nothing by itself about misspecified outcome rates, an unsuitable model, or changes in trial conduct. The correction is to separate numerical precision from the assumptions that determine the simulation results and to show the scenarios that probe those assumptions. Another pitfall is asking a broad question such as “Does the agency agree with the design?” Instead, identify the particular design choice, explain its consequences, and request feedback on that choice.

A briefing package has limits: it cannot guarantee agreement, replace complete documentation, or compensate for unsupported assumptions. Its value is that it makes a proposal legible enough for productive technical discussion. The strongest package is concise about the decision while detailed enough that a reviewer can follow the design, assumptions, evidence, and unresolved questions without reconstructing them independently.