Bayesian vs. Frequentist Inference: Key Differences

Introduction

Statistical inference is the process of using observed data to learn about a population, a treatment effect, or another unknown quantity.

Two major frameworks dominate modern statistical inference: frequentist inference and Bayesian inference.

Both frameworks use probability, likelihoods, statistical models, and observed data. Both can estimate treatment effects, quantify uncertainty, compare treatments, and support clinical decisions.

The fundamental difference is not that one framework uses mathematics while the other does not.

The deeper distinction is how the two frameworks interpret probability and unknown parameters.

The Central Question

Suppose a clinical trial compares an experimental treatment with a control. Let:

The investigator observes data \(D\) and wants to learn about \(\theta\).

Both approaches ask essentially the same scientific question: What do the observed data tell us about the treatment effect?

But they answer the question differently.

What Does "Frequentist" Mean?

The frequentist framework defines probability in terms of long-run behavior under repeated sampling.

Suppose a parameter \(\theta\) is fixed. The observed sample is one realization from a hypothetical process that could be repeated many times under the same conditions.

The parameter itself is not assigned a probability distribution in the standard frequentist formulation.

For example, suppose:

is the true response probability.

A frequentist treats \(p=0.30\) as a fixed value, even though investigators do not know what that value is.

The number of responses observed in a sample is random.

The probability distribution describes how \(X\) would behave over repeated samples if \(p\) were fixed.

What Does "Bayesian" Mean?

Bayesian inference takes a different approach.

Instead of treating the unknown parameter as a fixed value with no probability distribution, Bayesian inference represents uncertainty about the parameter using a probability distribution.

Before observing the current data, the investigator specifies a prior distribution:

After observing data \(D\), the prior is updated using Bayes' theorem to produce the posterior distribution:

where:

• \(\pi(\theta)\) is the prior distribution.
• \(L(\theta;D)\) is the likelihood.
• \(\pi(\theta\mid D)\) is the posterior distribution.

Bayes' Theorem

The mathematical foundation of Bayesian inference is Bayes' theorem.

The components have intuitive interpretations:

In many practical Bayesian analyses, the denominator is treated as a normalizing constant:

This equation is one of the most important formulas in Bayesian statistics.

The Likelihood Is Important in Both Frameworks

Bayesian and frequentist inference are not completely separate mathematical worlds.

Both commonly begin with a statistical model and likelihood.

For observed data \(D\), the likelihood is:

The interpretation is different.

A frequentist may use the likelihood to construct an estimator, test statistic, or confidence interval.

A Bayesian multiplies the likelihood by the prior distribution to obtain the posterior distribution.

A Simple Binary Example

Suppose a new treatment is evaluated in 10 patients. Assume that 4 patients respond.

The observed response rate is:

The statistical question is: What should we conclude about the true response probability \(p\)?

A frequentist might construct a confidence interval and test:

A Bayesian might instead specify a prior for \(p\), combine it with the binomial likelihood, and obtain a posterior distribution.

The Frequentist Binomial Model

For \(X=4\) responses among \(n=10\) patients:

The likelihood is:

The maximum likelihood estimate is:

A frequentist analysis can then use \(\hat p\), a confidence interval, and possibly a hypothesis test.

The Bayesian Binomial Model

Now suppose the investigator uses a Beta prior:

The Beta distribution is particularly convenient for a binomial endpoint because it is conjugate to the binomial likelihood.

The prior density is proportional to:

After observing \(x\) responses among \(n\) patients, the posterior distribution is:

This gives a direct probability distribution for the unknown response rate.

A Worked Bayesian Example

Suppose the prior is:

The prior mean is:

Thus, before observing the current trial, the prior centers on a 20% response probability.

Now observe:

The posterior becomes:

The posterior mean is:

Notice what happened. The observed sample proportion was 40%, but the posterior mean is 30%.

The Bayesian estimate has been pulled toward the prior because the sample size is relatively small.

What Happens With More Data?

Suppose instead that 40 responses are observed among 100 patients:

Using the same prior:

the posterior becomes:

with posterior mean:

The posterior mean is now much closer to the observed proportion of 40%.

This illustrates an important general principle: As the amount of informative data increases, the influence of a reasonable prior typically becomes smaller.

Prior Information Is Not Necessarily Subjective Guesswork

The word "prior" sometimes creates the impression that Bayesian analyses are based on arbitrary opinions.

That is not necessarily the case.

Prior distributions can be based on:

• Previous clinical trials
• Historical control data
• External observational studies
• Pharmacological or biological knowledge
• Expert elicitation
• Weakly informative distributions
• Explicitly skeptical or conservative assumptions

A prior can also be deliberately chosen to be weakly informative so that the current data dominate the analysis.

What Is a p-Value?

One of the most important differences between the two frameworks concerns hypothesis testing.

Suppose:

A p-value is the probability, under the null model, of observing a test statistic at least as extreme as the one actually observed, according to the specified test.

Symbolically:

The conditioning is critical.

The p-value is calculated assuming the null hypothesis is true.

What Does a p-Value of 0.03 Actually Mean?

Suppose a valid test produces:

The appropriate interpretation is approximately: If the null hypothesis were true, results at least as extreme as those observed would occur with probability 0.03 under the specified testing procedure.

It does not directly provide:

• The probability that the null hypothesis is true
• The probability that the alternative hypothesis is true
• The probability that the observed effect will replicate
• The probability that the treatment works

Bayesian Hypothesis Probabilities

Bayesian inference can directly calculate posterior probabilities of parameter regions.

For example:

can be interpreted as the posterior probability that the treatment effect is positive, given the observed data and the specified model and prior.

Similarly, a clinically important threshold can be evaluated directly:

where \(\delta\) is a clinically meaningful treatment-effect threshold.

Confidence Intervals vs. Credible Intervals

Another major difference concerns interval estimates.

Frequentist analyses commonly produce a confidence interval.

Bayesian analyses commonly produce a credible interval.

Although they may look mathematically similar, they have different interpretations.

Frequentist Confidence Interval

Suppose a 95% confidence interval for a treatment effect is:

The frequentist interpretation is not that there is a 95% probability that the fixed parameter lies between 0.10 and 0.50.

Instead, the procedure used to construct the interval has 95% coverage under the repeated-sampling framework, assuming the relevant model and conditions hold.

Informally: If we repeatedly generated samples under the same conditions and constructed intervals using this procedure, approximately 95% of those intervals would contain the true parameter.

Bayesian Credible Interval

Suppose the Bayesian posterior distribution produces a 95% credible interval:

The Bayesian interpretation is: Given the data, model, and prior, there is 95% posterior probability that the parameter lies between 0.10 and 0.50.

This is a direct probability statement about the parameter under the Bayesian model.

A Very Important Distinction

Consider the statement:

This is a Bayesian-style posterior probability statement.

A frequentist confidence interval does not generally justify writing:

because the standard frequentist framework does not assign a probability distribution to the fixed parameter.

Bayesian Point Estimates

Bayesian analyses can summarize a posterior distribution in several ways.

Common summaries include:

• Posterior mean
• Posterior median
• Posterior mode
• Maximum a posteriori estimate

The posterior mean is:

The posterior median is the value dividing the posterior probability into two equal halves.

The posterior mode is the value at which the posterior density is maximized.

These summaries can differ substantially when the posterior distribution is skewed.

Frequentist Point Estimates

Frequentist analyses commonly use the:

• Maximum likelihood estimator
• Method-of-moments estimator
• Least-squares estimator
• Other model-specific estimators

For a binomial response probability, the maximum likelihood estimator is:

The frequentist estimate is therefore based entirely on the observed likelihood under the specified model.

Maximum Likelihood vs. Maximum A Posteriori

The distinction becomes particularly clear when comparing maximum likelihood and Bayesian posterior mode estimation.

The maximum likelihood estimator is:

The maximum a posteriori estimator is:

The Bayesian estimate incorporates the prior through the factor \(\pi(\theta)\).

With a sufficiently weak prior or a very large dataset, the MLE and MAP estimate may be very similar.

Bayesian Posterior Predictive Inference

One particularly powerful Bayesian concept is the posterior predictive distribution.

Instead of asking only about the treatment parameter, we can ask: What outcomes should we expect for future patients?

The posterior predictive distribution is:

The integral averages predictions over the posterior uncertainty in \(\theta\).

This naturally incorporates both:

• Uncertainty about the parameter
• Random variation in future observations

Bayesian Decision Making

Bayesian inference is also closely connected to formal decision theory.

Suppose a treatment decision has possible actions \(a\), and each action has a utility or loss depending on the unknown parameter \(\theta\).

The Bayesian expected loss is:

The preferred action can then minimize posterior expected loss:

This provides a direct framework for connecting statistical uncertainty to a clinical or operational decision.

A Clinical Trial Example

Suppose an experimental drug is compared with standard therapy. The treatment effect is a difference in response rates:

Suppose the observed data produce:

An investigator might want to know whether the treatment provides at least a 5-percentage-point improvement.

Define:

The Bayesian question is naturally expressed as:

For example, if the posterior probability were 0.94, the statement would be: Given the model, prior, and observed data, there is 94% posterior probability that the treatment effect exceeds 5 percentage points.

A frequentist analysis might instead report a confidence interval and a hypothesis test for:

The two approaches are addressing related scientific questions, but the probability statements are different.

Bayesian Hypothesis Testing Is More Than "Bayesian p-Values"

Bayesian inference has several possible approaches to hypothesis comparison.

For example, investigators may use posterior probabilities:

or Bayes factors.

For two competing hypotheses \(H_1\) and \(H_0\), the Bayes factor in favor of \(H_1\) is:

The Bayes factor compares how well the observed data are predicted under the two hypotheses.

Why p-Values and Bayes Factors Can Disagree

A p-value and a Bayes factor do not measure the same quantity.

A p-value evaluates extremeness under a null model.

A Bayes factor compares the marginal likelihood of competing models, with parameter uncertainty integrated over the relevant prior distributions.

Consequently, the two approaches can occasionally produce apparently conflicting conclusions.

Statistical Significance vs. Clinical Importance

Neither framework automatically solves the problem of determining whether an effect is clinically meaningful.

Suppose a large trial estimates:

with extremely small uncertainty.

A frequentist analysis may produce a very small p-value.

A Bayesian analysis may produce a very high probability that:

But the treatment effect may still be clinically trivial.

The relevant question may instead be:

where \(\delta\) represents a clinically meaningful threshold.

What Does "Non-Significant" Mean?

Suppose a frequentist analysis produces:

It is incorrect to conclude: "There is no treatment effect."

The appropriate conclusion is that the analysis did not provide sufficient evidence to reject the null hypothesis under the prespecified testing procedure.

The confidence interval may still contain a wide range of clinically important effects.

Similarly, a Bayesian posterior can show substantial uncertainty rather than simply classifying a treatment as "effective" or "ineffective."

Bayesian Analysis Makes Uncertainty Explicit

Suppose the posterior distribution for a treatment effect is broad.

Even if the posterior mean is positive, there may be substantial probability that the effect is clinically negligible or even harmful.

For example:

might sound encouraging.

But suppose:

Then the probability of a positive effect is high, while the probability of a clinically substantial effect is much lower.

The full posterior distribution allows these questions to be distinguished.

The Role of Sample Size

Sample size affects both Bayesian and frequentist inference.

With larger samples, the likelihood becomes more concentrated around parameter values supported by the observed data.

In a Bayesian analysis, this generally means the posterior becomes increasingly dominated by the likelihood when the prior is reasonably specified.

In a frequentist analysis, larger samples generally produce more precise estimators and narrower confidence intervals.

Prior Sensitivity

A Bayesian conclusion can depend on the prior, especially when the dataset is small or weakly informative.

Suppose one analysis uses:

and another uses:

The resulting posterior distributions may differ.

A responsible Bayesian analysis therefore asks: Would reasonable alternative priors materially change the conclusion?

This is called prior sensitivity analysis.

Weakly Informative Priors

A weakly informative prior attempts to provide useful regularization without strongly determining the result.

For example, in a regression model, an extremely broad prior may permit implausibly large parameter values and create computational or inferential problems.

A weakly informative prior can instead constrain the parameter to a scientifically reasonable region while allowing the observed data to have substantial influence.

Hierarchical Bayesian Models

One of the major strengths of Bayesian modeling is the ability to construct hierarchical models.

Suppose several clinical studies estimate related treatment effects:

A hierarchical model might assume:

where:

• \(\mu\) represents the overall treatment-effect distribution.
• \(\tau^2\) represents between-study heterogeneity.
• \(\theta_i\) represents the effect in study \(i\).

The model allows information to be shared across studies while still allowing each study to have its own treatment effect.

This is often called partial pooling.

Why Partial Pooling Matters

Suppose one study has a very small sample size while another has a much larger sample.

A hierarchical Bayesian model can allow the smaller study to borrow information from the broader population of studies without forcing all studies to have identical treatment effects.

This can be useful for:

• Multi-center clinical trials
• Multiple related indications
• Subgroup analyses
• Longitudinal data
• Repeated studies
• Rare disease research

Bayesian vs. Frequentist Regression

The distinction is not limited to simple proportions.

Consider a linear regression model:

A frequentist analysis estimates \(\beta_1\) and may report:

• The estimated coefficient
• Standard error
• Confidence interval
• Test statistic
• p-value

A Bayesian analysis could assign a prior:

and obtain:

The posterior distribution can then be summarized with a posterior mean, credible interval, and probabilities such as:

Bayesian vs. Frequentist Logistic Regression

The same distinction applies to logistic regression.

The frequentist model might be:

The odds ratio is:

A frequentist analysis might report a 95% confidence interval for the odds ratio and a p-value for:

A Bayesian analysis might instead obtain a posterior distribution for \(\beta_1\) and transform it to obtain a posterior distribution for \(OR\).

It could then calculate:

or, more clinically usefully:

Bayesian vs. Frequentist Survival Analysis

The distinction also extends to time-to-event analysis.

For example, a frequentist Cox model may estimate:

with hazard ratio:

A Bayesian survival model could instead assign a prior to \(\beta\), the baseline hazard, or other model parameters and estimate their joint posterior distribution.

The posterior can then answer questions such as:

or:

depending on the clinical question.

Bayesian Clinical Trials

Bayesian methods are particularly attractive in some clinical trial settings because they can formally incorporate external information and make probabilistic statements about treatment effects.

Applications include:

• Adaptive clinical trials
• Rare disease studies
• Small populations
• Seamless development programs
• Platform trials
• Historical borrowing
• Subgroup modeling
• Dose-finding studies
• Decision-based development programs

Bayesian Adaptive Randomization

In some Bayesian clinical trials, treatment allocation probabilities can be updated as information accumulates.

For example, suppose two treatments have posterior probabilities:

and:

These probabilities can potentially inform future randomization probabilities, subject to prespecified design rules.

The goal may be to balance statistical learning with patient allocation.

Bayesian Sample Size and Assurance

Traditional frequentist sample-size calculations typically specify:

• Type I error \(\alpha\)
• Power \(1-\beta\)
• Effect size
• Variability
• Allocation ratio

Bayesian designs may instead evaluate quantities such as the probability of making a correct decision under a range of assumed true parameter values.

A Bayesian design might calculate the probability that the posterior satisfies:

where \(c\) is a prespecified decision threshold.

The design can then be evaluated through simulation under multiple plausible true parameter values.

Bayesian Assurance

When uncertainty about the true treatment effect is represented with a prior, a design can be evaluated using the prior predictive distribution.

Conceptually, Bayesian assurance averages the probability of meeting the decision criterion over the prior distribution of possible true effects.

This differs from traditional frequentist power, which is generally evaluated at one or more fixed parameter values.

Bayesian Prior Predictive Checking

Before collecting data, Bayesian analysts can examine what the model and prior predict for observable outcomes.

This is the prior predictive distribution:

If the prior predictive distribution implies highly implausible datasets, the prior or model may need to be reconsidered.

This provides an important diagnostic that can be performed before analyzing the actual study data.

Model Assumptions Matter in Both Frameworks

It is tempting to think of Bayesian inference as assumption-free because it does not rely on p-values.

That is incorrect.

Bayesian models can depend on:

• Likelihood assumptions
• Prior distributions
• Structural model assumptions
• Distributional assumptions
• Missing-data assumptions
• Exchangeability assumptions
• Independence assumptions

Frequentist models likewise depend on assumptions.

For example:

• Independence
• Correct specification of the mean structure
• Variance assumptions
• Distributional approximations
• Proportional hazards
• Missing-data assumptions

Frequentist Advantages

Frequentist methods have several important strengths.

• Well-established theoretical properties
• Familiar regulatory framework
• Widely available software
• Clear long-run error-control concepts
• Well-developed sample-size methodology
• Standardized procedures for many clinical trial designs
• No explicit prior distribution required in the standard formulation

For many confirmatory clinical trials, these properties make frequentist methods highly attractive.

Bayesian Advantages

Bayesian methods offer several different advantages.

• Direct probability statements about unknown parameters
• Formal incorporation of prior information
• Natural treatment of parameter uncertainty
• Flexible hierarchical modeling
• Natural posterior prediction
• Useful framework for adaptive designs
• Ability to model complex multilevel structures
• Natural integration with decision analysis

These strengths can be particularly useful when sample sizes are small or external information is scientifically important.

Frequentist Limitations

Frequentist methods can be difficult to interpret when users expect direct probability statements about parameters.

For example, a confidence interval is often incorrectly interpreted as a probability statement about the parameter.

Frequentist procedures can also become complicated when:

• Multiple sources of prior information exist
• Data are hierarchical
• Adaptive decisions are required
• Several related populations need information sharing
• Small samples create unstable asymptotic approximations

This does not mean frequentist methods cannot handle these problems. They often can, but Bayesian models can sometimes express the structure more directly.

Bayesian Limitations

Bayesian methods also have important limitations.

• Results can depend on the prior
• Prior specification requires justification
• Complex models can be computationally intensive
• MCMC diagnostics may be required
• Model convergence must be assessed
• Regulatory familiarity varies by setting
• Different reasonable priors can produce different results in small datasets

When Do Bayesian and Frequentist Results Look Similar?

In many well-powered analyses, Bayesian and frequentist estimates can be quite similar.

Suppose the likelihood is highly concentrated because the sample size is large.

If the Bayesian prior is reasonably weak, the posterior distribution may be dominated by the likelihood.

The posterior mean may then be close to the maximum likelihood estimate.

Similarly, Bayesian credible intervals and frequentist confidence intervals may have very similar numerical endpoints.

This is one reason the distinction is sometimes less obvious in large, well-informed datasets.

When Can They Differ More?

Differences are more likely when:

• The sample size is small
• The data are sparse
• The parameter is weakly identified
• The prior is informative
• The model is hierarchical
• The parameter is near a boundary
• The likelihood is highly asymmetric
• The analysis concerns a nonlinear transformation

In these settings, the choice of inferential framework can materially affect the conclusions.

A Practical Comparison

A Common Misconception: "Bayesian Uses Beliefs, Frequentist Uses Facts"

This is an oversimplification.

Frequentist analyses also require assumptions and modeling choices.

For example, choosing:

• A regression model
• A covariance structure
• A missing-data method
• A test statistic
• A transformation
• An analysis population

requires judgment.

Bayesian inference makes one additional component—the prior distribution— explicit.

The scientific credibility of either approach depends on whether the assumptions are reasonable and transparently justified.

A Common Misconception: "A Bayesian Prior Is Just Extra Data"

A prior is not literally an additional dataset.

It is a probability distribution that represents uncertainty about the parameter before incorporating the current data.

A prior can be informed by previous data, but this does not mean the prior is automatically equivalent to adding those observations to the current trial.

The way historical information enters the model depends on the prior construction and hierarchical assumptions.

A Common Misconception: "Bayesian Methods Always Need Informative Priors"

They do not.

Bayesian analyses can use weakly informative or relatively diffuse priors.

The important point is that some prior distribution must be specified.

Even when the analyst intends the prior to have little influence, it is good practice to assess whether plausible alternative priors materially change the result.

A Common Misconception: "Frequentist Methods Cannot Use Prior Information"

Frequentist methods can incorporate historical or external information in various ways.

For example, investigators can:

• Use historical data in study design
• Specify informative covariates
• Construct stratified analyses
• Use external data in model development
• Use meta-analytic methods
• Design trials using prior information about variability or effect size

The distinction is that the standard frequentist inferential probability framework does not treat an unknown parameter as having a prior probability distribution.

R Example: Bayesian Beta-Binomial Analysis

The conjugate Beta-binomial model can be implemented very simply in R.

x <- 4
n <- 10

alpha_prior <- 2
beta_prior  <- 8

alpha_post <- alpha_prior + x
beta_post  <- beta_prior + n - x

alpha_post
beta_post

The posterior distribution is:

# Posterior:
# Beta(6, 14)

The posterior mean can be calculated as:

posterior_mean <-
  alpha_post /
  (alpha_post + beta_post)

posterior_mean
# 0.30

Calculate a Bayesian Credible Interval

The 95% equal-tail credible interval can be calculated using Beta quantiles.

qbeta(
  c(0.025, 0.975),
  shape1 = alpha_post,
  shape2 = beta_post
)

The resulting interval contains the central 95% of the posterior probability.

Unlike a frequentist confidence interval, it has a direct posterior probability interpretation conditional on the model and prior.

Calculate a Posterior Probability

Suppose the clinically meaningful response-rate threshold is 25%.

We want:

For the Beta posterior, this can be calculated directly:

1 - pbeta(
  0.25,
  shape1 = alpha_post,
  shape2 = beta_post
)

This is a direct Bayesian probability statement about the unknown response rate.

Compare With a Frequentist Analysis

The same data can be analyzed using a frequentist binomial method.

x <- 4
n <- 10

binom.test(
  x,
  n,
  conf.level = 0.95
)

The frequentist analysis provides a confidence interval based on the specified binomial procedure.

Notice that both analyses start with:

But the Bayesian analysis additionally specifies:

That additional modeling choice changes the interpretation of the resulting inference.

Frequentist Testing in R

Suppose the null hypothesis is:

A two-sided exact binomial test can be performed with:

binom.test(
  x = 4,
  n = 10,
  p = 0.20,
  alternative = "two.sided"
)

The resulting p-value answers a frequentist question about the extremeness of the observed data under the null model.

The Same Scientific Question in Two Languages

Suppose the investigator wants to know whether the response rate exceeds 20%.

A frequentist analysis may ask:

and report a p-value and confidence interval.

A Bayesian analysis may ask:

These are related but fundamentally different inferential statements.

Which Framework Is Better?

There is no universal answer.

The appropriate framework depends on the scientific question, available information, regulatory environment, sample size, complexity of the model, and decision structure.

Bayesian and Frequentist Methods Can Be Combined

The choice does not always need to be ideological.

A clinical development program may use frequentist methods for the primary confirmatory analysis while using Bayesian methods for:

• Predictive modeling
• Subgroup estimation
• External-data modeling
• Decision support
• Pharmacometric modeling
• Safety signal assessment
• Prior predictive simulation

Similarly, Bayesian and frequentist operating characteristics can both be examined when evaluating a clinical trial design.

What Matters More Than the Label?

The label "Bayesian" or "frequentist" does not by itself establish whether an analysis is scientifically sound.

A high-quality analysis should address:

• Whether the statistical model is appropriate
• Whether the estimand is clearly defined
• Whether the endpoint is clinically meaningful
• Whether missing data are handled appropriately
• Whether assumptions are reasonable
• Whether sensitivity analyses have been conducted
• Whether multiplicity is appropriately addressed
• Whether the analysis was prespecified when required
• Whether the resulting uncertainty is communicated honestly

The Core Differences at a Glance

A Mental Model for Remembering the Difference

A useful way to remember the distinction is to focus on the direction of conditioning.

Frequentist inference often asks:

That is: How would the data behave if the parameter had a particular value?

Bayesian inference asks:

That is: Given the observed data, what is our updated uncertainty about the parameter?

A Practical Workflow for Frequentist Inference

A Practical Workflow for Bayesian Inference

Common Mistakes

1. Calling a p-value the probability that the null hypothesis is true. A p-value is calculated under the null hypothesis; it does not provide the posterior probability that the null is true.
2. Interpreting a confidence interval as a Bayesian credible interval. The numerical appearance may be similar, but the inferential interpretations are different.
3. Assuming Bayesian methods are assumption-free. Bayesian analyses require likelihood, prior, and model assumptions.
4. Choosing a prior without justification. Prior selection should be scientifically defensible and, when relevant, supported by sensitivity analysis.
5. Assuming a non-significant p-value proves no effect. Failure to reject the null is not evidence that the true effect is exactly zero.
6. Equating statistical significance with clinical importance. A statistically convincing effect can still be clinically trivial.
7. Ignoring prior sensitivity in a small dataset. When data are weakly informative, the prior can materially affect the posterior.
8. Assuming Bayesian and frequentist estimates must disagree. With large datasets and weakly influential priors, they can be numerically very similar.
9. Using posterior probability without considering the model. Posterior probabilities are conditional on the statistical model and prior.
10. Treating the two frameworks as competing philosophies rather than statistical tools. The best method depends on the scientific and decision-making problem.

What Should Be Reported in a Bayesian Analysis?

A Bayesian statistical analysis should generally document enough information to make the inference reproducible.

Important elements include:

• Primary estimand
• Likelihood/model specification
• Prior distributions
• Justification for prior choices
• Posterior estimation method
• Computational diagnostics
• Posterior summaries
• Credible intervals
• Clinically relevant posterior probabilities
• Posterior predictive assessments when appropriate
• Prior sensitivity analyses
• Model sensitivity analyses

What Should Be Reported in a Frequentist Analysis?

A frequentist analysis should likewise provide sufficient information to reproduce the inferential procedure.

• Primary estimand
• Statistical model
• Null and alternative hypotheses
• Analysis population
• Estimator
• Standard error method
• Confidence interval method
• Hypothesis-testing procedure
• Multiplicity adjustments where applicable
• Missing-data methods
• Sensitivity analyses

The Most Important Concept

The most important distinction between Bayesian and frequentist inference is not that one uses probability and the other does not.

Both use probability.

The key distinction is what probability is being used to describe.

In the standard frequentist framework, probability describes the behavior of data and statistical procedures under repeated sampling when the parameter is treated as fixed.

In the Bayesian framework, probability is also used to represent uncertainty about the unknown parameter itself.

This leads to the fundamental difference between:

and:

The first describes the data given a parameter value. The second describes the parameter given the observed data.

References

Bayes, T. (1763). An Essay towards solving a Problem in the Doctrine of Chances. Philosophical Transactions of the Royal Society of London, 53, 370–418.
Gelman, A., Carlin, J.B., Stern, H.S., Dunson, D.B., Vehtari, A. & Rubin, D.B. (2013). Bayesian Data Analysis, 3rd ed. CRC Press.
Casella, G. & Berger, R.L. (2002). Statistical Inference, 2nd ed. Duxbury.
Berry, D.A. (2006). Bayesian clinical trials. Nature Reviews Drug Discovery, 5, 27–36.
Spiegelhalter, D.J., Abrams, K.R. & Myles, J.P. (2004). Bayesian Approaches to Clinical Trials and Health-Care Evaluation. Wiley.
Wasserstein, R.L. & Lazar, N.A. (2016). The ASA statement on p-values: context, process, and purpose. The American Statistician, 70(2), 129–133.
McElreath, R. (2020). Statistical Rethinking: A Bayesian Course with Examples in R and Stan, 2nd ed. CRC Press.
Gelman, A. & Hill, J. (2006). Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge University Press.