Prior Distributions: Choosing and Justifying Priors

Introduction

A Bayesian analysis begins with a statistical model for the data and a probability distribution describing uncertainty about the unknown parameters before the current data are incorporated.

That probability distribution is the prior distribution.

For a parameter \(\theta\), the prior is commonly written:

After observing data \(D\), Bayesian inference combines the prior with the likelihood:

The result is the posterior distribution.

Choosing the prior is therefore not a cosmetic step in Bayesian analysis. It is part of the statistical model.

What Is a Prior Distribution?

Suppose \(\theta\) is an unknown treatment effect. Before observing the current trial, there is uncertainty about the value of \(\theta\). A Bayesian analysis represents that uncertainty with a probability distribution:

The prior distribution describes how plausible different values of \(\theta\) are before incorporating the current data.

This does not necessarily mean the prior represents personal belief in a subjective or informal sense.

It may instead be based on:

• Previous clinical trials
• Historical control data
• Meta-analyses
• External observational studies
• Pharmacological knowledge
• Biological constraints
• Expert elicitation
• Weakly informative regularization
• Conservative or skeptical assumptions

The Prior Is Part of the Model

A common mistake is to think of the prior as an optional adjustment added after the statistical model has been constructed.

It is more accurate to think of a Bayesian model as having at least two important components:

The likelihood describes how the observed data arise conditional on the parameters.

The prior describes uncertainty about the parameters before incorporating the current data.

Together they determine the posterior.

Bayes' Theorem

For a parameter \(\theta\) and data \(D\):

The numerator combines:

• The likelihood \(L(\theta;D)\)
• The prior \(\pi(\theta)\)

The denominator normalizes the resulting distribution so that it integrates to one.

The central relationship is:

Why Prior Choice Matters

The effect of a prior depends heavily on how much information the current data contain.

If the dataset is extremely informative, reasonable priors may have little effect on the posterior.

If the dataset is small or sparse, the prior may have a much larger influence.

This can be understood conceptually as:

This is not a universal algebraic identity, but it is a useful way to understand why prior strength matters.

A Simple Beta-Binomial Example

Suppose a clinical trial studies a binary response probability \(p\). The likelihood is:

Because \(p\) lies between 0 and 1, a natural prior is a Beta distribution:

The Beta density is:

The prior mean is:

The prior variance is:

The parameters \(\alpha\) and \(\beta\) therefore control both the location and concentration of the prior.

Prior Mean and Prior Strength

Suppose:

The prior mean is:

The quantity:

is often interpreted as the prior's equivalent sample size in the Beta-binomial conjugate setting.

Thus:

can be viewed as approximately 10 prior observations worth of information, with 2 corresponding to prior successes and 8 to prior failures.

Prior Effective Sample Size

Effective sample size is a useful way to communicate how strongly a prior can influence an analysis.

For the Beta prior:

a common conjugate interpretation is:

For example:

Notice that the last three priors have the same prior mean.

They differ substantially in how concentrated they are.

Therefore: Choosing a prior mean is not enough. The strength or precision of the prior must also be justified.

Same Mean, Different Prior Strength

Consider two priors:

and:

Both have mean:

But their concentrations are very different.

The second prior is much more concentrated around 20%.

Consequently, it represents substantially stronger prior information.

Informative Priors

An informative prior contributes substantial information to the posterior.

For example:

expresses relatively strong prior concentration around an effect of 0.20.

Informative priors can be appropriate when high-quality external information exists.

Examples include:

• A well-established historical control response rate
• A validated pharmacological parameter
• Multiple high-quality previous trials
• Strong biological knowledge
• Reliable external registry information

The stronger the prior, the stronger the justification should generally be.

Weakly Informative Priors

A weakly informative prior provides some structure without attempting to dominate the current data.

For example:

might be used for a standardized regression coefficient when very large effects are scientifically implausible but small and moderate effects remain quite plausible.

Weakly informative priors can be particularly useful for:

• Regularizing unstable estimates
• Preventing implausibly extreme parameter values
• Improving computational stability
• Handling sparse data
• Reducing sensitivity to separation in logistic regression

Skeptical Priors

A skeptical prior deliberately places greater probability on small or negligible treatment effects.

Suppose \(\theta\) is the treatment effect and:

The prior is centered on no treatment effect.

This does not mean the analysis assumes that the treatment has no effect.

Rather, it expresses the idea that large effects are considered less plausible before observing the current trial.

Skeptical priors can be useful in sensitivity analyses or when historical evidence suggests that large effects are uncommon.

Optimistic Priors

An optimistic prior places more prior probability on clinically favorable effects.

For example:

might represent an expectation that the treatment is likely to have a positive effect.

Optimistic priors can be scientifically appropriate when supported by substantial external evidence.

However, they should not be chosen merely because the investigator wants a positive result.

Diffuse and Vague Priors

A diffuse prior attempts to spread probability over a broad range of parameter values.

For example:

may appear extremely broad on a scale where plausible treatment effects are close to zero.

However, simply making a prior very broad does not automatically make it "objective" or harmless.

Diffuse priors can place substantial probability on scientifically implausible values.

They can also behave poorly after nonlinear transformations.

Noninformative and Reference Priors

Some Bayesian analyses use priors designed to have minimal influence under specific theoretical criteria.

These are sometimes called reference priors or objective priors, depending on the construction.

Examples include Jeffreys-type priors.

For a Bernoulli probability \(p\), the Jeffreys prior is:

This prior is not literally uniform. The uniform prior is:

The two priors therefore represent different constructions.

Uniform Priors Are Not Always Neutral

Suppose an investigator assigns:

This may look completely neutral.

But consider the transformed parameter:

A uniform prior on \(p\) does not correspond to a uniform prior on \(\operatorname{logit}(p)\).

Thus, "uniform" depends on the scale of the parameter.

Choose the Parameterization Carefully

The prior should be considered on the scale where the scientific interpretation is most meaningful.

For example, in logistic regression:

A prior on \(\beta\) is a prior on the log-odds scale.

If:

then the implied prior distribution on the odds ratio:

is log-normal.

Therefore, prior evaluation should consider both the model parameterization and the scientific scale on which effects will ultimately be interpreted.

Prior Choice Should Begin With the Scientific Question

A useful prior-selection process starts before choosing a distribution.

First ask: What parameter are we trying to describe?

For example:

• Response probability
• Mean treatment difference
• Log hazard ratio
• Log odds ratio
• Regression coefficient
• Variance component
• Between-study heterogeneity
• Pharmacokinetic clearance

The scientific meaning of the parameter should guide the prior.

Step 1: Define a Plausible Range

Suppose \(\theta\) is a treatment effect measured as a difference in response rates. Before selecting a mathematical distribution, identify a plausible range.

For example:

The range might be based on:

• Historical clinical trials
• Clinical expertise
• Biological constraints
• Regulatory precedent
• Published literature
• Mechanistic knowledge

The mathematical prior should then reflect this scientific information.

Step 2: Identify the Center of the Prior

The next question is: Where should the prior distribution be centered?

For example, suppose historical studies suggest an average response rate near 25%.

A prior might therefore be centered around:

But the center alone is insufficient. The analyst must also quantify uncertainty around that value.

Step 3: Quantify Prior Uncertainty

Suppose the prior mean is 0.25. We might believe that values between 0.10 and 0.40 are reasonably plausible.

The prior distribution should place most of its probability within a scientifically defensible range.

This is often more informative than choosing a standard deviation simply because it is mathematically convenient.

Prior Elicitation

Prior elicitation is the process of translating expert knowledge into a probability distribution.

Suppose experts are asked about an unknown treatment effect \(\theta\). Instead of asking: "Choose a standard deviation for a normal distribution." it is often better to ask questions such as:

• What value do you consider most plausible?
• What values would be surprising?
• What value would you regard as clinically important?
• What is the probability that the effect is positive?
• What is the probability that the effect exceeds a specified threshold?

These answers can then be used to construct or calibrate a prior distribution.

Quantile-Based Prior Elicitation

Suppose an expert believes:

and:

This suggests that approximately 95% of prior probability lies between 0.05 and 0.45.

A candidate prior distribution can then be calibrated to match those quantiles.

This is often more transparent than selecting a distribution first and attempting to invent a scientific explanation afterward.

Prior Predictive Distribution

One of the most important tools for evaluating a prior is the prior predictive distribution.

The prior predictive distribution is:

It describes the kinds of datasets that the model and prior predict before observing the actual data.

This is extremely useful because a prior can look reasonable when viewed only on the parameter scale but produce absurd predictions on the observable-data scale.

Prior Predictive Checking for a Binary Endpoint

Suppose:

and:

The prior predictive distribution describes the number of responses we would expect to observe in 50 patients before seeing the actual trial.

If the prior implies that 0 responses and 50 responses are both highly plausible, despite strong clinical knowledge that neither scenario is realistic, the prior may be poorly calibrated.

Prior Predictive Checking in Clinical Trials

Suppose a historical response rate is approximately 20%. An analyst proposes a very broad prior:

The prior itself places substantial probability near both 0 and 1.

That may be inconsistent with the historical clinical context.

A prior predictive simulation may reveal that the model frequently generates trials with extremely high or extremely low response counts.

This does not automatically prove the prior is wrong. It does, however, force the analyst to explain why those predictions are scientifically reasonable.

Prior Predictive Checks vs. Posterior Predictive Checks

Both are valuable, but they answer different questions.

Prior Sensitivity Analysis

A responsible Bayesian analysis should determine whether its conclusions are sensitive to reasonable alternative priors.

Suppose the primary prior is:

The analyst might also examine:

The resulting posterior distributions can then be compared.

The key question is: Would reasonable prior alternatives change the scientific conclusion?

What Counts as a Reasonable Alternative Prior?

Sensitivity analysis should not merely compare the primary prior with absurd alternatives.

Useful alternatives may include:

• A skeptical prior
• A neutral prior
• A moderately informative prior
• A weakly informative prior
• A prior based on a different credible historical source

The purpose is to determine whether conclusions depend on plausible modeling choices.

Worked Example: Response Rate

Suppose a Phase II study enrolls 20 patients and observes 6 responses.

The observed response rate is:

Suppose the historical response rate is approximately 20%. A moderately informative prior could be:

The prior mean is:

The prior effective sample size is approximately:

After observing 6 responses among 20 patients:

The posterior mean is:

The observed proportion was 30%, but the posterior mean is 25% because the historical prior contributes information centered at 20%.

Changing the Prior Strength

Now consider a much weaker prior:

This also has prior mean:

but the effective sample size is only:

The posterior becomes:

with posterior mean:

The prior mean is identical to the previous example, but the posterior is closer to the observed 30% because the prior is weaker.

Historical Data as Prior Information

Historical clinical data are a common source of Bayesian prior information.

Suppose previous studies provide information about a control response rate. A prior can be constructed from those data.

However, historical borrowing should not be automatic. The analyst should consider whether the historical studies are sufficiently comparable with the current study.

Important dimensions include:

• Patient population
• Eligibility criteria
• Disease severity
• Endpoint definition
• Assessment timing
• Standard of care
• Concomitant therapy
• Study conduct
• Geographic region
• Calendar period

Historical Borrowing Can Be Dangerous

Suppose a historical control group has a response rate of 20%. A new trial has a response rate of 35%. If the historical population is systematically different from the new population, blindly borrowing historical information could bias the posterior toward an inappropriate value.

The problem is not that Bayesian methods "use prior information." The problem is that the prior information may not be transportable.

This is why historical-data priors should be accompanied by sensitivity analyses and, when appropriate, models that allow historical information to be discounted.

Exchangeability and Historical Borrowing

Bayesian hierarchical models can explicitly represent uncertainty about whether different studies are sufficiently similar.

For example:

where \(\theta_i\) represents the treatment or control parameter in study \(i\).

The heterogeneity parameter \(\tau\) determines how strongly studies are expected to resemble one another.

If between-study heterogeneity is large, the model naturally permits less information sharing.

This is often preferable to treating historical data as if they were literally observed in the current trial.

Commensurate and Dynamic Borrowing

More sophisticated Bayesian models can allow the amount of borrowing from historical data to depend on how compatible those data are with the current study.

This is sometimes described as dynamic borrowing.

The basic concept is:

while:

Such models can be useful, but their operating characteristics should be carefully evaluated through simulation before being used in a clinical trial.

Prior Choice for Treatment Effects

Suppose \(\theta\) is a treatment difference. A common prior is:

Here:

• \(\mu_0\) is the prior mean.
• \(\sigma_0\) is the prior standard deviation.

For example:

places the prior center at no treatment difference while allowing clinically meaningful positive or negative effects.

If approximately 95% of the prior probability is desired between \(-0.20\) and \(0.20\), then a rough normal approximation is:

This provides a transparent way to connect the standard deviation to a scientifically meaningful range.

Skeptical Priors for Clinical Effects

Suppose large treatment effects are considered unlikely. A skeptical prior might be:

Approximately 95% of the prior probability lies within:

Thus, effects larger than approximately 10 percentage points in magnitude are given relatively low prior probability.

The appropriateness of such a prior depends entirely on the scientific context.

Prior Choice for Odds Ratios

Clinical investigators often interpret logistic regression results using odds ratios.

Because the odds ratio is positive:

it is often convenient to specify a prior on the log odds-ratio:

with:

This implies:

For example, if:

then the prior places substantial probability around odds ratios close to 1 while still allowing meaningful treatment effects.

The prior should always be examined on the odds-ratio scale as well as the log-odds scale.

Prior Choice for Hazard Ratios

The same principle applies to survival analysis.

If:

then a prior such as:

induces a prior distribution for the hazard ratio.

For example, if:

then approximately 95% of \(\beta\) lies between:

which corresponds approximately to:

This makes the scientific meaning of the prior much easier to assess.

Prior Choice for Variance Parameters

Variance parameters require special care because they must be positive.

For example:

Possible priors include:

• Half-normal distributions
• Half-\(t\) distributions
• Log-normal distributions
• Other positive-support distributions

A half-normal prior can be written:

The scale \(s\) should be chosen based on the expected magnitude of the standard deviation on the original measurement scale.

Variance priors should not be selected solely because they are computationally convenient.

Prior Choice for Regression Coefficients

In a linear regression:

one might use:

But the appropriate value of \(\sigma_\beta\) depends heavily on the scale of \(X\) and \(Y\).

A prior that is reasonable when \(X\) is measured in kilograms may be inappropriate when \(X\) is measured in grams.

Standardizing Predictors Can Simplify Prior Specification

Suppose a predictor \(X\) is standardized:

Then a coefficient prior can be specified on a scale where one unit corresponds to approximately one standard deviation of the predictor.

This can make the prior easier to interpret and compare across coefficients.

However, the transformation should be prespecified appropriately and the final results should still be translated back to clinically meaningful units.

Prior Predictive Checking for Regression

Suppose:

This may appear "noninformative."

But if \(X\) ranges from 0 to 100, the implied variation in:

could be enormous.

The resulting prior predictive distribution for \(Y\) might imply impossible values.

Prior predictive simulation would reveal this immediately.

Prior Choice for Logistic Regression

Logistic regression can be especially sensitive to prior specification when data are sparse.

The model is:

With sparse data, maximum likelihood estimates can become extremely large or even fail to exist because of complete or quasi-complete separation.

A weakly informative prior on regression coefficients can regularize the analysis.

For example:

may place meaningful probability on moderate effects while strongly downweighting extreme coefficients.

The exact scale should depend on predictor coding and the scientific context.

Prior Choice in Hierarchical Models

Hierarchical models often contain parameters describing between-group heterogeneity.

For example:

where \(\tau\) describes between-group variability.

A prior for \(\tau\) should reflect how much heterogeneity is plausible.

A common approach is:

or a half-\(t\) distribution.

The choice can have substantial influence when there are few groups.

Prior Choice in Subgroup Analysis

Suppose a trial estimates treatment effects in several subgroups:

A hierarchical prior might allow the subgroup effects to be related:

This can stabilize estimates for small subgroups through partial pooling.

However, if there is strong evidence that treatment effects differ radically across subgroups, aggressive pooling may be inappropriate.

The prior structure should therefore reflect plausible biological and clinical relationships among the populations.

Prior Choice Should Consider the Estimand

The same clinical question can have different estimands.

For example, a treatment effect might be:

• A treatment-policy estimand
• A hypothetical estimand
• A while-on-treatment estimand
• A difference in response probabilities
• A log hazard ratio
• A restricted mean survival-time difference

The prior should be specified for the parameter associated with the actual estimand.

Prior Choice in a Confirmatory Clinical Trial

Suppose a Phase III trial evaluates a treatment effect \(\theta\). A Bayesian confirmatory analysis might specify:

The choice of \(\sigma\) should be justified using relevant external evidence, clinical expertise, or a prespecified weakly informative rationale.

The statistical analysis plan should explain:

• Why the prior was selected
• What external information informed it
• How prior strength was determined
• What prior predictive checks were performed
• What sensitivity analyses were prespecified
• How the posterior will be summarized
• What decision threshold will be used

Prior Choice and Regulatory Transparency

In a regulated clinical development program, prior selection should be particularly transparent.

A reviewer should be able to understand:

• Where the prior information came from
• Why it is relevant to the current population
• How it was translated into a mathematical distribution
• How strongly it influences the analysis
• Whether alternative reasonable priors change the conclusion
• How the operating characteristics of the design were evaluated

Prior Selection Before Looking at the Current Data

One of the most important principles is temporal separation.

The prior should be specified using information that is legitimately available before the current data are used for the analysis.

Using the current trial results to choose a prior and then presenting that prior as if it were genuinely pre-data information can distort the Bayesian analysis.

For example, an analyst should not observe a surprisingly large treatment effect and then select an optimistic prior specifically because it produces a desired posterior.

Data-Dependent Priors

There are specialized Bayesian methods in which prior information can depend on other data.

For example, empirical Bayes methods estimate hyperparameters from observed data.

These methods can be useful, but they should not be casually described as ordinary fixed priors chosen independently of the data.

The inferential properties and interpretation are different.

Robust Priors

Sometimes an analyst wants to use historical information without allowing a small amount of prior-data conflict to dominate the current trial.

A robust prior can combine an informative component with a more diffuse component:

where \(0\le w\le1\).

The robust component provides protection against prior-data conflict.

This approach can be especially useful when historical information is relevant but its transportability is uncertain.

Prior-Data Conflict

A prior-data conflict occurs when the observed data are substantially inconsistent with the prior.

For example, suppose the prior strongly centers a response probability near 20%, but the current trial produces overwhelming evidence for a response rate near 60%.

The analyst should not simply ignore the discrepancy.

Possible explanations include:

• The historical population differs from the current population
• The endpoint changed
• The treatment context changed
• The prior was too strong
• The current trial has an unusual result
• The assumed likelihood is misspecified

Prior Predictive Probability of Extreme Results

A useful diagnostic is to calculate the prior predictive probability of observing results at least as extreme as the actual study.

Conceptually:

where \(T(D)\) is a suitable test statistic or discrepancy measure.

If this probability is extremely small under the prior predictive distribution, the prior/model may be in substantial conflict with the observed data.

Prior Choice Should Be Evaluated on Multiple Scales

A prior can be examined on at least three scales:

1. The model parameter scale
2. The clinically interpretable effect scale
3. The observable-data scale

For example, a prior on a logistic regression coefficient should be examined both as a distribution for \(\beta\) and as a distribution for the odds ratio.

It should also be examined through its implications for predicted response probabilities.

Prior Calibration Using Clinically Important Differences

Suppose the minimum clinically important difference is:

A prior can be evaluated by asking:

and:

before observing the current data.

These probabilities provide a useful language for discussing the prior with clinical experts.

For example: "Before the trial, we believe there is approximately a 20% probability that the true treatment benefit exceeds the clinically important threshold."

This is often much easier for collaborators to evaluate than an unexplained standard deviation.

Prior Probability of Benefit

Suppose the prior is:

If the clinically important threshold is:

then the prior probability of clinically important benefit is:

This probability can be calculated from the normal distribution.

The important point is that the prior can be communicated directly in terms of the clinical decision.

Prior Probability of Harm

The same prior can be used to quantify concern about harm.

For example:

might represent the prior probability that the treatment is meaningfully harmful.

A clinically useful prior therefore does not need to focus exclusively on benefit. It should represent the full range of scientifically plausible outcomes.

Prior Distributions Should Not Be Chosen by Convention Alone

There is no universal rule such as: "Always use a normal prior." or: "Always use a weakly informative prior."

The correct prior depends on:

• The parameter
• The parameterization
• The scientific context
• The amount of external information
• The current sample size
• The likelihood
• The intended decision

A conventional prior can be reasonable, but convention is not a substitute for justification.

Common Prior Distributions

Choosing a Distribution Family

The distribution family should respect the parameter's support.

For example, a normal prior is generally inappropriate for a probability because a normal distribution allows values below 0 and above 1.

Likewise, a standard normal prior is not directly appropriate for a variance because variance parameters must be positive.

This sounds obvious, but prior support becomes particularly important in complex hierarchical models.

Support Is Only the First Requirement

A prior can have the correct mathematical support and still be scientifically poor.

For example, both:

and:

respect the support \(0

But the second prior is far more concentrated around 0.50.

Thus prior selection requires both:

• Correct support
• Scientifically reasonable concentration

Prior Choice for Rare Events

Rare-event endpoints require particular care.

Suppose a safety event probability is expected to be below 1%. A prior centered around 50% would be obviously inappropriate.

Instead, the prior should be calibrated to plausible rare-event probabilities. For example:

with a mean near the expected background rate.

The prior predictive distribution should then be examined to ensure it produces reasonable numbers of events for the planned sample size.

Prior Choice for Count Outcomes

Suppose:

where \(\lambda_i>0\) is a rate. A natural prior might be specified on the log scale:

This guarantees positive values for \(\lambda\).

The prior can be interpreted by transforming it back to the count-rate scale.

Prior Choice for Bioequivalence and PK Parameters

Bayesian priors can also be useful for pharmacokinetic parameters such as clearance, volume of distribution, absorption rate, or interindividual variability.

For a positive parameter such as clearance:

a log-scale model may be appropriate:

The prior can then be calibrated using historical pharmacokinetic studies, mechanistic knowledge, or previous population-PK analyses.

This illustrates a broader principle: Prior specification should respect the biological scale and the natural geometry of the parameter.

Prior Choice in Dose-Response Models

Dose-response models often contain several parameters with different meanings and constraints.

For example, an Emax model can be written:

Here:

• \(E_0\) is the baseline response.
• \(E_{\max}\) is the maximum treatment effect.
• \(ED_{50}\) is the dose producing half of the maximum effect.

Each parameter may require a different prior.

For example, \(ED_{50}\) must be positive and may be naturally modeled on a log scale.

The prior should reflect plausible dose-response behavior rather than simply assigning the same distribution to every parameter.

Prior Choice for Correlations

Correlation parameters must lie between \(-1\) and \(1\).

A simple normal prior on the correlation itself can therefore be problematic.

In multivariate Bayesian models, correlation structures are often modeled using specialized distributions such as the LKJ prior.

For example:

where \(R\) is a correlation matrix and \(\eta\) controls the degree of concentration around particular correlation structures.

The choice of \(\eta\) should be justified in the context of the expected correlation structure.

Prior Choice for Missing Data Models

Bayesian models can incorporate uncertainty about missing-data mechanisms.

For example, parameters governing a missing-not-at-random model may require priors that encode assumptions about departures from missing-at-random behavior.

These priors can be especially influential because the missing-data mechanism may be weakly identified by the observed data.

Prior Choice and Weak Identification

A parameter is weakly identified when the observed data contain limited information about its value.

Examples include:

• Rare events
• Small samples
• Few clusters
• Complex hierarchical models
• Mixture models
• Missing-data sensitivity parameters
• Weakly informed subgroup effects

In these situations, the prior can have a large influence.

That is not necessarily a flaw. It simply means that the data alone do not determine the answer.

Prior Robustness and the Amount of Data

Suppose the same prior is used with two studies. Study A has:

while Study B has:

The prior may have substantial influence in Study A and relatively little influence in Study B.

Therefore, the same prior can be appropriately weak in one setting and effectively informative in another.

Why "Noninformative" Is a Dangerous Goal

Analysts sometimes attempt to remove subjectivity by choosing an extremely broad prior.

But a broad prior can create unintended consequences.

For example, it may:

• Generate implausible predictions
• Produce numerical instability
• Assign too much probability to extreme effects
• Behave differently under reparameterization
• Influence marginal likelihoods and Bayes factors

A more useful goal is often: Choose a prior that is weakly influential where appropriate but scientifically plausible.

Prior Choice for Estimation vs. Model Comparison

An important distinction is that prior sensitivity can be particularly strong for Bayes factors and other marginal-likelihood-based quantities.

The marginal likelihood is:

A diffuse prior spreads probability over a very large parameter space. This can reduce the marginal likelihood even when the posterior estimate is not substantially affected.

Consequently, a prior that is relatively harmless for estimation may still have a major effect on model comparison.

Prior Choice for Decision Thresholds

Suppose the clinical decision rule is:

Then prior selection should be evaluated with this decision rule in mind.

For example, if different reasonable priors lead to:

the decision could change depending on the prior.

That would indicate meaningful prior sensitivity.

If all reasonable priors produce:

then the decision is much more robust.

Prior Sensitivity Should Be Prespecified When Possible

In confirmatory clinical research, prior sensitivity analyses should ideally be defined before examining the final trial results.

The statistical analysis plan might specify:

• Primary prior
• Skeptical prior
• Weakly informative prior
• Alternative historical-data prior
• Decision criterion

This reduces the risk that prior selection becomes an outcome-dependent analysis decision.

How to Justify a Prior in a Statistical Analysis Plan

A strong prior justification should answer at least five questions.

1. What parameter does the prior describe?
2. Where did the prior information come from?
3. Why is that information relevant to the current population?
4. How was the strength of the prior determined?
5. Would reasonable alternative priors change the conclusion?

A weak justification might say:

A stronger justification would describe the scientifically plausible treatment effect range, explain why that range was chosen, show the prior predictive implications, and demonstrate robustness to alternative priors.

An Example of a Strong Prior Justification

Suppose \(\theta\) is a treatment difference in response probability. A statistical analysis plan might state conceptually:

This is much more informative than simply reporting the mathematical distribution.

What a Good Prior Justification Looks Like

Common Mistake: Choosing the Prior After Seeing the Data

Suppose the observed treatment effect is unexpectedly large. The analyst then selects a prior centered on a large positive effect.

This can make the resulting posterior appear more convincing.

But if the prior was chosen because of the observed current data, it is not a genuine pre-data prior in the usual sense.

The analysis should instead distinguish clearly between:

• Pre-data scientific information
• Current observed data
• Any empirical or data-dependent prior estimation procedure

Common Mistake: Treating Historical Data as Literal Current Data

Suppose 100 historical patients are available. It is tempting to say: "The prior is equivalent to adding 100 patients to the current trial."

That is often too simplistic.

Historical observations may differ in quality, relevance, population, endpoint definition, and study conduct.

A Bayesian model can discount or partially pool historical information rather than treating it as perfectly exchangeable with the current trial.

Common Mistake: Using a Prior That Is Too Strong

A prior may be mathematically valid but overly concentrated.

Suppose the current data strongly favor:

but the prior is extremely concentrated around:

The posterior may remain near zero even with meaningful current evidence.

This is not automatically wrong. But it requires strong justification for the prior information.

Common Mistake: Using a Prior That Is Too Weak

The opposite problem can occur. An extremely broad prior may imply:

• Implausibly large effects
• Unrealistic future outcomes
• Numerical instability
• Poorly behaved posterior distributions

A prior should not be made broad merely to avoid having to justify it.

Common Mistake: Ignoring the Prior on the Outcome Scale

An analyst may choose a prior that looks reasonable for \(\beta\).

But after transformation:

the implied odds ratios may be extreme.

Always examine the induced prior on the clinically meaningful scale.

Common Mistake: Assuming a Uniform Prior Is Neutral

A uniform prior:

But uniformity is parameterization-dependent.

A prior that is uniform on \(\theta\) will not generally be uniform on \(\log(\theta)\), \(1/\theta\), or another transformation.

Therefore, the choice should be justified in terms of the scientific parameterization.

Common Mistake: Calling Every Prior "Subjective"

Prior distributions can certainly encode expert judgment.

But they can also be constructed from:

• Published datasets
• Meta-analysis
• Historical controls
• Previous randomized trials
• Mechanistic constraints
• Physical or biological knowledge

The important issue is not whether the prior contains judgment. The important issue is whether the judgment is explicit, reproducible, and defensible.

Common Mistake: Assuming Bayesian Results Are Independent of the Prior

Bayesian inference is conditional on the prior.

The posterior is:

Changing \(\pi(\theta)\) can change the posterior.

The question is not whether the prior has an effect. The question is whether that effect is appropriate and whether the conclusion is robust to reasonable alternatives.

Common Mistake: Performing Sensitivity Analysis With Implausible Priors

A sensitivity analysis should represent genuine uncertainty about prior specification.

Comparing the primary prior with a wildly unrealistic distribution does not provide useful evidence of robustness.

Alternative priors should generally be scientifically plausible.

R Example: Beta Prior

A Beta prior can be specified in R as follows:

alpha_prior <- 2
beta_prior  <- 8

prior_mean <-
  alpha_prior /
  (alpha_prior + beta_prior)

prior_mean
# 0.20

The prior standard deviation is:

prior_sd <-
  sqrt(
    alpha_prior * beta_prior /
    (
      (alpha_prior + beta_prior)^2 *
      (alpha_prior + beta_prior + 1)
    )
  )

prior_sd

Plot the Prior Distribution

The prior should be visualized whenever practical.

p <- seq(0, 1, length.out = 1000)

prior_density <-
  dbeta(
    p,
    shape1 = 2,
    shape2 = 8
  )

plot(
  p,
  prior_density,
  type = "l",
  xlab = "Response probability",
  ylab = "Prior density",
  main = "Beta(2, 8) Prior"
)

The plot makes it immediately clear that the prior centers on lower response probabilities and assigns relatively little probability to values near 1.

Calculate Prior Quantiles

Prior quantiles can communicate uncertainty more intuitively than the Beta parameters alone.

qbeta(
  c(0.025, 0.50, 0.975),
  shape1 = 2,
  shape2 = 8
)

These values provide the lower 2.5th percentile, median, and upper 97.5th percentile of the prior.

This is often useful when documenting prior assumptions.

Calculate Prior Probability of Clinical Benefit

Suppose the clinically important response-rate threshold is 25%. The prior probability that the response rate exceeds this threshold is:

1 - pbeta(
  0.25,
  shape1 = 2,
  shape2 = 8
)

This answers:

before observing the current trial data.

Bayesian Updating in R

Suppose 6 responses are observed among 20 patients.

x <- 6
n <- 20

alpha_post <-
  alpha_prior + x

beta_post <-
  beta_prior + n - x

alpha_post
beta_post

The posterior distribution is:

because:

and:

The posterior mean is:

posterior_mean <-
  alpha_post /
  (alpha_post + beta_post)

posterior_mean
# 0.2667

Calculate a 95% Credible Interval

qbeta(
  c(0.025, 0.975),
  shape1 = alpha_post,
  shape2 = beta_post
)

The resulting values form an equal-tail 95% credible interval for the response probability.

The interpretation is conditional on the specified likelihood and prior: Given the model, prior, and observed data, 95% of the posterior probability lies within the interval.

Prior Sensitivity in R

Suppose we want to compare three priors with the same prior mean but different strength.

priors <- list(
  weak = c(1, 4),
  moderate = c(4, 16),
  strong = c(40, 160)
)

posterior_means <- sapply(
  priors,
  function(prior) {
    a <- prior[1] + 6
    b <- prior[2] + 14

    a / (a + b)
  }
)

posterior_means

All three priors have mean 20%, but their posterior influence differs because their effective sample sizes differ.

Prior Predictive Simulation in R

Prior predictive simulation is often the easiest way to understand what a prior implies.

set.seed(123)

nsim <- 10000
n <- 50

p_prior <-
  rbeta(
    nsim,
    shape1 = 2,
    shape2 = 8
  )

x_prior_predictive <-
  rbinom(
    nsim,
    size = n,
    prob = p_prior
  )

summary(x_prior_predictive)

quantile(
  x_prior_predictive,
  c(0.025, 0.50, 0.975)
)

This simulation asks: Before seeing the trial data, what number of responses does the model think could plausibly occur in 50 patients?

Prior Predictive Simulation for a Treatment Effect

Suppose:

We can simulate treatment effects from the prior:

set.seed(123)

theta_prior <-
  rnorm(
    10000,
    mean = 0,
    sd = 0.10
  )

quantile(
  theta_prior,
  c(0.025, 0.50, 0.975)
)

The resulting quantiles provide a direct summary of the prior's implied plausible treatment-effect range.

A Practical Prior-Selection Workflow

A Practical Checklist for Justifying a Prior

Prior Choice in Clinical Trial Design

Prior distributions can affect not only the final analysis but also the design of a Bayesian clinical trial.

Suppose a trial will declare success when:

The probability of satisfying this criterion depends on:

• The sample size
• The true treatment effect
• The likelihood
• The prior distribution
• The decision threshold

Therefore, the trial's operating characteristics should be evaluated under multiple plausible true parameter values and, when appropriate, multiple plausible priors.

Prior Predictive Operating Characteristics

A Bayesian trial design can be evaluated by simulation. For each simulated trial:

1. Draw a parameter value from the assumed data-generating distribution.
2. Generate trial data conditional on that parameter.
3. Apply the Bayesian analysis.
4. Calculate the posterior probability or decision criterion.
5. Record whether the trial meets the decision rule.

Repeating this process provides an estimate of the design's operating characteristics.

Prior Distribution vs. Sampling Distribution

The prior and sampling distributions answer different questions.

Prior, Likelihood, and Posterior: A Mental Model

A useful conceptual sequence is:

The likelihood determines how strongly different parameter values are supported by the current observations.

The prior determines how plausible those parameter values were before seeing the current observations.

The posterior combines both.

How Much Should the Prior Influence the Result?

There is no universal target such as: "The prior should contribute less than 5%."

The appropriate influence depends on the amount and quality of prior information.

If strong external evidence exists, a strong prior may be appropriate. If little external information exists, a weakly informative prior may be more appropriate.

The goal is not to minimize prior influence.

The goal is to represent prior uncertainty accurately.

Strong Prior Information Is Not a Statistical Error

Suppose decades of evidence establish that a biological parameter is within a narrow range. It would be inappropriate to deliberately use an extremely broad prior simply to appear objective.

Ignoring reliable prior information is itself a modeling decision.

The appropriate question is: How much reliable information exists, and how relevant is it to the current analysis?

When a Weak Prior Is Preferable

A weakly informative prior may be preferable when:

• Little external information exists
• The current trial is adequately powered
• The purpose is primarily to regularize estimation
• Strong prior information could introduce substantial bias
• The parameter is being estimated primarily from current data

Even then, the prior should be checked for reasonable predictive implications.

When an Informative Prior Is Preferable

An informative prior may be preferable when:

• High-quality historical data exist
• The current sample size is small
• The disease population is rare
• Biological information is strong
• Previous studies are highly relevant
• The goal is explicitly to borrow external information

The stronger the external information, the more important it becomes to demonstrate comparability and transportability.

Prior Choice in Rare Disease Trials

Rare disease trials often have limited sample sizes.

A purely data-driven analysis may therefore produce highly uncertain estimates.

Reliable external information may be especially valuable.

Bayesian methods can incorporate such information through:

• Historical controls
• Hierarchical models
• Informative priors
• Robust mixture priors
• Dynamic borrowing

However, the small sample size also means prior assumptions can be highly influential.

Prior Choice in Safety Analyses

Safety endpoints can also be analyzed using Bayesian priors.

For a rare adverse event probability \(p\), the prior should reflect the background event rate and uncertainty around that rate.

If the prior is too optimistic about safety, important adverse-event probabilities may be downweighted.

For this reason, safety priors may appropriately be conservative or skeptical when the cost of underestimating risk is high.

Prior Choice and Asymmetric Consequences

Not all decisions have symmetric consequences. Suppose falsely declaring a treatment effective is much more costly than failing to recognize a modest benefit.

A skeptical prior may be scientifically and operationally appropriate.

Conversely, if the goal is early signal detection for a rare disease with very limited alternatives, a different prior may be reasonable.

The prior should therefore be considered alongside the decision problem.

Prior Choice Is Not the Same as Decision Threshold

These concepts should be kept separate.

The prior describes uncertainty about the parameter before the current data. The decision threshold describes how much posterior evidence is required for a decision.

For example:

contains both a clinical threshold \(\delta\) and a posterior decision threshold of 0.95.

Changing either can change the decision, but they represent different assumptions.

Prior Choice and Clinical Meaning

A statistically convenient prior is not necessarily a clinically meaningful prior.

Whenever possible, prior assumptions should be translated into quantities that clinicians can understand.

For example:

• Expected response rate
• Expected hazard ratio
• Probability of clinically important benefit
• Probability of meaningful harm
• Expected treatment difference
• Expected variability

This makes prior elicitation and review much more transparent.

Common Mistakes

1. Choosing a prior solely by convention. A familiar distribution is not automatically appropriate for the parameter.
2. Reporting only the prior mean. Prior concentration or uncertainty must also be described.
3. Calling an extremely broad prior "noninformative." Broad priors can have important consequences and may generate implausible predictions.
4. Ignoring parameterization. A prior on \(\beta\) induces a different prior on \(e^\beta\).
5. Failing to perform prior predictive checks. A prior can look reasonable on the parameter scale while producing unrealistic data.
6. Using historical data without assessing comparability. Historical information must be relevant to the current population and estimand.
7. Choosing the prior after seeing the current data. This can compromise the intended interpretation of the prior as pre-data information.
8. Ignoring prior sensitivity. Small or weakly identified datasets can be highly sensitive to prior choice.
9. Using implausible alternatives in sensitivity analyses. Sensitivity analyses should examine scientifically reasonable priors.
10. Assuming a prior is equivalent to adding literal observations. Equivalent sample size is model-dependent and should not be overinterpreted.
11. Ignoring the consequences of the prior for decision thresholds. A small change in the posterior can matter greatly when a decision threshold is near the posterior probability.
12. Confusing prior uncertainty with posterior uncertainty. The prior describes pre-data uncertainty; the posterior incorporates the current evidence.

What Should Be Reported for a Bayesian Prior?

A reproducible Bayesian analysis should clearly report:

• The parameter being assigned the prior
• The parameterization
• The prior distribution
• The numerical hyperparameters
• The source of prior information
• The rationale for the prior center
• The rationale for the prior concentration
• The prior predictive assessment
• The sensitivity analyses
• Any historical-data borrowing
• The method used to estimate the posterior

For complex hierarchical models, the complete prior structure should be reported, not merely the prior on the primary treatment effect.

What Should Be Included in a Statistical Analysis Plan?

For a prespecified Bayesian clinical trial analysis, the SAP should ideally describe:

• Primary estimand
• Likelihood
• Prior distributions
• Prior rationale
• Historical-data sources
• Borrowing methodology
• Primary posterior summary
• Credible interval definition
• Decision threshold
• Prior predictive checks
• Prior sensitivity analyses
• Model diagnostics
• Missing-data assumptions
• Simulation-based operating characteristics

Primary Prior vs. Sensitivity Priors

A clinical analysis may specify one prior as the primary analysis prior.

Additional priors can be used for sensitivity analyses.

How to Interpret Prior Sensitivity Results

Suppose three priors produce posterior probabilities:

The decision appears robust.

Now suppose they produce:

The conclusion is highly prior-sensitive.

That does not automatically invalidate the analysis. It means that the available current data do not completely resolve the question independently of prior assumptions.

The Relationship Between Prior and Sample Size

In conjugate models, prior strength can sometimes be described using an equivalent sample size. Suppose:

and the trial observes \(n\) patients. The prior-to-data information ratio can be approximated by:

when the Beta-binomial interpretation is appropriate.

For example, if:

and:

the prior represents roughly one-fifth of the sample size under the conjugate interpretation.

If instead:

the prior and current data have comparable nominal information.

Why Effective Sample Size Is Only an Approximation

For simple conjugate models, equivalent sample size is intuitive.

For a complex hierarchical model, however, there may be no single meaningful number representing prior information.

For example, a prior can constrain:

• A variance component
• A correlation matrix
• A regression coefficient
• A mixture weight

in ways that cannot be represented as a simple number of observations.

Prior Choice and Computational Stability

Priors can also influence computational behavior.

Extremely broad priors may create regions of parameter space with enormous likelihood contributions or weak identification.

This can lead to:

• Divergent transitions in HMC
• Poor mixing
• Slow convergence
• Numerical overflow
• Weakly identified posterior distributions

A scientifically reasonable weakly informative prior can sometimes improve both inference and computation.

Prior Choice and MCMC Diagnostics

When posterior inference is performed using MCMC, prior specification should be considered alongside computational diagnostics.

Important diagnostics can include:

• \(\hat R\)
• Effective sample size
• Trace plots
• Posterior geometry
• Divergent transitions
• Energy diagnostics

Poorly chosen priors can sometimes make the posterior difficult to explore.

However, good computational diagnostics do not prove that the prior is scientifically appropriate.

Prior Predictive Checking Before MCMC

Prior predictive simulation can often be performed before fitting the actual posterior model.

This is valuable because it separates two questions:

1. Is the prior scientifically reasonable?
2. Can the computational algorithm successfully fit the resulting model?

A model should ideally pass both checks.

A Practical Bayesian Prior Workflow for Clinical Research

What Makes a Prior Defensible?

A defensible prior has several characteristics.

• Scientifically plausible: it reflects realistic parameter values.
• Transparent: its source and construction are documented.
• Appropriately scaled: its concentration reflects actual uncertainty.
• Relevant: external information applies to the current estimand and population.
• Predictively reasonable: it produces plausible observable data.
• Robust: reasonable alternatives do not produce unexplained contradictions.
• Prespecified when appropriate: especially for confirmatory analyses.

What Makes a Prior Weak?

A prior is not weak merely because its variance is numerically large.

A prior is better described as weakly informative when it provides limited information relative to the current data while still imposing scientifically reasonable constraints.

This distinction is important.

What Makes a Prior Strong?

A prior is strong when it places substantial probability mass in a relatively narrow region of the parameter space.

This may be appropriate when:

• Reliable external evidence exists
• The parameter is biologically constrained
• Historical evidence is highly relevant
• The current study is small
• The analysis intentionally borrows external information

The important issue is not avoiding strong priors. It is justifying them.

A Final Worked Conceptual Example

Suppose a new treatment has an unknown response probability \(p\). Historical studies suggest approximately 20% response. The investigator believes that 10% to 35% are plausible values, with values outside this range possible but increasingly unlikely.

A prior might be constructed around:

with a concentration calibrated so that the desired prior quantiles match the historical uncertainty.

Before the current trial, the analyst performs prior predictive simulation. The simulations produce plausible numbers of responders.

The analyst then evaluates three priors:

• Primary historical-information prior
• Weakly informative prior
• Skeptical prior

The resulting posterior probabilities of clinically meaningful benefit are similar.

This is a strong analysis because:

• The prior has a scientific basis.
• The uncertainty around the prior is explicit.
• The prior generates plausible data.
• The conclusion is robust to reasonable alternatives.

The Most Important Principle

The most important principle in prior selection is simple:

Do not choose a prior because it produces the result you want. Do not choose an extremely broad prior merely to avoid discussing assumptions. Do not borrow historical information simply because it is available. Do not report only the prior mean.

Instead:

• Define the parameter.
• Understand its scale.
• Identify relevant prior information.
• Quantify uncertainty.
• Check the implied predictions.
• Assess sensitivity.
• Document the rationale.

Prior Selection: A Compact Decision Framework

Bottom Line

References

Gelman, A., Carlin, J.B., Stern, H.S., Dunson, D.B., Vehtari, A. & Rubin, D.B. (2013). Bayesian Data Analysis, 3rd ed. CRC Press.
Gelman, A., Simpson, D. & Betancourt, M. (2017). The prior can often only be understood in the context of the likelihood. Entropy, 19(10), 555.
Gelman, A. (2006). Prior distributions for variance parameters in hierarchical models. Bayesian Analysis, 1(3), 515–534.
McElreath, R. (2020). Statistical Rethinking: A Bayesian Course with Examples in R and Stan, 2nd ed. CRC Press.
Spiegelhalter, D.J., Abrams, K.R. & Myles, J.P. (2004). Bayesian Approaches to Clinical Trials and Health-Care Evaluation. Wiley.
Berry, D.A. (2006). Bayesian clinical trials. Nature Reviews Drug Discovery, 5, 27–36.
Neuenschwander, B., Capkun-Niggli, G., Branson, M. & Spiegelhalter, D.J. (2010). Summarizing historical information on controls in clinical trials. Clinical Trials, 7(1), 5–18.
Hobbs, B.P., Carlin, B.P., Mandrekar, S.J. & Sargent, D.J. (2011). Hierarchical commensurate and power prior models for adaptive incorporation of historical information in clinical trials. Biometrics, 67(3), 1047–1056.
Kass, R.E. & Wasserman, L. (1996). The selection of prior distributions by formal rules. Journal of the American Statistical Association, 91(435), 1343–1370.
Gelman, A. (2006). Prior distributions for variance parameters in hierarchical models. Bayesian Analysis, 1(3), 515–534.