Bayesian Meta-Analysis Methods: A Practical Guide

$$ i=1,\ldots,K $$

$$ (y_1,s_1),\ldots,(y_K,s_K) $$

$$ p(\theta\mid y) \propto p(y\mid\theta)p(\theta) $$

$$ \theta=(\mu,\tau,\theta_1,\ldots,\theta_K) $$

$$ y_i $$

$$ s_i $$

$$ y_i\mid\theta_i \sim N(\theta_i,s_i^2) $$

$$ \mu $$

$$ y_i\mid\mu \sim N(\mu,s_i^2) $$

$$ \theta_i\mid\mu,\tau \sim N(\mu,\tau^2) $$

$$ y_i\mid\theta_i \sim N(\theta_i,s_i^2) $$

$$ \theta_i\mid\mu,\tau \sim N(\mu,\tau^2) $$

$$ \mu\sim N(0,1.5^2) $$

$$ HR=e^\mu $$

$$ \tau>0 $$

$$ \tau\sim\operatorname{HalfNormal}(0,0.5) $$

$$ \mu\sim N(0,1.5^2) $$

$$ HR=e^\mu $$

$$ p(y\mid\theta) = \prod_{i=1}^{K} p(y_i\mid\theta_i) $$

$$ p(y_i\mid\theta_i) = \frac{1}{\sqrt{2\pi s_i^2}} \exp\left[ -\frac{(y_i-\theta_i)^2}{2s_i^2} \right] $$

$$ p(\mu,\tau,\theta_1,\ldots,\theta_K\mid y) \propto p(y\mid\theta_1,\ldots,\theta_K) p(\theta_1,\ldots,\theta_K\mid\mu,\tau) p(\mu) p(\tau) $$

Study	Log HR	SE
Study 1	-0.35	0.18
Study 2	-0.22	0.12
Study 3	-0.41	0.25

Quantity	Meaning
\(s_i\)	Sampling uncertainty within study \(i\)
\(s_i^2\)	Within-study variance
\(\tau\)	Between-study standard deviation
\(\tau^2\)	Between-study variance
\(\mu\)	Mean treatment effect across the study population

$$ P(L<\mu

A direct Bayesian interpretation is therefore possible: conditional on the model and observed data, the posterior probability that $\mu$ lies between $L$ and $U$ is 95%.

Do not confuse a credible interval with a confidence interval. A frequentist confidence interval and a Bayesian credible interval are constructed using different inferential frameworks and do not have the same probabilistic interpretation.

Posterior Probability of Benefit

One of the most intuitive Bayesian quantities is:

P(\mu<0\mid y)

when negative values represent benefit, as they do for a log hazard ratio.

For example, a posterior probability of:

P(\mu<0\mid y)=0.97

means that 97% of the posterior probability lies below zero.

A more clinically meaningful threshold can also be used. Suppose a hazard ratio below 0.80 is considered clinically important. On the log scale:

\log(0.80)\approx-0.223

The posterior probability of clinically important benefit is then:

P(\mu<\log(0.80)\mid y)

A Worked Bayesian Meta-Analysis Example

Consider five hypothetical randomized clinical trials evaluating the same treatment. The primary effect measure is the log hazard ratio. Negative values favor treatment.

Study	Log HR	SE	Approx. HR
Study 1	-0.35	0.18	0.70
Study 2	-0.22	0.12	0.80
Study 3	-0.41	0.25	0.66
Study 4	-0.10	0.20	0.90
Study 5	-0.28	0.15	0.76

The studies generally favor treatment, but their estimates are not identical. The Bayesian question is: What does the combined evidence imply about the overall treatment effect and the degree of between-study heterogeneity?

Specify the Sampling Model

For each study:

y_i\mid\theta_i \sim N(\theta_i,s_i^2)

For this example:

\begin{aligned} y_1&=-0.35,\quad s_1=0.18\\ y_2&=-0.22,\quad s_2=0.12\\ y_3&=-0.41,\quad s_3=0.25\\ y_4&=-0.10,\quad s_4=0.20\\ y_5&=-0.28,\quad s_5=0.15 \end{aligned}

Specify the Random-Effects Model

Assume:

\theta_i\mid\mu,\tau \sim N(\mu,\tau^2)

This allows the underlying treatment effects to vary across studies.

Specify the Priors

For illustration, suppose:

\mu\sim N(0,1.5^2)

and:

\tau\sim\operatorname{HalfNormal}(0,0.5)

These are example weakly informative priors, not universal defaults. The appropriate prior scale depends on the effect measure and scientific context.

Understanding Partial Pooling

One of the most important features of hierarchical Bayesian meta-analysis is partial pooling.

The model does not necessarily treat every study estimate as completely independent when estimating its underlying effect. Instead, each study-specific effect is informed by:

Its own observed data
The overall mean effect
The estimated between-study heterogeneity

A study with a very large standard error may therefore be pulled more strongly toward the overall mean.

This is commonly called shrinkage.

Why Shrinkage Occurs

Consider a study with a highly imprecise estimate. If:

s_i\gg\tau

the study provides relatively little information about its own underlying effect compared with the information available from the hierarchical distribution.

The posterior estimate of $\theta_i$ will therefore tend to be closer to $\mu$.

Conversely, a highly precise study with:

s_i\ll\tau

will generally receive more weight from its own data.

Clinical interpretation: Shrinkage is not simply a computational trick. It reflects the assumption that the studies are related members of a broader population of studies rather than completely unrelated experiments.

Marginalizing Over the Study Effects

The hierarchical model can be expressed without explicitly sampling every $\theta_i$ by integrating over the study-specific effects. Because both levels are normal:

y_i\mid\mu,\tau \sim N(\mu,s_i^2+\tau^2)

This equation is particularly important. It shows that the variance of each observed study estimate consists of two components:

s_i^2+\tau^2

The within-study variance and between-study variance are added because they represent distinct sources of variation.

The Posterior for the Overall Effect

Conditional on a fixed value of $\tau$, the model resembles a weighted normal meta-analysis. The study weights are:

w_i= \frac{1}{s_i^2+\tau^2}

The corresponding weighted estimate is:

\hat{\mu}(\tau) = \frac{ \sum_{i=1}^{K}w_i y_i }{ \sum_{i=1}^{K}w_i }

The key Bayesian difference is that $\tau$ is itself uncertain. Rather than simply plugging in one estimate of heterogeneity, the Bayesian model integrates over the posterior uncertainty in $\tau$.

Posterior Distribution of Heterogeneity

The parameter $\tau$ is usually one of the most interesting quantities in a random-effects meta-analysis.

Its posterior distribution answers questions such as:

How much variation exists across studies?
How uncertain is that heterogeneity estimate?
What values of $\tau$ remain plausible after observing the data?

A posterior median might be reported together with a credible interval. For example, an analysis might produce a result such as:

\operatorname{median}(\tau)=0.15

with a wide 95% credible interval. The exact numerical result depends on the data and priors.

Do not overinterpret a small estimate of heterogeneity. With few studies, the posterior for $\tau$ may be broad even when the posterior median is near zero. “Low estimated heterogeneity” is not necessarily equivalent to “heterogeneity has been ruled out.”

The Prediction Problem

The overall mean effect $\mu$ and the effect expected in a new study are not the same quantity.

The overall mean describes:

\mu

whereas a future study's underlying effect is:

\theta_{\text{new}} \sim N(\mu,\tau^2)

Conditional on $\mu$ and $\tau$, the distribution of the effect in a new study is therefore:

\theta_{\text{new}}\mid\mu,\tau \sim N(\mu,\tau^2)

The posterior predictive distribution integrates over uncertainty in both $\mu$ and $\tau$.

Why Prediction Intervals Can Be Wider

A credible interval for $\mu$ describes uncertainty about the average effect. A predictive interval for a new study additionally incorporates genuine between-study variation.

Therefore, when $\tau$ is appreciable, the predictive interval can be much wider than the credible interval for $\mu$.

Quantity	Question
Posterior for $\mu$	What is the average treatment effect?
Posterior for $\tau$	How much do treatment effects vary across studies?
Posterior predictive distribution	What treatment effect might occur in a new study?

Probability of Benefit in a Future Study

Suppose negative log hazard ratios favor treatment. The posterior probability that a new study will have a beneficial underlying effect is:

P(\theta_{\text{new}}<0\mid y)

This can be much more informative than simply reporting the posterior probability that the average effect is beneficial.

For example, the overall evidence might strongly support:

P(\mu<0\mid y)

while substantial heterogeneity could make:

P(\theta_{\text{new}}<0\mid y)

considerably smaller.

Fixed-Effect vs. Random-Effects Bayesian Meta-Analysis

Feature	Fixed Effect	Random Effects
Underlying effects	Common $\mu$	Study-specific $\theta_i$
Between-study heterogeneity	Not modeled	Modeled through $\tau$
Future-study prediction	Not naturally represented	Directly represented
Model complexity	Lower	Higher
Potential prior sensitivity	Usually lower	Often greater, especially for $\tau$

The choice should follow the scientific question. If all studies are viewed as repeated measurements of exactly one common effect, a fixed-effect model may be appropriate. If the studies represent a population of clinically related settings with potentially different underlying effects, a random-effects model is often more natural.

What Does "Heterogeneity" Actually Mean?

Heterogeneity is not merely statistical disagreement among observed estimates. It can arise because the underlying treatment effect genuinely varies across studies. Potential causes include:

Different patient populations
Different disease severity
Different comparator treatments
Different treatment doses
Different follow-up periods
Different endpoint definitions
Differences in background therapy
Differences in study conduct

A Bayesian random-effects model represents this variation statistically, but it does not explain its clinical cause.

Meta-Regression

If study-level characteristics are available, the hierarchical model can be extended. Suppose $x_i$ is a study-level covariate such as:

Median age
Baseline disease severity
Dose
Follow-up duration
Proportion receiving background therapy

A meta-regression model might be:

\theta_i = \beta_0+\beta_1x_i+u_i

where:

u_i\sim N(0,\tau^2)

The coefficient $\beta_1$ represents the expected change in the underlying study effect associated with a one-unit increase in $x_i$.

Bayesian Meta-Regression

The Bayesian framework allows priors to be assigned to regression coefficients:

\beta_0\sim N(0,\sigma_0^2)

and:

\beta_1\sim N(0,\sigma_1^2)

The posterior then combines information from the studies with these prior distributions.

Caution: Meta-regression uses study-level covariates. Associations observed at the study level should not automatically be interpreted as individual-patient causal relationships.

Bayesian Meta-Analysis of Binary Outcomes

The normal model for log hazard ratios is convenient, but Bayesian meta-analysis is not restricted to approximately normal effect estimates. Suppose each study reports:

$r_{iT}$ responses among $n_{iT}$ treated patients
$r_{iC}$ responses among $n_{iC}$ control patients

A binomial likelihood can be specified directly:

r_{iT}\sim\operatorname{Binomial}(n_{iT},p_{iT})

and:

r_{iC}\sim\operatorname{Binomial}(n_{iC},p_{iC})

The treatment effect could then be represented on the log-odds scale:

\theta_i = \operatorname{logit}(p_{iT}) - \operatorname{logit}(p_{iC})

This is sometimes preferable to first calculating an approximate study-level log odds ratio and standard error.

Bayesian Meta-Analysis of Hazard Ratios

For time-to-event outcomes, published studies frequently provide estimated hazard ratios and standard errors. A common approximate model is:

\log(HR_i) \sim N(\theta_i,s_i^2)

with:

\theta_i\sim N(\mu,\tau^2)

After estimating the posterior for $\mu$, the overall hazard ratio is obtained by exponentiation:

HR_{\text{overall}}=e^\mu

Bayesian Meta-Analysis of Mean Differences

For continuous outcomes, a study may report a mean difference and standard error. The same framework can be used:

y_i\mid\theta_i \sim N(\theta_i,s_i^2)

and:

\theta_i\sim N(\mu,\tau^2)

In this setting, $\mu$ remains on the original measurement scale. For example, $\mu=3.2$ could represent an average treatment effect of 3.2 points on a clinical score.

Standardized Mean Differences

When studies use different measurement scales for the same underlying construct, standardized mean differences may be used. The effect is commonly expressed as a standardized difference such as:

d_i = \frac{\bar X_{iT}-\bar X_{iC}}{S_i}

The resulting study-specific estimates can then be incorporated into a hierarchical Bayesian model, provided their sampling distributions and standard errors are appropriately represented.

Prior Sensitivity Analysis

A Bayesian meta-analysis should generally examine whether substantive conclusions depend strongly on reasonable prior choices.

For example, consider three priors for $\tau$:

Prior	Purpose
Weakly informative	Allows a broad range of plausible heterogeneity
More concentrated	Represents stronger prior belief that heterogeneity is modest
Diffuse	Allows substantially larger heterogeneity

The analysis can be repeated under each prior. If the posterior conclusions remain similar, the substantive inference is less sensitive to that modeling choice.

Why Prior Sensitivity Matters for $\tau$

The heterogeneity parameter is often weakly identified when the number of studies is small. For example, a meta-analysis containing only five studies provides much less information about $\tau$ than one containing fifty studies.

As a result, the prior can influence:

The posterior distribution of $\tau$
The posterior distribution of $\mu$
The predictive distribution for a new study

Sensitivity analysis is therefore particularly important for random-effects Bayesian meta-analysis with few studies.

Forest Plots in Bayesian Meta-Analysis

A forest plot remains useful in a Bayesian meta-analysis. It can display:

Observed study-specific effect estimates
Study-level uncertainty intervals
Posterior estimates of study-specific effects
The posterior estimate of the overall effect
A predictive interval for a new study

The key difference is that the intervals shown can represent posterior credible intervals rather than frequentist confidence intervals.

Bayesian Evidence Synthesis Is More Than a Weighted Average

Traditional inverse-variance meta-analysis can be represented as a weighted average of study estimates. Bayesian hierarchical modeling goes further. The model simultaneously estimates:

The overall effect
Study-specific effects
Between-study heterogeneity
Uncertainty in all of these quantities

The resulting posterior distribution therefore represents uncertainty across the entire hierarchy.

Markov Chain Monte Carlo

Many Bayesian meta-analysis models do not have simple closed-form posterior distributions. Markov Chain Monte Carlo, or MCMC, provides a general method for sampling from the posterior distribution.

Conceptually, MCMC produces a sequence:

\theta^{(1)},\theta^{(2)},\ldots,\theta^{(M)}

where the draws approximate the posterior distribution after appropriate convergence and warm-up procedures.

Posterior quantities can then be estimated from these draws.

Posterior Probability From MCMC Draws

Suppose MCMC produces $M$ posterior draws of $\mu$. The posterior probability that the treatment effect is beneficial can be estimated as:

P(\mu<0\mid y) \approx \frac{1}{M} \sum_{m=1}^{M} I(\mu^{(m)}<0)

where $I(\cdot)$ is an indicator function.

For example, if 9,700 of 10,000 posterior draws satisfy $\mu<0$, the estimated posterior probability of benefit is:

\frac{9700}{10000}=0.97

Posterior Predictive Simulation

A posterior predictive distribution can also be generated from MCMC draws. For each posterior draw:

Draw $\mu^{(m)}$ and $\tau^{(m)}$ from the posterior.
Generate a future study effect:

\theta_{\text{new}}^{(m)} \sim N(\mu^{(m)},(\tau^{(m)})^2)

The collection of simulated future effects forms the posterior predictive distribution.

Convergence Diagnostics

MCMC output should not be accepted simply because the software produced a posterior summary. Important diagnostics include:

Trace plots
R-hat
Effective sample size
Autocorrelation
Monte Carlo standard error

A commonly used diagnostic is the split-$\hat R$ statistic. Values close to 1 indicate that the chains are mixing consistently, although no single diagnostic should be interpreted in isolation.

Important: Convergence diagnostics address whether the computational algorithm has adequately explored the posterior distribution. They do not determine whether the statistical model itself is scientifically appropriate.

R-Hat

Suppose several independent MCMC chains are run. R-hat compares within-chain and between-chain variation. Conceptually:

\hat R \approx \sqrt{ \frac{\text{estimated marginal variance}} {\text{within-chain variance}} }

For a well-mixed parameter, $\hat R$ should be very close to 1.

Effective Sample Size

MCMC draws are generally autocorrelated. Therefore, 10,000 MCMC iterations do not necessarily contain the same amount of information as 10,000 independent random draws. The effective sample size accounts for this dependence.

A low effective sample size indicates that additional sampling may be needed for precise Monte Carlo estimation.

Bayesian vs. Frequentist Meta-Analysis

Feature	Frequentist	Bayesian
Unknown parameters	Fixed but unknown	Assigned probability distributions
Prior information	Not part of standard inference	Explicitly modeled
Point estimate	Estimator	Posterior summary
Interval	Confidence interval	Credible interval
Probability of benefit	Not directly a parameter probability	Direct posterior probability
Heterogeneity	Estimated using frequentist methods	Has a posterior distribution
Prediction	Requires predictive framework	Posterior predictive distribution

The Bayesian framework is particularly useful when investigators want probabilistic statements about unknown parameters or wish to incorporate external evidence through explicit prior distributions.

Bayesian Meta-Analysis With Historical Information

One potential advantage of Bayesian methods is the ability to incorporate relevant external information. Suppose previous studies provide strong evidence about a treatment effect. A prior could be constructed to represent that evidence:

\mu\sim N(\mu_0,\sigma_0^2)

The new studies then update that prior through the likelihood.

However, external information must be sufficiently relevant to justify borrowing. Differences in:

Patient population
Endpoint definition
Comparator
Treatment regimen
Study quality
Calendar time

can make apparently similar studies scientifically different.

Dynamic Borrowing

More sophisticated Bayesian evidence-synthesis models can allow the amount of borrowing from historical evidence to depend on compatibility between the historical and current data.

Conceptually:

\text{More compatible data} \Rightarrow \text{more borrowing}

while:

\text{Less compatible data} \Rightarrow \text{less borrowing}

This is sometimes called dynamic borrowing. It can be useful in settings such as historical-control augmentation, but it requires careful model specification and sensitivity analysis.

Publication Bias

Meta-analysis combines available studies, but the available studies may not represent all studies that were conducted. Studies with unfavorable or statistically nonsignificant results may be less likely to be published or identified.

A Bayesian hierarchical model does not automatically eliminate publication bias. Potential approaches include:

Selection models
Pattern-mixture models
Copas-type approaches
Explicit models for study availability
Sensitivity analyses

These approaches require additional assumptions beyond the basic random-effects model.

Small-Study Effects

Smaller studies may produce systematically different results from larger studies. Potential explanations include:

Publication bias
Differences in study populations
Differences in study quality
Random variation
Other methodological differences

A Bayesian model can incorporate covariates or selection mechanisms, but the presence of a sophisticated Bayesian model does not itself solve these issues.

Missing Data

Missing study-level information can also affect a meta-analysis. Examples include missing standard errors or incomplete reporting of outcomes.

The appropriate Bayesian approach depends on what is missing and why. A model may jointly represent:

The observed outcome
The missing quantity
The mechanism generating missingness

This can propagate uncertainty about missing information into the final posterior distribution.

Bayesian Network Meta-Analysis

When multiple treatments are compared across a network of trials, Bayesian methods can be extended to network meta-analysis. Suppose treatments $A$, $B$, and $C$ are available. Some trials may compare:

$A$ vs. $B$
$B$ vs. $C$
but not $A$ vs. $C$

A network model can combine direct and indirect evidence.

For example:

\delta_{AC} = \delta_{AB}+\delta_{BC}

under an appropriate consistency framework.

Important: Network meta-analysis requires additional assumptions, particularly regarding comparability and consistency of treatment effects across the network.

Bayesian Meta-Analysis With Multiple Endpoints

Clinical evidence syntheses sometimes involve several related outcomes. For example:

Overall survival
Progression-free survival
Objective response
Quality of life

A multivariate Bayesian model can allow correlations between treatment effects across endpoints.

For a vector of study effects:

\boldsymbol{\theta}_i \sim N(\boldsymbol{\mu},\Sigma)

where $\Sigma$ represents the covariance structure among endpoints.

This can improve efficiency when the endpoints are genuinely related, but it also introduces additional modeling assumptions.

Clinical Interpretation of the Overall Effect

Suppose the posterior distribution for the overall log hazard ratio is:

\mu\mid y \sim \text{posterior distribution}

The most useful clinical summaries may include:

Posterior median HR
95% credible interval for HR
Posterior probability HR < 1
Posterior probability HR < a clinically meaningful threshold
Posterior predictive interval for a future study

For example, an analysis could report:

Illustrative interpretation: The posterior median hazard ratio was 0.76, with a 95% credible interval of 0.62 to 0.93. The posterior probability that the hazard ratio was below 1 was approximately 99%, while the predictive distribution for a future study was wider because of between-study heterogeneity.

The numerical values above are illustrative rather than results from the five-study example.

R Implementation With brms

One convenient way to fit hierarchical Bayesian meta-analysis models in R is with the brms package. The package provides an interface to Stan and allows Bayesian hierarchical models to be specified using familiar R formulas.

library(brms)

dat <- data.frame(
  study = paste0("Study_", 1:5),
  yi = c(-0.35, -0.22, -0.41, -0.10, -0.28),
  sei = c(0.18, 0.12, 0.25, 0.20, 0.15)
)

Specify the Bayesian Random-Effects Model

A meta-analysis can be represented by giving each study estimate its known standard error.

fit <- brm(
  yi | se(sei, sigma = TRUE) ~ 1 + (1 | study),
  data = dat,
  family = gaussian(),
  prior = c(
    prior(normal(0, 1.5), class = "Intercept"),
    prior(normal(0, 0.5), class = "sd", group = "study")
  ),
  chains = 4,
  iter = 4000,
  warmup = 2000,
  seed = 1234
)

The exact prior specification should be reviewed carefully for the scale and scientific context of the analysis.

Inspect the Posterior

summary(fit)

The output provides posterior summaries for the overall effect and the between-study standard deviation.

Transform the Overall Effect to a Hazard Ratio

If the modeled effect is a log hazard ratio, exponentiate the posterior draws.

draws <- as_draws_df(fit)

draws$HR <- exp(draws$b_Intercept)

quantile(
  draws$HR,
  probs = c(0.025, 0.50, 0.975)
)

This gives posterior quantiles for the overall hazard ratio.

Calculate Posterior Probability of Benefit

If an HR below 1 represents benefit:

mean(draws$HR < 1)

This estimates:

P(HR<1\mid y)

A clinically meaningful threshold can be substituted. For example, if HR < 0.80 represents meaningful benefit:

mean(draws$HR < 0.80)

This estimates:

P(HR<0.80\mid y)

Inspect MCMC Diagnostics

plot(fit)

summary(fit)

The trace plots should show good mixing without obvious trends or chain separation. The reported R-hat values should generally be close to 1, and effective sample sizes should be adequate for the posterior quantities being reported.

Posterior Predictive Checks

A Bayesian model should be checked not only for MCMC convergence but also for whether it can reproduce important features of the observed data.

pp_check(fit)

Posterior predictive checks compare observed data with data simulated from the fitted model.

Large systematic discrepancies may indicate that the likelihood or hierarchical structure is inadequate.

Posterior Predictive Distribution for a New Study

The posterior predictive distribution incorporates uncertainty about both the overall effect and heterogeneity. Conceptually:

\theta_{\text{new}} \sim N(\mu,\tau^2)

In software, posterior predictive draws can be generated and transformed to the clinical effect scale.

pred <- posterior_predict(fit)

quantile(
  pred,
  probs = c(0.025, 0.50, 0.975)
)

For a meta-analysis of study-level normal effects, it is important to distinguish prediction of a future observed study estimate from prediction of its underlying true treatment effect. The former also includes the future study's sampling error.

A More Explicit Stan Model

For statisticians who want complete control over the hierarchical structure, the model can be written directly in Stan. The conceptual model is:

y_i\sim N(\theta_i,s_i)

\theta_i\sim N(\mu,\tau)

\mu\sim N(0,1.5)

\tau\sim\operatorname{HalfNormal}(0,0.5)

The advantage of direct Stan modeling is flexibility. For example, the model can be extended to include:

Meta-regression
Multiple outcomes
Correlated effects
Non-normal likelihoods
Mixture models
Historical borrowing

Non-Centered Parameterization

Hierarchical models can sometimes produce difficult posterior geometries. A non-centered parameterization can improve computational efficiency. Instead of directly sampling:

\theta_i\sim N(\mu,\tau^2)

write:

z_i\sim N(0,1)

and:

\theta_i=\mu+\tau z_i

This parameterization is particularly useful when $\tau$ is weakly identified.

Computational insight: A statistically correct hierarchical model can still sample poorly if it is parameterized inefficiently. Non-centered parameterizations are one of the standard tools for improving MCMC behavior in hierarchical models.

Checking Sensitivity to the Prior on $\tau$

Suppose the primary model uses:

\tau\sim\operatorname{HalfNormal}(0,0.5)

A sensitivity analysis might repeat the model using a broader prior. For example:

\tau\sim\operatorname{HalfNormal}(0,1)

The posterior summaries can then be compared.

# Example sensitivity model

fit_sensitive <- brm(
  yi | se(sei, sigma = TRUE) ~ 1 + (1 | study),
  data = dat,
  family = gaussian(),
  prior = c(
    prior(normal(0, 1.5), class = "Intercept"),
    prior(normal(0, 1), class = "sd", group = "study")
  ),
  chains = 4,
  iter = 4000,
  warmup = 2000,
  seed = 1234
)

The goal is not necessarily to find one “correct” prior. The goal is to understand whether the scientific conclusion is robust to reasonable prior assumptions.

Comparing Bayesian Models

Several candidate models might be considered:

Model	Structure
Fixed effect	$\theta_i=\mu$
Random effects	$\theta_i\sim N(\mu,\tau^2)$
Meta-regression	$\theta_i=\mathbf{x}_i^\top\beta+u_i$
Multivariate	$\boldsymbol{\theta}_i\sim N(\boldsymbol{\mu},\Sigma)$
Network meta-analysis	Multiple interconnected treatment effects

The model should be selected based on the scientific question, data-generating process, and plausibility of its assumptions.

Model Checking vs. Model Selection

These are related but different activities. Model checking asks whether the fitted model can reasonably reproduce important aspects of the observed data.

Model selection asks whether one candidate model provides a preferable representation according to a prespecified or explicitly justified criterion.

Bayesian tools can be used for both, but neither should replace substantive clinical reasoning.

Common Mistakes

Treating the overall effect as if every study has the same true effect. A random-effects model explicitly allows study-specific treatment effects to vary.
Confusing $s_i$ with $\tau$. The standard error describes sampling uncertainty within a study; $\tau$ describes variation between underlying study effects.
Assuming a random-effects model automatically solves heterogeneity. The model quantifies heterogeneity but does not explain its clinical source.
Ignoring prior sensitivity. The posterior for $\tau$ can be sensitive to the prior when the number of studies is small.
Reporting only the posterior mean. The full posterior distribution contains substantially more information.
Confusing credible and confidence intervals. They have different interpretations.
Ignoring the predictive distribution. The average treatment effect and the effect in a future study can differ when heterogeneity is present.
Running MCMC without checking convergence. Posterior summaries are not reliable if the sampler has not adequately explored the posterior.
Using vague priors automatically. A mathematically diffuse prior can imply surprisingly strong or implausible beliefs on a transformed clinical scale.
Assuming Bayesian models are automatically objective. Prior distributions and structural assumptions remain modeling choices.
Ignoring publication bias. A sophisticated hierarchical model cannot automatically recover studies that were never observed.
Overinterpreting study-level covariates. Meta-regression associations are ecological relationships and do not necessarily represent individual-level effects.

A Practical Bayesian Meta-Analysis Workflow

1

Define the clinical question and estimand.

2

Define the treatment effect measure and its analysis scale.

3

Extract study-level estimates and their standard errors, or obtain the underlying outcome data.

4

Assess whether a fixed-effect or random-effects structure is scientifically appropriate.

5

Specify clinically defensible prior distributions.

6

Perform prior predictive checks where appropriate.

7

Fit the Bayesian model using MCMC or another appropriate computational method.

8

Evaluate convergence diagnostics and effective sample sizes.

9

Perform posterior predictive checks.

10

Summarize the posterior distribution of the overall effect.

11

Quantify posterior uncertainty about heterogeneity.

12

Evaluate clinically meaningful posterior and predictive probabilities.

13

Perform prior and model sensitivity analyses.

14

Report assumptions, priors, diagnostics, and posterior results transparently.

What Should Be Reported in a Statistical Analysis Plan?

A Bayesian meta-analysis should be sufficiently specified that another statistician could reproduce the analysis. At minimum, document:

Primary estimand
Effect measure
Analysis scale
Study inclusion criteria
Sampling model
Fixed-effect or random-effects structure
Prior distributions
Prior parameterization
Heterogeneity model
Any meta-regression covariates
MCMC algorithm and settings
Number of chains
Warm-up iterations
Sampling iterations
Convergence diagnostics
Posterior summaries
Credible interval definition
Posterior predictive analyses
Sensitivity analyses
Handling of missing data
Assessment of publication bias where relevant

How to Report a Bayesian Meta-Analysis

A concise report might state:

Illustrative reporting: A Bayesian random-effects meta-analysis was performed using a normal likelihood for the study-level log hazard ratios. Study-specific effects were modeled as normally distributed around an overall mean effect, with a half-normal prior assigned to the between-study standard deviation. The overall treatment effect was summarized using its posterior median and 95% credible interval. Posterior probabilities for clinically meaningful benefit and posterior predictive quantities were also calculated. MCMC convergence was assessed using trace plots, R-hat, and effective sample size, and sensitivity analyses were performed using alternative priors for between-study heterogeneity.

Interpreting a Bayesian Meta-Analysis Clinically

The statistical output should ultimately answer a clinical question. For example:

What is the estimated average treatment effect?
How uncertain is that estimate?
How much do treatment effects vary across studies?
What is the probability of a clinically meaningful benefit?
How likely is benefit in a future study?
How sensitive are these conclusions to prior assumptions?

These questions are generally more useful than simply asking whether a single null hypothesis was rejected.

The Most Important Distinction: Mean Effect vs. Future Effect

Perhaps the most important conceptual distinction in random-effects Bayesian meta-analysis is between:

\mu

and:

\theta_{\text{new}}

The first is the average underlying treatment effect across the modeled population of studies. The second is the underlying treatment effect in a future study.

When $\tau>0$, they are not the same quantity.

A treatment can therefore have a strongly beneficial posterior mean effect while still having substantial uncertainty about whether the treatment will produce a beneficial effect in every future population.

Bottom line: A Bayesian random-effects meta-analysis estimates not only the average treatment effect but also how treatment effects vary across studies. This makes the posterior predictive distribution an especially important component of clinical interpretation.

Worked Example Summary

Component	Bayesian Meta-Analysis
Observed study effect	$y_i$
Within-study SE	$s_i$
Study-specific effect	$\theta_i$
Overall effect	$\mu$
Between-study SD	$\tau$
Sampling model	$y_i\mid\theta_i\sim N(\theta_i,s_i^2)$
Hierarchical model	$\theta_i\mid\mu,\tau\sim N(\mu,\tau^2)$
Prior for overall effect	Example: $\mu\sim N(0,1.5^2)$
Prior for heterogeneity	Example: $\tau\sim\operatorname{HalfNormal}(0,0.5)$
Primary posterior quantity	$p(\mu\mid y)$
Heterogeneity posterior	$p(\tau\mid y)$
Future-study quantity	$p(\theta_{\text{new}}\mid y)$
Computational method	MCMC

The Most Important Concepts to Remember

There are several ideas that should become automatic when working with Bayesian meta-analysis.

The study estimate is not the true study effect. The observed estimate $y_i$ contains sampling error.
The overall effect is not necessarily the effect in every study. Random-effects models allow study-specific effects to vary around $\mu$.
$\tau$ represents between-study heterogeneity. It is distinct from the standard error $s_i$.
The posterior combines prior information with the observed studies.
Credible intervals describe posterior probability. They are not confidence intervals with Bayesian terminology substituted.
Posterior probability can answer clinically meaningful questions. For example, $P(HR<0.80\mid y)$.
The predictive distribution describes future studies. It incorporates uncertainty about both the overall effect and heterogeneity.
Prior sensitivity matters. This is especially important when few studies inform heterogeneity.
MCMC diagnostics are essential. A posterior summary is only useful if the computational approximation is adequate.
Bayesian modeling does not eliminate scientific assumptions. It makes many of them explicit.

Final takeaway: Bayesian meta-analysis is a hierarchical framework for combining evidence across clinical studies while propagating uncertainty about treatment effects and between-study heterogeneity. The basic random-effects model separates within-study sampling error from genuine variation among study effects. Priors are assigned to unknown quantities such as the overall effect and heterogeneity, and Bayes' theorem produces a posterior distribution. From that posterior, investigators can estimate the overall treatment effect, calculate credible intervals and clinically meaningful probabilities, quantify heterogeneity, and predict treatment effects in future studies. For clinical evidence synthesis, the combination of hierarchical modeling, explicit uncertainty, and posterior prediction makes Bayesian meta-analysis a powerful extension of conventional meta-analytic methods.

References

Gelman, A., Carlin, J.B., Stern, H.S., Dunson, D.B., Vehtari, A. & Rubin, D.B. (2013). Bayesian Data Analysis, 3rd ed. CRC Press.

Borenstein, M., Hedges, L.V., Higgins, J.P.T. & Rothstein, H.R. (2009). Introduction to Meta-Analysis. Wiley.

Higgins, J.P.T., Thompson, S.G., Deeks, J.J. & Altman, D.G. (2003). Measuring inconsistency in meta-analyses. BMJ, 327, 557–560.

Gelman, A. & Hill, J. (2006). Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge University Press.

Röver, C. (2020). Bayesian random-effects meta-analysis using the bayesmeta R package. Journal of Statistical Software, 93(6).

McElreath, R. (2020). Statistical Rethinking: A Bayesian Course with Examples in R and Stan, 2nd ed. CRC Press.

Stan Development Team. Stan Modeling Language User's Guide and Reference Manual.

Bürkner, P.-C. (2017). brms: An R package for Bayesian multilevel models using Stan. Journal of Statistical Software, 80(1).

Quantity	Question
Posterior for \(\mu\)	What is the average treatment effect?
Posterior for \(\tau\)	How much do treatment effects vary across studies?
Posterior predictive distribution	What treatment effect might occur in a new study?

Feature	Fixed Effect	Random Effects
Underlying effects	Common \(\mu\)	Study-specific \(\theta_i\)
Between-study heterogeneity	Not modeled	Modeled through \(\tau\)
Future-study prediction	Not naturally represented	Directly represented
Model complexity	Lower	Higher
Potential prior sensitivity	Usually lower	Often greater, especially for \(\tau\)

Model	Structure
Fixed effect	\(\theta_i=\mu\)
Random effects	\(\theta_i\sim N(\mu,\tau^2)\)
Meta-regression	\(\theta_i=\mathbf{x}_i^\top\beta+u_i\)
Multivariate	\(\boldsymbol{\theta}_i\sim N(\boldsymbol{\mu},\Sigma)\)
Network meta-analysis	Multiple interconnected treatment effects

Component	Bayesian Meta-Analysis
Observed study effect	\(y_i\)
Within-study SE	\(s_i\)
Study-specific effect	\(\theta_i\)
Overall effect	\(\mu\)
Between-study SD	\(\tau\)
Sampling model	\(y_i\mid\theta_i\sim N(\theta_i,s_i^2)\)
Hierarchical model	\(\theta_i\mid\mu,\tau\sim N(\mu,\tau^2)\)
Prior for overall effect	Example: \(\mu\sim N(0,1.5^2)\)
Prior for heterogeneity	Example: \(\tau\sim\operatorname{HalfNormal}(0,0.5)\)
Primary posterior quantity	\(p(\mu\mid y)\)
Heterogeneity posterior	\(p(\tau\mid y)\)
Future-study quantity	\(p(\theta_{\text{new}}\mid y)\)
Computational method	MCMC

Bayesian Meta-Analysis Methods

What You'll Learn

Introduction

What Is Meta-Analysis?

Why Bayesian Meta-Analysis?

The Basic Bayesian Framework

Study-Level Effects

Fixed-Effect Bayesian Meta-Analysis

Interpretation of the Fixed-Effect Assumption

Random-Effects Bayesian Meta-Analysis

Within-Study vs. Between-Study Variation

The Hierarchical Structure

The Prior for the Overall Effect

Choosing the Prior for Heterogeneity

Prior Predictive Reasoning

The Likelihood

The Posterior Distribution

Posterior Mean and Median

Credible Intervals

Posterior Probability of Benefit

A Worked Bayesian Meta-Analysis Example

Specify the Sampling Model

Specify the Random-Effects Model

Specify the Priors

Understanding Partial Pooling

Why Shrinkage Occurs

Marginalizing Over the Study Effects

The Posterior for the Overall Effect

Posterior Distribution of Heterogeneity

The Prediction Problem

Why Prediction Intervals Can Be Wider

Probability of Benefit in a Future Study

Fixed-Effect vs. Random-Effects Bayesian Meta-Analysis

What Does "Heterogeneity" Actually Mean?

Meta-Regression

Bayesian Meta-Regression

Bayesian Meta-Analysis of Binary Outcomes

Bayesian Meta-Analysis of Hazard Ratios

Bayesian Meta-Analysis of Mean Differences

Standardized Mean Differences

Prior Sensitivity Analysis

Why Prior Sensitivity Matters for \(\tau\)

Forest Plots in Bayesian Meta-Analysis

Bayesian Evidence Synthesis Is More Than a Weighted Average

Markov Chain Monte Carlo

Posterior Probability From MCMC Draws

Posterior Predictive Simulation

Convergence Diagnostics

R-Hat

Effective Sample Size

Bayesian vs. Frequentist Meta-Analysis

Bayesian Meta-Analysis With Historical Information

Dynamic Borrowing

Publication Bias

Small-Study Effects

Missing Data

Bayesian Network Meta-Analysis

Bayesian Meta-Analysis With Multiple Endpoints

Clinical Interpretation of the Overall Effect

R Implementation With brms

Specify the Bayesian Random-Effects Model

Inspect the Posterior

Transform the Overall Effect to a Hazard Ratio

Calculate Posterior Probability of Benefit

Inspect MCMC Diagnostics

Posterior Predictive Checks

Posterior Predictive Distribution for a New Study

A More Explicit Stan Model

Non-Centered Parameterization

Checking Sensitivity to the Prior on \(\tau\)

Comparing Bayesian Models

Model Checking vs. Model Selection

Common Mistakes

A Practical Bayesian Meta-Analysis Workflow

What Should Be Reported in a Statistical Analysis Plan?

How to Report a Bayesian Meta-Analysis

Interpreting a Bayesian Meta-Analysis Clinically

The Most Important Distinction: Mean Effect vs. Future Effect

Worked Example Summary

The Most Important Concepts to Remember