```html Network Meta-Analysis Explained: A Practical Guide — Clinical Biostats
Tutorials › Biostatistics › Network Meta-Analysis Explained

Home  /  Tutorials  /  Meta-Analysis

Evidence Synthesis

Network Meta-Analysis Explained

A practical guide to network meta-analysis, including treatment networks, direct and indirect evidence, transitivity, consistency, heterogeneity, relative treatment effects, treatment rankings, SUCRA, league tables, node-splitting, interpretation, and implementation in R.

Intermediate 22 min read

What You'll Learn

  • What network meta-analysis is and when it is useful
  • How direct and indirect evidence are combined
  • How transitivity, consistency, and heterogeneity affect validity
  • How treatment effects are estimated across the network
  • How treatment rankings, SUCRA, and P-scores should be interpreted
  • How to prepare data and perform NMA in R

Introduction

Suppose a clinical question involves five competing treatments: A, B, C, D, and E. A conventional pairwise meta-analysis can directly synthesize studies that compare the same two treatments.

But what happens when the evidence contains:

  • A versus B
  • A versus C
  • B versus D
  • C versus D
  • D versus E

There may be few or no randomized trials directly comparing B with C, B with E, or A with E.

A network meta-analysis (NMA) extends conventional meta-analysis by combining evidence from a connected network of treatments. It can estimate comparisons that have not been studied directly, provided the assumptions required for valid indirect comparison are reasonable.

Key idea: Network meta-analysis combines direct evidence from studies that compare two treatments with indirect evidence obtained through a common comparator or connected treatment network.

Pairwise Meta-Analysis vs. Network Meta-Analysis

A conventional pairwise meta-analysis answers a relatively focused question:

$$ \text{What is the average effect of Treatment A versus Treatment B?} $$

A network meta-analysis asks a broader question:

$$ \text{How do several treatments compare with one another within a connected evidence network?} $$
Feature Pairwise Meta-Analysis Network Meta-Analysis
Number of treatments Usually 2 3 or more
Direct comparisons Primary evidence Primary evidence
Indirect comparisons Generally not estimated Yes
Ranking treatments Limited Possible
League table No Common
Network geometry Not applicable Central diagnostic display

What Is a Treatment Network?

A treatment network represents the structure of the available randomized evidence.

Each treatment is represented by a node. A direct randomized comparison between two treatments is represented by an edge connecting the nodes.

The number of studies or participants contributing to a comparison can optionally be reflected in the thickness of the edge or size of the nodes.

Figure 1. Example Treatment Network
Nodes represent treatments. Lines represent direct randomized comparisons. The numbers indicate the illustrative number of studies contributing to each direct comparison.

Hover over a treatment node or comparison to inspect the illustrative evidence structure. The network is simulated for teaching purposes.

How to Read Network Geometry

The geometry of a network immediately tells you several things about the evidence base.

  • Which treatments have been compared directly?
  • Which comparisons have many studies?
  • Which treatments are poorly connected?
  • Which treatment comparisons rely heavily on indirect evidence?
  • Is the network connected?

A network must generally be connected for a single coherent NMA model to estimate relative effects among all treatments.

A Simple Three-Treatment Network

Consider three treatments:

  • A = standard treatment
  • B = new treatment 1
  • C = new treatment 2

Suppose randomized trials directly compare:

  • A vs B
  • A vs C

but no trial directly compares B vs C.

The network is nevertheless connected.

This allows an indirect B-versus-C comparison through A.

Figure 2. Indirect Comparison Through a Common Comparator
B and C have no direct comparison, but both have randomized evidence against A.

The Logic of an Indirect Comparison

Suppose the treatment effect of B versus A is:

$$ \theta_{BA} $$

and the treatment effect of C versus A is:

$$ \theta_{CA} $$

On a consistent additive treatment-effect scale, the indirect B-versus-C effect can be represented as:

$$ \theta_{BC} = \theta_{BA}-\theta_{CA} $$

This is the basic mathematical intuition behind indirect evidence in an NMA.

A Numerical Example

Suppose lower risk is better and the estimated log risk ratios are:

$$ \log(RR_{BA})=-0.20 $$

and:

$$ \log(RR_{CA})=-0.50 $$

Then:

$$ \log(RR_{BC}) = -0.20-(-0.50) = 0.30 $$

Converting back to the risk-ratio scale:

$$ RR_{BC}=e^{0.30}\approx1.35 $$

Thus, the indirect estimate suggests approximately a 35% higher relative risk for B compared with C under this illustrative calculation.

Important: The numerical calculation is simple; the validity of the indirect comparison is not. It depends on assumptions about the comparability of the trials contributing the different comparisons.

Direct Evidence

Direct evidence comes from randomized trials that explicitly compare the two treatments of interest.

For example:

$$ A\quad\text{vs.}\quad B $$

If five randomized trials compare A with B, the pairwise A-versus-B estimate is direct evidence from those five trials.

Indirect Evidence

Indirect evidence arises when two treatments can be compared through one or more common comparators.

For example:

$$ B\rightarrow A\rightarrow C $$

The indirect B-versus-C estimate uses evidence from B-versus-A and C-versus-A comparisons.

Mixed Evidence

An NMA estimate can combine both:

  • Direct evidence for a comparison
  • Indirect evidence for the same comparison

The resulting estimate is often called a network estimate or mixed estimate.

This is one of the major advantages of NMA: all relevant evidence can contribute to the comparison while preserving the network structure.

The Central Assumption: Transitivity

The most important conceptual assumption in an indirect comparison is transitivity.

Informally, transitivity means that the studies contributing different comparisons are sufficiently comparable with respect to effect modifiers that would influence relative treatment effects.

Imagine trials comparing:

  • B versus A in younger, low-risk patients
  • C versus A in older, high-risk patients

If age and baseline disease severity substantially modify treatment effects, the indirect B-versus-C comparison may be biased.

Think of transitivity as a comparability question: Would it be reasonable to expect the B-versus-C comparison to be transported across the A comparison because the important trial characteristics are sufficiently similar?

Common Effect Modifiers

Potential effect modifiers depend on the clinical question, but may include:

  • Age
  • Disease severity
  • Previous treatment
  • Disease stage
  • Baseline risk
  • Biomarker status
  • Duration of follow-up
  • Dose or treatment intensity
  • Outcome definition
  • Timing of outcome assessment

These variables should be considered during protocol development and evidence assessment rather than only after seeing the NMA results.

Transitivity vs. Consistency

These concepts are related but not identical.

Concept Core Question
Transitivity Are the studies sufficiently comparable for indirect comparison?
Consistency Do direct and indirect estimates agree statistically within the network?
Heterogeneity How much do treatment effects vary across studies beyond sampling error?

Consistency in Network Meta-Analysis

Consistency, sometimes called coherence, concerns agreement between direct and indirect evidence.

Suppose direct trials estimate:

$$ \theta_{BC}^{direct} $$

while the network's indirect evidence estimates:

$$ \theta_{BC}^{indirect} $$

If the two estimates are substantially different, the network may exhibit inconsistency.

A Simple Consistency Check

Consider:

$$ \theta_{BC}^{direct}=0.10 $$

and:

$$ \theta_{BC}^{indirect}=0.45 $$

The difference is:

$$ 0.45-0.10=0.35 $$

The practical importance of that difference depends on its uncertainty, clinical context, and the statistical model.

Do not interpret a consistency test in isolation. Evidence of inconsistency should trigger investigation of study characteristics, effect modifiers, outcome definitions, risk of bias, and other sources of disagreement.

Heterogeneity in NMA

NMA retains the concept of between-study heterogeneity from conventional meta-analysis.

Under a random-effects framework, the treatment effect in study \(i\) can be conceptually represented as:

$$ \theta_i=\theta+\epsilon_i $$

where:

  • \(\theta\) is the average treatment effect
  • \(\epsilon_i\) represents study-specific deviation

The variance of these deviations is commonly represented by:

$$ \tau^2 $$

or by the heterogeneity standard deviation:

$$ \tau $$

Why Heterogeneity Matters in NMA

An NMA may combine evidence from dozens of trials and several treatment comparisons.

If treatment effects vary substantially across studies, a common-effect model may be unrealistic.

A random-effects NMA allows for between-study variability.

However, random effects do not automatically solve problems caused by clinical heterogeneity or violation of transitivity.

Important distinction: Statistical heterogeneity is not the same thing as lack of transitivity. A random-effects model addresses variation in treatment effects; it does not make fundamentally incomparable populations exchangeable.

Relative Treatment Effects

NMA estimates relative effects between treatments.

The appropriate effect measure depends on the outcome.

Outcome Common Effect Measures
Binary Risk ratio, odds ratio, risk difference
Continuous Mean difference, standardized mean difference
Time-to-event Hazard ratio

Choosing a Reference Treatment

NMA models generally require a treatment parameterization relative to a reference treatment.

The reference could be:

  • Placebo
  • Standard of care
  • Usual care
  • A clinically established treatment
  • Any treatment chosen for model identification

The choice of reference does not change the underlying relative information, but it changes how the model parameters are expressed.

Example With Risk Ratios

Suppose placebo is treatment A.

An NMA estimates:

Comparison RR 95% CI
B vs A 0.80 0.68–0.94
C vs A 0.65 0.54–0.78
D vs A 0.72 0.60–0.87

If lower risk is better, C has the lowest estimated risk ratio versus A in this illustrative example.

But this does not automatically mean C is definitively the best treatment. The uncertainty intervals and all other network comparisons must be considered.

The League Table

One of the most recognizable outputs from NMA is the league table.

A league table presents pairwise network estimates for all treatments in the network.

A B C D E
A — 0.82 (0.70–0.96) 0.68 (0.55–0.83) 0.75 (0.61–0.92) 0.90 (0.72–1.13)
B 1.22 (1.04–1.43) — 0.83 (0.67–1.03) 0.91 (0.74–1.12) 1.10 (0.88–1.37)
C 1.47 (1.20–1.82) 1.20 (0.97–1.49) — 1.09 (0.87–1.36) 1.32 (1.02–1.70)
D 1.33 (1.09–1.64) 1.10 (0.89–1.35) 0.92 (0.74–1.15) — 1.21 (0.95–1.54)
E 1.11 (0.88–1.39) 0.91 (0.73–1.14) 0.76 (0.59–0.98) 0.83 (0.65–1.05) —

The direction of the effect must always be defined clearly.

League-table warning: A league table can be visually compact but statistically dense. Readers must know which treatment is the row treatment, which is the column treatment, what effect measure is used, and whether lower or higher values represent benefit.

Ranking Treatments

Because NMA simultaneously compares multiple treatments, investigators often want to rank treatments.

Common ranking summaries include:

  • Probability of being best
  • Probability of each possible rank
  • SUCRA
  • P-scores

These measures can be useful summaries, but they should not be interpreted as absolute proof that one treatment is superior.

Rank Probabilities

Suppose five treatments are being compared. Each treatment has a probability of occupying each rank.

Treatment Rank 1 Rank 2 Rank 3 Rank 4 Rank 5
A 8% 15% 22% 28% 27%
B 18% 25% 27% 20% 10%
C 45% 31% 15% 7% 2%
D 23% 22% 24% 21% 10%
E 6% 7% 12% 24% 51%

Treatment C has the highest probability of being ranked first, but it is not ranked first with certainty.

What Is SUCRA?

SUCRA stands for Surface Under the Cumulative Ranking Curve.

It summarizes the cumulative probability that a treatment occupies favorable ranks.

For \(k\) treatments, SUCRA can be represented conceptually as:

$$ SUCRA_i = \frac{1}{k-1} \sum_{r=1}^{k-1} \left(1-C_{ir}\right) $$

where \(C_{ir}\) represents the cumulative probability associated with the ranking distribution under the chosen ranking convention.

The exact implementation depends on the ranking direction and software.

Interpretation: A higher SUCRA generally indicates a treatment is more likely to occupy favorable positions in the ranking distribution. SUCRA is a relative ranking summary, not a direct estimate of clinical benefit.

SUCRA Example

Treatment Illustrative SUCRA Interpretation
C 91% Frequently occupies favorable ranks
D 74% Generally favorable ranking
B 63% Intermediate ranking
A 39% Less favorable ranking
E 18% Frequently occupies less favorable ranks

A SUCRA of 91% should not be interpreted as "91% effective" or as a 91% probability that treatment C is clinically superior to every other treatment.

P-Scores

In frequentist network meta-analysis, P-scores provide a related treatment-ranking summary based on the certainty that one treatment is better than another across comparisons.

Like SUCRA, P-scores range conceptually from 0 to 1.

A larger value indicates more favorable ranking under the specified direction of benefit.

SUCRA and P-scores are related but arise from different inferential frameworks and should not be treated as interchangeable without understanding the underlying methodology.

Ranking Is Not the Same as Estimating Treatment Effects

Consider two treatments:

$$ RR=0.95 \qquad 95\%\,CI=(0.70,1.29) $$

The treatment could still rank highly in an NMA if other treatments perform similarly.

A high ranking does not necessarily imply a large or precisely estimated clinical effect.

Best practice: Report the relative treatment effects and their uncertainty first. Use rankings as a supplementary summary rather than allowing the ranking measure to replace effect estimates and clinical interpretation.

Network Meta-Analysis Models

Two broad modeling frameworks are commonly used:

  • Frequentist NMA
  • Bayesian NMA

Both can synthesize a connected treatment network, but they express uncertainty and produce some summaries differently.

Frequentist NMA

Frequentist approaches estimate treatment effects and uncertainty using sampling distributions.

Common outputs include:

  • Network treatment-effect estimates
  • 95% confidence intervals
  • Heterogeneity estimates
  • Consistency assessments
  • P-scores

Bayesian NMA

Bayesian NMA specifies prior distributions and combines them with the likelihood contributed by the observed trial data.

The result is a posterior distribution for each treatment effect.

Conceptually:

$$ p(\theta\mid data) \propto p(data\mid\theta)\,p(\theta) $$

where:

  • \(p(\theta\mid data)\) is the posterior distribution
  • \(p(data\mid\theta)\) is the likelihood
  • \(p(\theta)\) is the prior distribution

Bayesian NMA can directly produce posterior probabilities of treatment rankings and probabilities that one treatment is better than another.

Fixed-Effect vs. Random-Effects NMA

As with pairwise meta-analysis, NMA can use fixed-effect or random-effects assumptions.

Model Assumption
Fixed effect Underlying treatment effect is common across studies
Random effects Underlying treatment effects vary across studies

A random-effects model is often attractive when clinical and methodological diversity makes a common effect implausible, but model choice should be specified and justified rather than selected solely because it produces a preferred result.

Study-Level Data

A basic NMA dataset commonly contains:

Variable Description
Study ID Unique randomized trial identifier
Treatment Intervention assigned to the arm
Events Number of events for a binary outcome
Sample size Number randomized or analyzed, according to the analysis
Mean / SD Continuous-outcome summaries when applicable
Follow-up Time point associated with the outcome

Multi-Arm Trials

Multi-arm randomized trials require special attention.

Suppose one study randomizes participants to:

  • A
  • B
  • C

The three pairwise contrasts from the same trial are statistically correlated.

A naïve analysis that treats all contrasts as independent can give the trial too much influence and underestimate uncertainty.

Important: Multi-arm trials should be modeled using methods that account for the correlation among treatment comparisons arising from the same randomized study.

Network Geometry and Evidence Contribution

The amount of information supporting a comparison can differ dramatically across the network.

For example:

Comparison Direct Studies Participants Evidence Type
A vs B 14 4,820 Mostly direct
A vs C 8 2,740 Direct
B vs C 0 — Indirect
C vs D 3 740 Direct + indirect
B vs D 1 180 Mostly indirect

The absence of direct studies does not mean that an estimate cannot be obtained. It does mean that the estimate depends more heavily on indirect evidence and therefore on the assumptions underlying that evidence.

Contribution of Direct and Indirect Evidence

For a particular treatment comparison, it is useful to determine whether the network estimate is primarily driven by:

  • Direct evidence
  • Indirect evidence
  • A combination of both

This becomes especially important when direct and indirect estimates disagree.

Node Splitting

Node splitting is one approach for investigating inconsistency between direct and indirect evidence for a particular comparison.

The direct evidence and indirect evidence are separated and estimated independently.

For example:

1
Estimate the B-versus-C effect using only direct B-versus-C trials.
2
Estimate the B-versus-C effect using indirect paths through the network.
3
Compare the direct and indirect estimates.
4
Investigate clinically and statistically important discrepancies.

Local vs. Global Inconsistency

Inconsistency can be assessed at different levels.

Approach Purpose
Node splitting Examines direct vs indirect evidence for a particular comparison
Design-by-treatment interaction Assesses inconsistency more globally across the network
Loop-specific approaches Investigate disagreement within particular evidence loops

Evidence Loops

A closed loop exists when treatments can be connected through multiple independent paths.

For example:

$$ A\rightarrow B\rightarrow C\rightarrow A $$

This structure permits a comparison of direct and indirect information around the loop.

If:

$$ \theta_{AB}+\theta_{BC}+\theta_{CA}\neq0 $$

on the appropriate additive treatment-effect scale, there is evidence of disagreement that may indicate inconsistency.

Why Log Scales Are Convenient

For ratio measures such as risk ratios, odds ratios, and hazard ratios, treatment effects are often modeled on the logarithmic scale.

For example:

$$ \theta_{AB}=\log(HR_{AB}) $$

This makes reciprocal comparisons additive:

$$ \log(HR_{AB})=-\log(HR_{BA}) $$

and treatment paths can be combined algebraically.

Worked Example: Three Treatments

Suppose a network contains treatments A, B, and C.

Comparison Observed Effect Interpretation
B vs A RR = 0.80 B has lower event risk than A
C vs A RR = 0.60 C has lower event risk than A

The indirect B-versus-C comparison can be derived using the relative effects on the logarithmic scale:

$$ \log(RR_{BC}) = \log(RR_{BA})-\log(RR_{CA}) $$

Therefore:

$$ RR_{BC} = \frac{RR_{BA}}{RR_{CA}} = \frac{0.80}{0.60} = 1.33 $$

The illustrative indirect estimate is therefore:

$$ RR_{BC}=1.33 $$

Under this outcome definition, B has an estimated 33% higher risk than C.

Why the Arithmetic Alone Is Not Enough

The calculation above assumes that the B-versus-A and C-versus-A evidence can legitimately be connected.

If the B-versus-A trials enrolled substantially different patients from the C-versus-A trials and important effect modifiers differed, the mathematical relationship may not provide a valid clinical comparison.

This is why network meta-analysis is fundamentally both a statistical and a clinical evidence-synthesis exercise.

Risk of Bias

Risk of bias should be assessed at the study level and incorporated into the interpretation of the network.

Important domains can include:

  • Randomization
  • Allocation concealment
  • Blinding
  • Missing outcome data
  • Outcome measurement
  • Selective reporting

A statistically precise NMA estimate can still be clinically unreliable if the contributing evidence has substantial risk of bias.

Publication Bias and Small-Study Effects

Selective publication can affect both pairwise meta-analysis and NMA.

Small studies may also differ systematically from large studies.

Assessment may include:

  • Comparison-adjusted funnel plots
  • Small-study-effect investigations
  • Search strategies designed to identify unpublished evidence
  • Registry and protocol searches

No single graphical test can establish the presence or absence of publication bias with certainty.

Network Connectivity

Before fitting an NMA, verify that the treatment network is connected.

For example:

Figure 3. Connected vs. Disconnected Networks
The left network permits a common synthesis. The right network contains separate components and cannot produce one common set of relative effects without additional assumptions or information.

Connected Network

If:

$$ A\leftrightarrow B\leftrightarrow C\leftrightarrow D $$

all four treatments belong to the same connected evidence component.

Disconnected Network

If:

$$ A\leftrightarrow B \qquad\qquad C\leftrightarrow D $$

there is no evidence path connecting the two components.

A standard NMA cannot estimate a single A-versus-C network effect from these data alone.

Network Geometry Is More Than a Pretty Figure

The network graph should be treated as a diagnostic tool.

A sparse edge may indicate:

  • Few randomized studies
  • Small sample size
  • Limited precision
  • Heavy reliance on indirect evidence

A highly connected network may provide more opportunities for cross-validation between direct and indirect evidence.

Outcome Definition Matters

All studies contributing to an NMA should be sufficiently compatible with the outcome being synthesized.

For example, "treatment response" may mean very different things depending on:

  • Response criteria
  • Assessment time point
  • Responder definition
  • Analysis population
  • Follow-up duration

Combining superficially similar outcomes without examining their definitions can undermine the validity of the network.

Time Points and Follow-Up

For time-dependent outcomes, differences in follow-up can be particularly important.

For example, overall survival measured after five years and response measured at eight weeks represent different estimands and should not be mixed merely because both are clinical outcomes.

For repeated or time-specific outcomes, the analysis should define the relevant estimand and time window in advance.

Continuous Outcomes

Suppose treatment effects are measured using a mean difference.

For study \(i\):

$$ MD_i=\bar X_{i,T}-\bar X_{i,C} $$

The NMA then synthesizes the relative treatment effects across the connected network.

When scales differ across studies, a standardized mean difference may be appropriate if the underlying construct is sufficiently comparable.

Binary Outcomes

For binary outcomes, an NMA may model:

$$ \log(RR),\qquad \log(OR),\qquad \text{or}\qquad RD $$

The effect measure should reflect the clinical question and statistical properties of the outcome.

Time-to-Event Outcomes

For survival outcomes, the hazard ratio is frequently used:

$$ \theta_{AB}=\log(HR_{AB}) $$

The interpretation of a hazard ratio is not identical to a risk ratio. It describes a relative rate or hazard under the assumptions of the survival model and should not automatically be translated into a percentage change in absolute risk.

Absolute Effects From NMA

Relative treatment effects are often easier to transport across studies than absolute effects.

If a network estimates:

$$ RR=0.75 $$

the corresponding absolute risk reduction depends on the baseline risk.

If baseline risk is 20%:

$$ Risk_T=0.75\times0.20=0.15 $$

giving an illustrative absolute risk reduction of:

$$ 0.20-0.15=0.05 $$

or 5 percentage points.

If baseline risk were 5%, the absolute reduction would be much smaller.

Clinical interpretation: A treatment ranking based on relative effects does not automatically determine which treatment has the greatest absolute benefit for a particular patient population.

Network Meta-Regression

If important effect modifiers differ across studies, meta-regression may be considered.

Conceptually:

$$ \theta_i = \theta + \beta X_i + \epsilon_i $$

where \(X_i\) represents a study-level covariate.

Potential covariates might include:

  • Mean age
  • Baseline risk
  • Percentage previously treated
  • Disease severity

Network meta-regression requires strong justification and sufficient information. It should not be used merely as a mechanical correction for every observed difference between studies.

Effect Modification vs. Prognostic Differences

A variable may predict outcomes without modifying the relative treatment effect.

For NMA, the key concern is whether the variable changes the comparative effect between treatments.

This distinction is important when deciding which study characteristics threaten transitivity.

What Happens When Direct and Indirect Evidence Disagree?

Suppose:

$$ RR_{BC}^{direct}=0.70 $$

while:

$$ RR_{BC}^{indirect}=1.10 $$

The disagreement is substantial.

Possible explanations include:

  • Effect modification
  • Different patient populations
  • Different outcome definitions
  • Different follow-up
  • Risk of bias
  • Random variation
  • Model misspecification

The correct response is investigation, not simply selecting whichever estimate looks more favorable.

Global Model Fit

In Bayesian NMA, model fit can be evaluated using quantities such as residual deviance and comparisons between observed and fitted data.

Poor fit can indicate:

  • Inconsistency
  • Heterogeneity
  • Incorrect likelihood
  • Model misspecification

Model diagnostics should be considered alongside clinical knowledge rather than interpreted mechanically.

Prediction Intervals

In a random-effects framework, confidence intervals describe uncertainty in the estimated average effect.

A prediction interval additionally reflects expected variation in a future study.

Conceptually:

$$ \theta \pm z_{0.975}\sqrt{SE(\theta)^2+\tau^2} $$

The prediction interval can therefore be substantially wider than the confidence interval when heterogeneity is appreciable.

Why this matters: A treatment may have a statistically favorable average effect while the prediction interval indicates substantial uncertainty about the effect in a new clinical setting.

Certainty of Evidence

An NMA estimate should not be judged solely by its point estimate.

Evidence certainty can be influenced by:

  • Risk of bias
  • Imprecision
  • Indirectness
  • Inconsistency
  • Publication bias
  • Network structure
  • Dependence on indirect evidence

Frameworks such as GRADE have been adapted for network meta-analysis, including considerations specific to direct and indirect evidence.

Ranking Probability Plot

A useful graphical summary is the probability of occupying each possible rank.

Figure 4. Illustrative Treatment-Ranking Probabilities
Each horizontal bar represents the probability that a treatment occupies a favorable ranking position. These values are simulated.
Treatment Probability of Rank 1
A 8%
B 18%
C 45%
D 23%
E 6%

Why Rank 1 Probability Can Be Misleading

Suppose treatment C has a 45% probability of being best.

That means it has the largest probability among the treatments in this illustrative example.

But there is still a 55% probability that another treatment occupies the top rank.

Ranking distributions should therefore be examined in their entirety.

Choosing the "Best" Treatment

An NMA should not automatically conclude:

$$ \text{Highest SUCRA}=\text{best treatment} $$

Clinical decision-making also requires consideration of:

  • Magnitude of benefit
  • Precision
  • Safety
  • Treatment burden
  • Cost
  • Patient preferences
  • Evidence certainty
  • Applicability to the target population

Benefit-Risk Considerations

A treatment can rank first for efficacy while ranking poorly for safety.

Therefore, separate NMAs may sometimes be performed for:

  • Efficacy
  • Adverse events
  • Serious adverse events
  • Discontinuation due to toxicity

These results can then inform a broader benefit-risk assessment.

NMA and Multiple Outcomes

An NMA for overall survival may produce a different ranking from an NMA for progression-free survival.

Likewise, efficacy and safety rankings can differ.

Do not collapse multiple clinical dimensions into one ranking without a prespecified framework. A single "best treatment" ranking can conceal important trade-offs between benefit and harm.

Preparing Data for NMA

A practical workflow begins before statistical modeling.

1
Define the clinical question and estimand.
2
Define eligibility criteria and the treatment nodes.
3
Identify all relevant randomized trials.
4
Extract treatment-arm outcome data.
5
Standardize outcome definitions and time points.
6
Assess clinical and methodological similarity.
7
Construct and inspect the treatment network.
8
Fit the prespecified NMA model.
9
Assess heterogeneity and consistency.
10
Interpret treatment effects, rankings, and certainty together.

R Implementation

Several R packages can be used for network meta-analysis. One commonly used frequentist framework is provided by netmeta. 

install.packages("netmeta")

library(netmeta)

Pairwise Data Format

A common approach begins with arm-level data and converts them into pairwise comparisons. 

data <- data.frame(
  study = c(
    "Study1", "Study1",
    "Study2", "Study2",
    "Study3", "Study3"
  ),
  treatment = c(
    "A", "B",
    "A", "C",
    "B", "C"
  ),
  event = c(
    40, 30,
    42, 25,
    35, 28
  ),
  n = c(
    100, 100,
    110, 110,
    100, 100
  )
)

For a production analysis, the exact data structure should follow the requirements of the selected software and the prespecified analysis plan.

Creating Pairwise Comparisons

pw <- pairwise(
  treat = treatment,
  event = event,
  n = n,
  studlab = study,
  data = data,
  sm = "RR"
)

The resulting object contains the pairwise treatment comparisons that can be used by the NMA procedure.

Running a Frequentist NMA

nma <- netmeta(
  TE,
  seTE,
  treat1,
  treat2,
  studlab,
  data = pw,
  sm = "RR",
  random = TRUE
)

summary(nma)

The exact argument structure depends on the version of the package and the analysis data. Always validate the code against the current package documentation before using it in a regulated analysis.

Network Plot in R

netgraph(
  nma,
  plastic = FALSE,
  thickness = "number.of.studies",
  points = TRUE,
  cex.points = 4
)

A network graph should be reviewed alongside the extracted trial dataset to ensure that every edge represents the intended randomized comparison.

Forest Plot

forest(
  nma,
  ref = "A"
)

A forest plot can summarize network estimates relative to a chosen reference treatment.

League Table in R

netleague(
  nma,
  digits = 2
)

The league table should be accompanied by a clear explanation of treatment direction and the effect measure.

Treatment Ranking in R

netrank(
  nma,
  small.values = "good"
)

The ranking direction must be specified correctly. For an outcome where smaller values indicate benefit, the software needs to know that smaller effects are favorable.

Common programming error: A ranking calculation can be numerically correct but clinically reversed if the software is not told whether larger or smaller treatment effects represent benefit.

Bayesian NMA in R

Bayesian analyses can be performed using packages such as gemtc or other Bayesian modeling frameworks. 

install.packages("gemtc")

library(gemtc)

network <- mtc.network(
  data.ab = arm.data
)

model <- mtc.model(
  network,
  type = "consistency",
  linearModel = "random"
)

result <- mtc.run(model)

summary(result)

Bayesian model specification requires additional decisions concerning prior distributions, number of iterations, burn-in, thinning where applicable, convergence assessment, and model diagnostics.

Bayesian Convergence

A Bayesian NMA should not be considered complete simply because a model has produced output.

Convergence diagnostics may include:

  • Trace plots
  • Potential scale reduction statistics
  • Effective sample size
  • Autocorrelation

Poor convergence can invalidate posterior summaries.

Sensitivity Analyses

Sensitivity analyses can evaluate how robust the network conclusions are to reasonable alternative assumptions.

Examples include:

  • Fixed-effect vs random-effects model
  • Alternative outcome definitions
  • Different follow-up windows
  • Excluding high-risk-of-bias studies
  • Alternative treatment-node definitions
  • Alternative priors in Bayesian analyses

Combining Treatment Nodes

One difficult clinical decision is whether two interventions should be represented by the same treatment node.

Suppose two studies compare:

  • Drug A, 10 mg daily
  • Drug A, 20 mg daily

Whether these should be one node or separate nodes depends on the clinical question and whether dose materially modifies the treatment effect.

Node definitions should be clinically justified and specified before interpreting the network.

Class Effects vs Individual Treatments

Some NMAs compare individual treatments while others compare treatment classes.

For example:

  • ACE inhibitor A
  • ACE inhibitor B
  • ARB C
  • ARB D

One analysis might preserve four individual nodes. Another might compare two classes.

These are different scientific questions and should not be conflated.

Network Meta-Analysis vs Pairwise Meta-Analysis

Question Pairwise NMA
A vs B? Yes Yes
B vs C with no direct trials? No Potentially yes
Compare 5 treatments simultaneously? No Yes
Use indirect evidence? No Yes
Rank treatments? Limited Yes
Require transitivity assessment? Less central Critical

NMA vs. Traditional Indirect Comparison

A traditional adjusted indirect comparison may compare B and C through a common comparator A.

NMA generalizes this principle to an entire connected network.

Instead of considering only:

$$ B\rightarrow A\rightarrow C $$

the NMA can use all connected evidence paths.

Common Mistakes

  1. Assuming network connectivity guarantees validity. A connected graph is necessary for a single network synthesis, but it does not establish transitivity.
  2. Ignoring treatment effect modifiers. Major differences in effect modifiers can invalidate indirect comparisons.
  3. Assuming random effects solve inconsistency. Heterogeneity and inconsistency are different problems.
  4. Treating SUCRA as a clinical efficacy percentage. SUCRA is a ranking summary, not a treatment-effect measure.
  5. Reporting only the ranking. Relative effects and uncertainty should remain central.
  6. Ignoring multi-arm trials. The correlation among comparisons from the same trial must be handled.
  7. Using incompatible outcome definitions. Superficially similar endpoints may not represent the same estimand.
  8. Failing to specify treatment-node definitions. Different doses, formulations, or treatment classes may not be interchangeable.
  9. Ignoring disconnected networks. Separate network components cannot generally be synthesized into one common ranking without additional information or assumptions.
  10. Interpreting indirect evidence as equivalent to direct evidence. Indirect comparisons depend on assumptions that must be evaluated.

What a High-Quality NMA Should Report

A transparent NMA report should describe:

  • Clinical question
  • Eligibility criteria
  • Treatment-node definitions
  • Outcome definitions
  • Search strategy
  • Included trials
  • Risk-of-bias assessment
  • Network geometry
  • Statistical model
  • Effect measure
  • Heterogeneity assumptions
  • Consistency assessment
  • Treatment-effect estimates
  • Ranking summaries
  • Sensitivity analyses
  • Certainty of evidence

PRISMA-NMA Considerations

Network meta-analyses should follow reporting guidance specifically developed for systematic reviews involving network meta-analysis.

The report should make clear:

  • How the network was constructed
  • How transitivity was assessed
  • How inconsistency was evaluated
  • How rankings were generated
  • How uncertainty was quantified

Clinical Interpretation Example

Imagine an NMA of five treatments for prevention of a clinical event.

Treatment RR vs Standard 95% CI SUCRA
A 1.00 Reference 39%
B 0.82 0.70–0.96 63%
C 0.68 0.55–0.83 91%
D 0.75 0.61–0.92 74%
E 0.90 0.72–1.13 18%

A reasonable interpretation is that C has the most favorable estimated ranking in this illustrative analysis and appears to reduce risk relative to A.

However, a complete interpretation still requires examination of:

  • Direct vs indirect evidence
  • Network consistency
  • Risk of bias
  • Heterogeneity
  • Precision
  • Absolute event rates
  • Safety outcomes

The Difference Between "Best" and "Most Certain"

A treatment can have the highest point estimate of efficacy while having substantial uncertainty.

Another treatment may have a slightly smaller estimated effect but much greater precision.

Therefore:

$$ \text{Ranking} \neq \text{Certainty} \neq \text{Magnitude of Effect} $$

These are three different dimensions of evidence interpretation.

Absolute Risk Makes Results More Clinically Useful

Suppose:

$$ RR=0.70 $$

This represents the same relative effect regardless of baseline risk.

But if baseline risk is:

$$ 30\% $$

the treated risk is approximately:

$$ 0.70\times0.30=21\% $$

The absolute reduction is:

$$ 30\%-21\%=9\% $$

If baseline risk is only 3%, the corresponding absolute reduction is approximately 0.9 percentage points.

Thus, the same network relative effect can imply very different clinical benefits in different populations.

Publication-Quality Network Diagram

A final NMA network diagram should make the evidence structure easy to understand without requiring the reader to inspect the underlying dataset.

At minimum, consider displaying:

  • Treatment names
  • Direct comparison edges
  • Number of studies per comparison
  • Number of participants when useful
  • A clear legend

Quality Control of an NMA

1
Verify every included randomized trial.
2
Verify treatment-node assignments.
3
Verify outcome definitions and time points.
4
Verify multi-arm trial handling.
5
Verify network connectivity.
6
Inspect network geometry.
7
Check heterogeneity assumptions.
8
Assess consistency.
9
Validate treatment-effect estimates against source data.
10
Validate rankings and league tables independently.

How to Validate a League Table

For every pair of treatments, verify:

  • Correct treatment direction
  • Correct effect measure
  • Correct point estimate
  • Correct confidence or credible interval
  • Correct treatment labels
  • Correct reference treatment

A common reporting error is reversing the comparison direction.

For example:

$$ RR_{AB} = \frac{Risk_A}{Risk_B} $$

is the reciprocal of:

$$ RR_{BA} = \frac{Risk_B}{Risk_A} $$

Therefore:

$$ RR_{BA}=\frac{1}{RR_{AB}} $$

Interpreting Confidence Intervals

Suppose:

$$ RR=0.75 \qquad 95\%\,CI=(0.60,0.94) $$

The interval excludes 1, which is consistent with a statistically detectable difference under the specified model and conventional two-sided testing framework.

But statistical significance does not automatically establish clinical importance.

Clinical Importance vs Statistical Significance

Suppose another treatment has:

$$ RR=0.97 \qquad 95\%\,CI=(0.94,1.00) $$

A highly precise small effect may be statistically interesting while having limited clinical importance.

NMA interpretation should therefore consider prespecified clinically important differences where available.

Prediction for a New Setting

When heterogeneity is substantial, the average network effect may not describe every future setting well.

A prediction interval can help communicate this uncertainty.

For example:

$$ RR=0.75 \quad 95\%\,CI=(0.65,0.87) $$

could coexist with a substantially wider prediction interval.

This indicates that the average evidence favors treatment, while the expected effect in another setting may vary considerably.

When NMA Is Appropriate

NMA is particularly useful when:

  • Several interventions address the same clinical question.
  • Direct head-to-head evidence is incomplete.
  • The treatment network is sufficiently connected.
  • Trials are clinically comparable enough to support indirect inference.
  • A simultaneous comparison of interventions is clinically useful.

When NMA May Be Inappropriate

Caution is warranted when:

  • Trials involve fundamentally different populations.
  • Important effect modifiers differ substantially.
  • Outcome definitions are incompatible.
  • The network is disconnected.
  • Risk of bias is extreme or highly unbalanced.
  • There is strong unexplained inconsistency.
  • Treatment nodes combine clinically distinct interventions.

NMA Is Not a Substitute for Clinical Judgment

The statistical model can combine evidence, but it cannot determine whether two clinical populations are sufficiently comparable.

That requires subject-matter knowledge.

Similarly, the software can calculate SUCRA, but it cannot determine whether a small efficacy advantage outweighs a meaningful safety disadvantage.

Practical NMA Checklist

1
Define the clinical question and estimand.
2
Define the treatment nodes.
3
Define the outcome and time point.
4
Identify all relevant randomized evidence.
5
Assess risk of bias.
6
Draw the treatment network.
7
Assess transitivity.
8
Fit the prespecified NMA model.
9
Assess heterogeneity and consistency.
10
Report relative effects, uncertainty, rankings, and certainty together.

The Most Important Concepts

Concept Meaning
Network Connected structure of randomized treatment comparisons
Direct evidence Evidence from trials directly comparing two treatments
Indirect evidence Evidence obtained through one or more common comparators
Transitivity Clinical and methodological comparability supporting indirect inference
Consistency Agreement between direct and indirect evidence
Heterogeneity Variation in treatment effects across studies
League table Matrix of network treatment comparisons
SUCRA Summary of cumulative treatment-ranking probabilities
P-score Frequentist summary of relative treatment ranking
Node splitting Comparison of direct and indirect evidence for a particular comparison

Bottom Line

Network meta-analysis extends conventional meta-analysis by allowing multiple treatments to be compared simultaneously through a connected network of randomized evidence. Its major strength is the ability to combine direct and indirect evidence and to estimate comparisons for which head-to-head trials may be unavailable. Its major vulnerability is that indirect inference depends on assumptions, especially transitivity and consistency. A credible NMA therefore requires more than a sophisticated statistical model. The analyst must evaluate the clinical comparability of trials, inspect the network structure, account for heterogeneity and multi-arm studies, assess inconsistency, quantify uncertainty, and interpret treatment rankings in the context of effect magnitude, safety, and evidence certainty.

References

Higgins, J.P.T., Jackson, D., Barrett, J.K., Lu, G., Ades, A.E., & White, I.R. (2012). Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies. Research Synthesis Methods, 3(2), 98–110.
Salanti, G. (2012). Indirect and mixed-treatment comparison, network meta-analysis and multi-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Research Synthesis Methods, 3(2), 80–97.
Chaimani, A., Caldwell, D.M., Li, T., Higgins, J.P.T., & Salanti, G. (2017). Chapter 11: Undertaking network meta-analyses. In Cochrane Handbook for Systematic Reviews of Interventions.
Caldwell, D.M., Ades, A.E., & Higgins, J.P.T. (2005). Simultaneous comparison of multiple treatments: combining direct and indirect evidence. BMJ, 331, 897–900.
Dias, S., Welton, N.J., Sutton, A.J., & Ades, A.E. (2011). Evidence synthesis for decision making 2: A generalized linear modeling framework for pairwise and network meta-analysis of randomized controlled trials. Medical Decision Making, 33(5), 607–617.
Rücker, G. & Schwarzer, G. (2015). Ranking treatments in frequentist network meta-analysis works without resampling methods. BMC Medical Research Methodology, 15, 58.
Page, M.J., McKenzie, J.E., Bossuyt, P.M., et al. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ, 372, n71.
Hutton, B., Salanti, G., Caldwell, D.M., et al. (2015). The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Annals of Internal Medicine, 162(11), 777–784.

Related Tutorials

Related Tools

← Back to All Tutorials
```