Fixed-Effect vs. Random-Effects Meta-Analysis

This means the evidence can support a beneficial average effect while still allowing the true effect in a new setting to be little different from no effect, or potentially even in the opposite direction.

Leave-One-Out Analysis

A useful sensitivity analysis is to repeat the meta-analysis after removing each study one at a time.

This can reveal whether:

• One study dominates the pooled result
• One outlier drives heterogeneity
• The pooled estimate is robust

In R, this can be implemented using functions available in the metafor package.

Influence Diagnostics

Random-effects meta-analysis can be sensitive to individual studies, particularly when the number of studies is small.

Influence diagnostics can examine:

• Cook's distance
• Studentized residuals
• Hat values
• Changes in \(\tau^2\)
• Changes in the pooled effect

These diagnostics should identify potentially influential observations, not automatically justify deleting them.

Outliers Are Not Automatically Errors

An unusually large or small study estimate may reflect:

• A data error
• A methodological problem
• A genuinely different population
• A treatment-effect modifier
• Random sampling variation

Removing an outlier solely because it changes the pooled result can introduce bias.

Sensitivity Analysis

A robust meta-analysis often examines whether conclusions change under reasonable alternative assumptions.

Possible sensitivity analyses include:

• Fixed-effect vs. random-effects
• REML vs. alternative \(\tau^2\) estimators
• Hartung-Knapp vs. conventional inference
• Exclusion of high-risk-of-bias studies
• Exclusion of influential studies
• Alternative effect definitions

Publication Bias Is a Separate Issue

The choice between fixed-effect and random-effects models does not solve publication bias.

Potential tools include:

• Funnel plots
• Small-study effect assessments
• Egger-type regression methods
• Selection-model approaches

These methods answer different questions from heterogeneity analysis.

Forest Plot vs. Funnel Plot

R Implementation With metafor

The metafor package provides a flexible framework for fixed-effect and random-effects meta-analysis.

install.packages("metafor")

library(metafor)

Example Dataset

dat <- data.frame(
  study = c(
    "Study A",
    "Study B",
    "Study C",
    "Study D",
    "Study E"
  ),
  yi = c(
    -0.35,
    -0.18,
    -0.05,
     0.08,
     0.22
  ),
  sei = c(
    0.12,
    0.20,
    0.15,
    0.18,
    0.25
  )
)

Fixed-Effect Analysis in R

fit_fe <- rma(
  yi = yi,
  sei = sei,
  data = dat,
  method = "FE"
)

summary(fit_fe)

The model estimates a common underlying effect.

Random-Effects Analysis Using REML

fit_re <- rma(
  yi = yi,
  sei = sei,
  data = dat,
  method = "REML"
)

summary(fit_re)

The model estimates:

• The pooled mean effect
• Between-study variance \(\tau^2\)
• \(I^2\)
• Heterogeneity statistics
• Confidence interval for the pooled mean

Forest Plot in R

forest(
  fit_re,
  slab = dat$study,
  xlab = "Log Hazard Ratio"
)

For a hazard-ratio analysis, the forest plot can be displayed on the original ratio scale.

forest(
  fit_re,
  slab = dat$study,
  atransf = exp,
  xlab = "Hazard Ratio"
)

Prediction Interval in R

For a random-effects model:

predict(
  fit_re,
  transf = exp
)

The output can provide a prediction interval for the effect in a future comparable study.

Estimating Tau-Squared With Different Methods

It is often useful to examine how the estimated heterogeneity changes under alternative estimators.

fit_re_reml <- rma(
  yi,
  sei = sei,
  data = dat,
  method = "REML"
)

fit_re_dl <- rma(
  yi,
  sei = sei,
  data = dat,
  method = "DL"
)

fit_re_pm <- rma(
  yi,
  sei = sei,
  data = dat,
  method = "PM"

)

fit_re_reml$tau2
fit_re_dl$tau2
fit_re_pm$tau2

Hartung-Knapp-Type Inference in R

The metafor framework supports alternative inference choices for random-effects models.

For example, one commonly used approach is:

fit_hk <- rma(
  yi,
  sei = sei,
  data = dat,
  method = "REML",
  test = "knha"
)

summary(fit_hk)

The resulting interval may differ from the conventional normal-based interval, particularly with few studies.

Manual Fixed-Effect Calculation in R

The fixed-effect calculation can be implemented directly to understand the underlying mathematics.

dat$vi <- dat$sei^2

dat$w_fe <- 1 / dat$vi

theta_fe <- sum(
  dat$w_fe * dat$yi
) / sum(dat$w_fe)

se_fe <- sqrt(
  1 / sum(dat$w_fe)
)

ci_fe <- c(
  theta_fe - 1.96 * se_fe,
  theta_fe + 1.96 * se_fe
)

theta_fe
se_fe
ci_fe

Manual Random-Effects Weighting

Suppose, for illustration, that:

tau2 <- 0.04

Then:

dat$w_re <- 1 / (
  dat$vi + tau2
)

theta_re <- sum(
  dat$w_re * dat$yi
) / sum(dat$w_re)

se_re <- sqrt(
  1 / sum(dat$w_re)
)

theta_re

This example illustrates the weighting equation directly.

In a real analysis, \(\tau^2\) should generally be estimated using the prespecified meta-analysis method rather than inserted arbitrarily.

Calculating Normalized Weights

dat$weight_fe_pct <-
  100 * dat$w_fe / sum(dat$w_fe)

dat$weight_re_pct <-
  100 * dat$w_re / sum(dat$w_re)

dat[, c(
  "study",
  "weight_fe_pct",
  "weight_re_pct"
)]

Calculating Cochran's Q Manually

Q <- sum(
  dat$w_fe *
  (dat$yi - theta_fe)^2
)

df <- nrow(dat) - 1

p_Q <- pchisq(
  Q,
  df = df,
  lower.tail = FALSE
)

Q
df
p_Q

Calculating I-Squared

I2 <- max(
  0,
  (Q - df) / Q
) * 100

I2

The resulting value should be interpreted as a descriptive measure of heterogeneity rather than as an automatic model-selection switch.

Data Structure for a Meta-Analysis

A practical analysis dataset might contain:

Data Preparation Workflow

Common Mistake: Choosing the Model After Seeing the Result

Suppose the analyst first calculates the fixed-effect result.

Then the analyst calculates the random-effects result.

If the random-effects result is statistically significant, the analyst chooses it.

If it is not statistically significant, the analyst chooses fixed effect.

This is poor practice because the model has effectively been selected based on the desired conclusion.

Common Mistake: Using Fixed Effect Because I² Is Low

A low \(I^2\) does not automatically prove that all studies share one true effect.

For example, with only three studies: