Bayesian Predictive Probability for Trial Monitoring

These values are illustrative rather than universal.

Frequentist Operating Characteristics Still Matter

A Bayesian design can be specified using posterior and predictive probabilities, but that does not mean frequentist operating characteristics should be ignored.

For a confirmatory or highly consequential trial, investigators may evaluate the probability of incorrect decisions under repeated sampling.

For example:

• Probability of early stopping for efficacy under an ineffective treatment
• Probability of early stopping for futility under an effective treatment
• Overall probability of final success
• Overall type I error
• Power
• Expected sample size
• Probability of stopping at each interim analysis

These quantities are generally estimated by simulation.

Bayesian Decision Rules Need Operating-Characteristic Evaluation

Suppose a trial has three interim analyses. At each analysis, predictive probability is calculated. The trial may stop when:

or:

Because the rule is applied repeatedly, its long-run operating characteristics depend on the entire sequence of monitoring decisions.

Therefore, the design should be simulated across plausible values of the true treatment effect.

Simulation-Based Design Evaluation

A typical simulation proceeds as follows:

Prior Sensitivity

Bayesian predictive probability depends on the prior distribution. Therefore, prior sensitivity should be evaluated.

Suppose three priors are considered:

With only a small amount of interim information, these priors may produce meaningfully different predictive probabilities.

As the sample size increases, the likelihood generally contributes more information and the influence of a weak prior decreases.

Prior Predictive Checks

Before the trial begins, investigators can examine what the prior implies about the response rate. For a Beta(\(a,b\)) prior:

and:

A prior should generate clinically plausible response rates.

This is often more informative than simply describing a prior as "noninformative."

Information Borrowing

In some Bayesian clinical trials, historical data may be incorporated into the prior. For example:

where the parameters represent information derived from historical studies.

The posterior then becomes:

This can make the predictive probability more responsive to historical information, particularly early in the trial.

Multiple Interim Analyses

Bayesian predictive monitoring can be performed repeatedly. For example, a 100-patient trial might have interim analyses after:

• 25 patients
• 50 patients
• 75 patients
• 100 patients

At each interim analysis, the predictive probability can be recalculated.

The important point is that the stopping rules should be established before observing the interim results.

Predictive Probability Can Be Used for Futility Without Proving Harm

Suppose the predictive probability of eventual success is only 3%. That does not necessarily mean:

Those are different statements.

A low predictive probability says that final success is unlikely. It does not necessarily quantify the posterior probability of a particular alternative or null hypothesis.

A More General Success Criterion

The final success criterion does not have to be "at least \(c\) responses." For example, suppose success is defined by:

Then predictive probability can be defined as the probability that the final analysis will satisfy the decision criterion.

For more complicated endpoints, the final decision might depend on:

• A treatment effect exceeding a clinically meaningful difference
• A posterior probability exceeding a threshold
• A hazard ratio below a target
• A response rate exceeding a benchmark
• A credible interval satisfying a prespecified condition
• A composite endpoint

Predictive Probability for Continuous Outcomes

Suppose the primary endpoint is continuous. Let the final success criterion be:

The Bayesian model produces a posterior distribution for \(\Delta\). Future observations can then be simulated from the posterior predictive distribution.

The predictive probability becomes:

The same conceptual framework therefore applies even when a beta-binomial closed form is unavailable.

Predictive Probability for Time-to-Event Endpoints

For survival studies, predictive monitoring can incorporate both currently observed events and future events. For example, a trial might ultimately require:

or a specified number of events.

A posterior survival model can be used to simulate future event times and censoring patterns. The predictive probability is then the proportion of simulated completed trials satisfying the final success criterion.

Predictive Probability in Hierarchical Models

Bayesian predictive monitoring becomes particularly powerful in hierarchical models. Suppose several centers or subgroups have:

and the study contains a common population-level distribution. The posterior distribution can borrow information across groups. Future observations are then generated from the posterior predictive distribution of the hierarchical model.

The conceptual monitoring rule remains unchanged:

R Implementation: Beta-Binomial Example

The worked example can be implemented using base R. First specify the design:

a <- 1
b <- 1

n_interim <- 20
y_interim <- 8

N_final <- 40
success_threshold <- 12

n_future <- N_final - n_interim

p_eff <- 0.90
p_fut <- 0.10

Calculate the Posterior Parameters

a_post <- a + y_interim
b_post <- b + n_interim - y_interim

a_post
b_post

The result is:

a_post
# 9

b_post
# 13

Therefore:

Calculate the Predictive Probability Directly

Base R does not provide a beta-binomial distribution function in the standard distribution package, so we can calculate the predictive probability directly using the beta function.

required_future <-
  success_threshold - y_interim

x <- required_future:n_future

predictive_probs <-
  choose(n_future, x) *
  beta(x + a_post,
       n_future - x + b_post) /
  beta(a_post, b_post)

PP <- sum(predictive_probs)

PP

The result is approximately:

PP
# 0.9482

Thus:

Implementing a General Predictive Probability Function

predictive_probability <- function(
    y,
    n,
    N,
    threshold,
    a = 1,
    b = 1
) {

  a_post <- a + y
  b_post <- b + n - y

  n_future <- N - n

  required <- threshold - y

  if (required <= 0) {
    return(1)
  }

  if (required > n_future) {
    return(0)
  }

  x <- required:n_future

  probs <- choose(n_future, x) *
    beta(
      x + a_post,
      n_future - x + b_post
    ) /
    beta(a_post, b_post)

  sum(probs)
}

The worked example can then be calculated with:

predictive_probability(
  y = 8,
  n = 20,
  N = 40,
  threshold = 12
)

which gives approximately:

# 0.9482

Apply the Monitoring Decision Rule in R

PP <- predictive_probability(
  y = 8,
  n = 20,
  N = 40,
  threshold = 12
)

decision <- ifelse(
  PP <= p_fut,
  "Stop for futility",
  ifelse(
    PP >= p_eff,
    "Stop for efficacy",
    "Continue"
  )
)

PP
decision

For the worked example, the output is approximately:

PP
# 0.9482

decision
# "Stop for efficacy"

Evaluate Predictive Probability Across Interim Response Counts

A useful design-development exercise is to calculate the predictive probability for every possible number of responses at the interim analysis.

results <- data.frame(
  responses = 0:n_interim
)

results$predictive_probability <-
  sapply(
    results$responses,
    function(y) {
      predictive_probability(
        y = y,
        n = n_interim,
        N = N_final,
        threshold = success_threshold
      )
    }
  )

results

This produces a complete mapping from the observed interim response count to the corresponding predictive probability.

The Monitoring Decision Can Be Expressed as Response Boundaries

Once the predictive-probability thresholds are fixed, the Bayesian decision rule may correspond to specific response-count boundaries. For example:

This is an important practical point. A Bayesian monitoring rule may look probabilistic mathematically, but for a simple binomial endpoint the resulting decision can often be translated into integer response-count boundaries at each interim analysis.

Why Predictive Probability Changes During the Trial

Early in the trial, many outcomes remain possible. Therefore, predictive probability can be highly uncertain. As more patients are observed, fewer future outcomes remain capable of changing the final decision.

Consequently, predictive probability often becomes more extreme as information accumulates.

Predictive Probability and Sample Size Reassessment

Bayesian predictive probability can also be used in adaptive sample-size designs. For example, an interim analysis might determine that:

• The trial is very unlikely to succeed.
• The trial is highly likely to succeed.
• The trial remains uncertain and may benefit from additional information.

A prespecified rule might allow the sample size to increase in the uncertain region.

This creates a framework for:

Such adaptations require careful simulation because changing the maximum sample size changes the predictive probability calculation itself.

Predictive Probability and Adaptive Enrichment

In more advanced trials, predictive probability can be calculated separately for patient subgroups. For example:

might be compared with:

to determine whether a particular population appears more likely to meet the clinical success criterion.

Such designs require hierarchical modeling and prespecified decision rules to avoid ad hoc subgroup selection.

Simulation of a Bayesian Monitoring Design

The following R code illustrates the basic structure of a simulation.

simulate_trial <- function(
    p_true,
    N = 40,
    interim_n = 20,
    threshold = 12,
    a = 1,
    b = 1,
    futility = 0.10,
    efficacy = 0.90
) {

  y <- rbinom(
    1,
    interim_n,
    p_true
  )

  PP <- predictive_probability(
    y = y,
    n = interim_n,
    N = N,
    threshold = threshold,
    a = a,
    b = b
  )

  if (PP <= futility) {
    return("Futility")
  }

  if (PP >= efficacy) {
    return("Efficacy")
  }

  y_future <- rbinom(
    1,
    N - interim_n,
    p_true
  )

  y_final <- y + y_future

  if (y_final >= threshold) {
    "Final success"
  } else {
    "Final failure"
  }
}

Repeat the Simulation

set.seed(123)

sim_results <- replicate(
  10000,
  simulate_trial(
    p_true = 0.30
  )
)

table(sim_results)

prop.table(
  table(sim_results)
)

Repeating this process under different true response rates allows investigators to estimate the operating characteristics of the Bayesian monitoring design.

Operating Characteristics to Examine

A serious design evaluation should generally examine more than the predictive probability threshold itself. Important quantities include:

• Probability of stopping for efficacy
• Probability of stopping for futility
• Probability of reaching the final analysis
• Probability of final success
• Expected sample size
• Maximum sample size
• Probability of each possible interim decision
• Trial duration
• Sensitivity to the prior

Evaluating the Design Under the Null

Suppose the clinically uninteresting response rate is:

Simulating thousands of trials under \(p=0.10\) allows investigators to estimate how frequently the Bayesian rule incorrectly declares efficacy.

This quantity is analogous to a frequentist type I error, although the Bayesian decision rule itself is defined using posterior or predictive probabilities.

Evaluating the Design Under the Alternative

Now suppose the clinically meaningful response rate is:

The simulation estimates how frequently the Bayesian monitoring design leads to eventual success.

This provides a frequentist operating characteristic corresponding to the Bayesian decision rule.

Common Mistake: Confusing Predictive Probability with Observed Probability

Suppose eight of 20 patients respond. It is incorrect to state:

The 40% is the observed interim response rate. It is not a probability of final success.

The Bayesian predictive probability in our example is approximately:

because the calculation incorporates the possible outcomes of the remaining 20 patients.

Common Mistake: Using the Posterior Mean as the True Response Rate

Another common mistake is:

followed by an ordinary binomial calculation.

This ignores posterior uncertainty. The correct calculation is:

Common Mistake: Ignoring the Prior

A Bayesian predictive probability cannot be interpreted independently of its prior. Two analysts using identical interim data but different priors may obtain different predictive probabilities.

Therefore, the statistical analysis plan should document:

• The prior distribution
• The rationale for the prior
• Any historical data used
• The prior sensitivity analyses
• The predictive probability calculation
• The monitoring thresholds

Common Mistake: Treating 90% as a Universal Bayesian Threshold

There is no universal rule that Bayesian efficacy monitoring must use:

Likewise, there is no universal requirement that futility use:

These are design parameters. Their appropriateness depends on the consequences of false efficacy and false futility decisions, the prior, the number and timing of interim analyses, and the clinical context.

Common Mistake: Forgetting That Monitoring Is Sequential

If predictive probability is evaluated repeatedly, the overall behavior of the trial depends on all the interim analyses.

For example, a trial monitored at 20, 30, 40, and 50 patients has more opportunities to stop early than a trial monitored only once.

The complete monitoring schedule therefore belongs in the design simulation.

Common Mistake: Changing the Prior After Seeing the Data

The prior should be specified independently of the observed trial outcomes. Changing the prior after seeing interim results because it produces a more favorable or unfavorable predictive probability undermines the prespecified operating characteristics.

Common Mistake: Ignoring Missing Data

The simple beta-binomial example assumes that each patient contributes a well-defined binary outcome. Real clinical trials may include:

• Missing assessments
• Loss to follow-up
• Early treatment discontinuation
• Delayed responses
• Unevaluable patients

The Bayesian model must specify how these observations are handled.

For more complex missing-data mechanisms, predictive probability may need to integrate over missing outcomes as well as the treatment-effect parameter.

Predictive Probability with Missing Outcomes

Suppose some interim patients have not yet reached their response assessment time. Rather than simply discarding those patients, a Bayesian model can potentially represent the uncertainty about their eventual outcomes.

The predictive calculation can then incorporate:

This can be particularly useful in trials where response assessment is delayed.

Bayesian Predictive Probability and Data Maturity

Enrollment and information availability are not always the same thing. A trial may have enrolled 50 patients while only 35 have mature primary endpoint assessments.

A monitoring plan should therefore specify whether interim analyses are based on:

• Number enrolled
• Number evaluable
• Number with mature endpoint data
• Number of events
• A calendar-based information fraction

The predictive model should reflect the actual information structure of the trial.

Bayesian Predictive Probability and Randomization

The same framework can be extended to randomized trials. Suppose:

is the treatment-control response difference. The posterior distribution of \(\Delta\) can be used to generate future outcomes in both treatment groups.

The predictive probability might then be:

where \(\delta\) is the clinically meaningful treatment difference.

For more complicated randomized designs, posterior simulation is generally more practical than closed-form calculations.

Bayesian Predictive Probability and Clinical Relevance

The final success criterion should be clinically meaningful. For example, a trial should not define success simply because:

The predictive probability threshold is only part of the decision rule. The underlying target should reflect the clinical question.

For example:

or:

may define the scientifically meaningful objective.

A Practical Bayesian Monitoring Workflow

What Should Be Specified in the Protocol?

A Bayesian predictive monitoring design should be sufficiently detailed that another statistician could reproduce the decision. At minimum, specify:

• Primary endpoint
• Statistical model
• Prior distribution
• Rationale for the prior
• Historical information incorporated into the prior
• Interim analysis timing
• Final sample size
• Final success criterion
• Definition of predictive probability
• Efficacy predictive-probability threshold
• Futility predictive-probability threshold
• Handling of missing data
• Handling of unevaluable patients
• Posterior computation method
• Predictive simulation method, if applicable
• Prior sensitivity analyses
• Operating-characteristic simulations

Worked Example Summary

The Most Important Concept

The most important conceptual point is that Bayesian predictive probability looks forward, not merely backward.

An interim response rate describes what has already happened. A posterior distribution describes what is believed about the unknown parameter after observing those data. A posterior predictive distribution describes what may happen in the future, conditional on the current information.

Predictive probability then asks whether those future outcomes are likely to produce the final clinical-trial success criterion.

For a simple beta-binomial model, this can be calculated exactly. For complex clinical trials, the same concept can be implemented using posterior simulation and Monte Carlo methods.

References

Berry, D.A. (2006). Bayesian clinical trials. Nature Reviews Drug Discovery, 5, 27–36.
Berry, S.M., Carlin, B.P., Lee, J.J. & Muller, P. (2010). Bayesian Adaptive Methods for Clinical Trials. CRC Press.
Gelman, A., Carlin, J.B., Stern, H.S., Dunson, D.B., Vehtari, A. & Rubin, D.B. (2013). Bayesian Data Analysis, 3rd ed. CRC Press.
Spiegelhalter, D.J., Abrams, K.R. & Myles, J.P. (2004). Bayesian Approaches to Clinical Trials and Health-Care Evaluation. Wiley.
Thall, P.F., Simon, R. & Estey, E.H. (1995). Bayes' statistical approach to Phase II clinical trials in small populations. Journal of Clinical Oncology, 13, 2116–2124.
Thall, P.F. & Simon, R. (1994). Recent developments in the design of phase II clinical trials. Current Opinion in Oncology, 6, 371–376.
Lee, J.J. & Liu, D.D. (2008). A predictive probability approach to interim monitoring of clinical trials. Clinical Trials, 5, 491–498.