Tutorials › Bayesian Clinical Trials › Posterior Decision Rules for Either-Or Endpoint Criteria

Posterior and Predictive Probability Rules · Tutorial 149 of 696

Posterior Decision Rules for Either-Or Endpoint Criteria

Learn how to construct and evaluate a posterior success rule when meeting any one of several endpoint criteria is sufficient.

Intermediate 10 min read

What You'll Learn

  • Define endpoint-specific posterior events and combine them into an either-or success event.
  • Calculate the posterior probability of a union using overlap or joint posterior draws.
  • Explain why separate marginal probabilities do not determine the probability of either-or success.
  • Estimate Monte Carlo error for a union probability calculated from posterior simulation.
  • Check whether endpoint definitions, dependence, and simulation diagnostics support the decision rule.

When Success Means Meeting Any One Criterion

A clinical trial may assess more than one clinically relevant endpoint, while its decision rule may require evidence for at least one of several prespecified benefits. For example, a treatment might qualify by exceeding a meaningful benefit threshold on either symptom improvement or physical function. This is an either-or rule: the trial’s posterior success event is the union of the endpoint-specific events.

The distinction between a union and separate marginal probabilities is central. Knowing the posterior probability that each criterion is met does not, by itself, tell us the probability that at least one is met. The criteria can overlap in the posterior distribution, and the amount of overlap changes the union probability. The joint posterior distribution, not just a list of marginal probabilities, is therefore needed to evaluate the rule.

Definition: Let \(A_j\) be the posterior event that endpoint criterion \(j\) is satisfied, for \(j=1,\ldots,K\). An either-or success event is \(A=\bigcup_{j=1}^{K} A_j\): at least one criterion is satisfied. Given observed data \(D\), its posterior success probability is \(q(D)=P(A\mid D)\). A prespecified posterior decision rule declares that the either-or criterion is met when \(q(D)\geq \gamma\), where \(\gamma\) is the chosen posterior probability threshold.

Define the Events Before Calculating

Each \(A_j\) must express a clinically interpretable criterion for a specified estimand. For example, let \(\theta_1\) be the treatment benefit on a symptom scale, with positive values favoring treatment, and let \(\theta_2\) be the treatment benefit on a function scale, also oriented so positive values favor treatment. If the minimum clinically meaningful benefits are \(m_1\) and \(m_2\), then the endpoint events could be \(A_1=\{\theta_1>m_1\}\) and \(A_2=\{\theta_2>m_2\}\). The either-or event is \(A_1\cup A_2\). The criteria should also specify the relevant population, analysis time, and endpoint definitions.

The threshold \(\gamma\) and the endpoint criteria should be prespecified. Choosing criteria after examining the posterior can make the rule look more favorable than the rule actually evaluated. The posterior probability threshold is a feature of this decision rule; it is not automatically a frequentist significance level or a universal standard.

The posterior model must provide a joint distribution for the endpoint parameters. It should represent dependence induced by shared participants, common controls, correlated outcomes, and any other relevant features of the design and analysis. The marginal posterior for each \(\theta_j\) may be useful, but it is not enough to calculate a union probability unless the overlap between the events is also known.

Worked Example: Calculate a Two-Criterion Union

Suppose a hypothetical trial has two prespecified criteria, \(A_1\) and \(A_2\). In 1,000 posterior draws from the joint model, criterion \(A_1\) is met in 620 draws, criterion \(A_2\) in 540 draws, and both are met in 380 draws. The posterior threshold is \(\gamma=0.75\).

For two events, the probability of their union is the sum of their probabilities minus the probability of their intersection. Using the posterior draw frequencies,

$$ P(A_1\cup A_2\mid D) = P(A_1\mid D)+P(A_2\mid D)-P(A_1\cap A_2\mid D). $$

Substituting the counts gives

$$ \widehat{P}(A_1\cup A_2\mid D) = \frac{620}{1000}+\frac{540}{1000}-\frac{380}{1000} =0.620+0.540-0.380 =0.780. $$

The estimated posterior probability of meeting at least one criterion is 0.780, which exceeds 0.75. Thus the specified either-or rule declares the criterion met. Neither marginal probability alone reaches 0.75, but that is not the rule being evaluated: the rule concerns the union.

Why Marginal Probabilities Are Not Enough

For two criteria, the inclusion-exclusion identity follows by partitioning the union into disjoint events: \(A_1\cup A_2\) consists of \(A_1\) and the part of \(A_2\) not already in \(A_1\). Therefore,

$$ P(A_1\cup A_2\mid D) = P(A_1\mid D)+P(A_2\cap A_1^c\mid D) = P(A_1\mid D)+P(A_2\mid D)-P(A_1\cap A_2\mid D). $$

The intersection term is what prevents double-counting posterior draws in which both criteria are satisfied. If the intersection is unknown, the marginals alone do not give an exact union probability. For \(K\) criteria, inclusion-exclusion continues with pairwise intersections, triple intersections, and higher-order intersections. In practice, calculating the union directly from joint posterior draws is often simpler and less error-prone.

Worked Example: Same Marginals, Different Union Probabilities

Consider two hypothetical sets of 1,000 joint posterior draws. In each set, \(A_1\) occurs in 550 draws and \(A_2\) occurs in 500 draws. In the first set, both occur in 300 draws; in the second, both occur in 450 draws. Apply a posterior threshold of 0.70.

In the first set, the number of draws meeting at least one criterion is \(550+500-300=750\). Equivalently,

$$ \widehat{P}(A_1\cup A_2\mid D) = 0.550+0.500-0.300 = 0.750. $$

This exceeds 0.70. In the second set, there are \(550+500-450=600\) draws meeting at least one criterion, so

$$ \widehat{P}(A_1\cup A_2\mid D) = 0.550+0.500-0.450 = 0.600. $$

This does not reach 0.70. Both sets have the same marginal probabilities, but their overlap differs, so their union probabilities and rule outcomes differ. The example illustrates why treating either marginal as a substitute for the joint either-or probability can be misleading.

Estimating the Union From Posterior Draws

Suppose the fitted model provides \(S\) draws \(\theta^{(1)},\ldots,\theta^{(S)}\) from the joint posterior distribution of all parameters needed to assess the criteria. For each draw, evaluate every criterion, then record whether at least one is met. The posterior probability is estimated by the fraction of draws for which the union event occurs.

$$ \widehat{q} = \frac{1}{S}\sum_{s=1}^{S} I\!\left\{\theta^{(s)}\in\bigcup_{j=1}^{K}A_j\right\}, $$

where \(I\{\cdot\}\) equals 1 when the stated event is true and 0 otherwise. This calculation automatically accounts for overlap, provided each draw represents the joint posterior. It also avoids needing to calculate all intersection probabilities separately.

Worked Example: Check a Multi-Endpoint Rule and Its Simulation Error

In a hypothetical two-endpoint trial, define \(A_1\) as a symptom benefit greater than 2 points and \(A_2\) as a function benefit greater than 5 points. Both benefits are oriented so larger values favor treatment. Among 2,000 joint posterior draws, \(A_1\) occurs 1,240 times, \(A_2\) occurs 1,100 times, and both occur 920 times. The prespecified threshold is \(\gamma=0.70\).

The union occurs in \(1,240+1,100-920=1,420\) draws. Therefore,

$$ \widehat{q} = \frac{1420}{2000} = 0.710. $$

The estimated posterior probability is just above the threshold, so the rule is met based on this estimate. Because the draws may be autocorrelated, the number of draws alone does not determine the Monte Carlo precision. Suppose the effective sample size for the union-event indicator is estimated as 900. An approximate Monte Carlo standard error is

$$ \widehat{SE}_{MC} \approx \sqrt{\frac{\widehat{q}(1-\widehat{q})}{900}} = \sqrt{\frac{0.710(0.290)}{900}} \approx 0.0151. $$

This Monte Carlo error describes uncertainty in the numerical estimate caused by finite simulation; it is not the posterior uncertainty in the endpoint effects. Since the estimate is close to the threshold relative to this simulation error, the analyst should improve or check the precision of the computation before treating the numerical comparison as stable.

Assumptions, Diagnostics, and Alternatives

A meaningful union rule requires more than correct arithmetic. The endpoint criteria must be clinically coherent, their directions and units must be correct, and the posterior model must be suitable for the trial data and estimands. The joint posterior must preserve relevant dependence. If posterior draws were generated separately for each endpoint and then paired arbitrarily, their apparent overlap would not generally represent the true joint posterior.

For simulation-based evaluation, inspect convergence and mixing for the parameters and for the union-event indicator itself. A chain may explore endpoint parameters adequately overall yet move slowly between regions where different criteria are met. Check chain agreement and effective sample size, and assess Monte Carlo error when the estimate is near \(\gamma\). Also verify that the event coding matches the prespecified rule: a reversed benefit direction, an incorrect margin, or an accidental requirement that all criteria be met can change the result.

An either-or rule differs from a co-primary rule, where success requires all specified criteria to be met. It also differs from reporting several marginal posterior probabilities without combining them into a single rule. A composite endpoint is another alternative: it defines one outcome or estimand rather than the union of separate endpoint-specific events. These approaches answer different clinical questions and should not be interchanged after seeing the results.

For many criteria, the union can be broad: adding another criterion can only leave the union probability unchanged or increase it. That is a mathematical property, not evidence that each endpoint is independently persuasive. Prespecification, transparent endpoint reporting, and evaluation of the full joint event help prevent an expanding list of opportunities from being mistaken for a stronger single-endpoint result.

Common pitfall: “The posterior probability is 0.72 for endpoint 1 and 0.68 for endpoint 2, so the probability that at least one criterion is met is 1.40.” This adds the marginal probabilities and double-counts draws where both criteria are met. The correct calculation subtracts the intersection, or directly counts draws satisfying at least one criterion.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

  1. In 1,000 joint posterior draws, \(A_1\) occurs 400 times, \(A_2\) occurs 350 times, and both occur 150 times. What is the estimated posterior probability of \(A_1\cup A_2\)?
  2. Why can two posterior analyses with the same marginal probabilities for \(A_1\) and \(A_2\) produce different probabilities for \(A_1\cup A_2\)?
  3. A trial’s either-or rule has \(\gamma=0.80\). The estimated posterior probabilities are 0.76 for \(A_1\), 0.73 for \(A_2\), and 0.60 for \(A_1\cap A_2\). Does the union rule meet its threshold?
  4. What is the main risk of evaluating an either-or event using separately sampled endpoint posteriors that are then paired arbitrarily?

Solutions. (1) The union probability is \(0.400+0.350-0.150=0.600\). (2) The intersection probability can differ even when the two marginals are identical; since the union equals the sum of the marginals minus their intersection, the union changes with the overlap. (3) The union probability is \(0.76+0.73-0.60=0.89\), which exceeds 0.80. (4) Arbitrary pairing does not preserve the joint posterior dependence, so it can give an incorrect overlap and therefore an incorrect union probability.

Clinical Biostats