Tutorials › Bayesian Clinical Trials › Posterior Equivalence Rules Using a Practical-Equivalence Region

Home  /  Tutorials  /  Posterior and Predictive Probability Rules

Posterior and Predictive Probability Rules · Tutorial 147 of 696

Posterior Equivalence Rules Using a Practical-Equivalence Region

You will calculate the posterior probability that a continuous treatment difference lies within a prespecified practical-equivalence region and use it to make a transparent decision.

Intermediate 10 min read

What You'll Learn

  • Define practical equivalence using a symmetric region around zero on the treatment-difference scale.
  • Calculate the posterior probability inside that region from a normal posterior or posterior draws.
  • Apply a prespecified probability threshold without treating it as evidence that the effects are identical.
  • Check Monte Carlo precision and recognize when a decision is sensitive to the equivalence margin.
  • Distinguish posterior equivalence rules from superiority, noninferiority, and frequentist equivalence procedures.

Why Define Equivalence Practically?

A treatment difference can be close to zero without being exactly zero. In a clinical decision, the important question is often whether the difference is small enough to be clinically unimportant, rather than whether it is exactly zero. A posterior equivalence rule makes that question explicit by defining a region of treatment differences regarded as practically equivalent and asking how much posterior probability lies inside it.

Let \(\delta\) denote the treatment effect on a continuous outcome, defined here as the treatment-group mean minus the control-group mean. Positive values therefore indicate a higher mean in the treatment group. The units of \(\delta\) are the outcome’s units. If the direction or definition of the contrast is different in an application, it must be stated before interpreting the region.

Definition: For a prespecified practical-equivalence margin \(\Delta>0\), the symmetric practical-equivalence region is \((-\Delta,\Delta)\). An equivalence decision rule declares the treatments practically equivalent when \(P(-\Delta<\delta<\Delta\mid y)\geq \tau_E\), where \(y\) denotes the observed trial data and \(\tau_E\) is a prespecified posterior-probability threshold.

The margin \(\Delta\) has the same units as the treatment difference. For example, if the outcome is a symptom score measured in points, then \(\Delta=2\) points means that differences strictly between −2 and 2 points are regarded as practically unimportant for this decision. The clinical and scientific rationale for choosing that range should be established independently of the observed treatment estimate.

Calculating the Posterior Probability

The rule requires a posterior distribution for \(\delta\), conditional on the data and the specified statistical model and prior. If that posterior is continuous, the probability at either exact boundary is zero, so using the closed interval \([-\Delta,\Delta]\) gives the same probability. The probability is conditional on the model, prior, and observed data; it is not a probability that the treatment effects are exactly equal.

When the posterior is normal, write \(\delta\mid y\sim N(m,s^2)\), where \(m\) is the posterior mean and \(s\) is the posterior standard deviation. Standardizing each boundary gives a direct calculation using the standard normal cumulative distribution function \(\Phi\).

$$ \begin{aligned} P(-\Delta<\delta<\Delta\mid y) &=P\left(\frac{-\Delta-m}{s}<\frac{\delta-m}{s}<\frac{\Delta-m}{s}\,\middle|\,y\right)\\ &=\Phi\left(\frac{\Delta-m}{s}\right) -\Phi\left(\frac{-\Delta-m}{s}\right). \end{aligned} $$

This formula subtracts the posterior probability below the lower boundary from the probability below the upper boundary. If \(m=0\), symmetry gives the simpler expression \(2\Phi(\Delta/s)-1\). When the posterior is not normal, or when no convenient analytic distribution is available, posterior draws provide a direct estimate: count the draws inside the region and divide by the total number of draws.

Worked Example: A Normal Posterior With High Equivalence Probability

Suppose a hypothetical trial measures a continuous symptom score in points, with lower scores indicating fewer symptoms. The estimated treatment difference is defined as treatment minus control. Assume the posterior distribution is \(\delta\mid y\sim N(0.10,0.20^2)\) points. Clinicians have prespecified \(\Delta=0.50\) points and a decision threshold \(\tau_E=0.95\). The model and prior are assumed to be appropriate for this analysis, and the margin is based on clinical judgment rather than the trial result.

Standardize the upper and lower boundaries:

$$ \frac{0.50-0.10}{0.20}=2.00, \qquad \frac{-0.50-0.10}{0.20}=-3.00. $$

Therefore,

$$ \begin{aligned} P(-0.50<\delta<0.50\mid y) &=\Phi(2.00)-\Phi(-3.00)\\ &\approx 0.9772499-0.0013499\\ &\approx 0.9759. \end{aligned} $$

Because \(0.9759\geq0.95\), this hypothetical trial meets the prespecified posterior equivalence criterion. In context, the model assigns about 97.59% posterior probability to a treatment-minus-control difference between −0.50 and 0.50 symptom-score points. This conclusion is about the defined practical range, not a claim that the true difference is exactly zero.

Using Posterior Draws and Checking Computation

With posterior simulation, let \(\delta^{(1)},\ldots,\delta^{(M)}\) be draws from the posterior distribution. The indicator \(I(-\Delta<\delta^{(m)}<\Delta)\) equals 1 when draw \(m\) lies inside the region and 0 otherwise. Averaging these indicators estimates the desired posterior probability. The estimate is only as reliable as the posterior simulation: convergence, mixing, and effective sample size matter, especially when the probability is near the decision threshold.

$$ \widehat{P}_{\mathrm{eq}} = \frac{1}{M}\sum_{m=1}^{M} I(-\Delta<\delta^{(m)}<\Delta). $$

Worked Example: Applying the Rule to Posterior Draws

In a hypothetical analysis, \(10{,}000\) retained posterior draws represent the treatment-minus-control difference in a continuous laboratory measure, in units per liter. The prespecified equivalence region is \((-1.0,1.0)\) units per liter, and the decision threshold is \(0.80\). Of the draws, \(8{,}426\) lie inside this region. Assume the simulation diagnostics indicate adequate convergence and mixing.

The draw-based estimate is

$$ \widehat{P}_{\mathrm{eq}}=\frac{8{,}426}{10{,}000}=0.8426. $$

Since \(0.8426\geq0.80\), the estimated posterior probability meets the rule. To assess simulation precision, suppose the effective sample size for these indicator values is estimated as \(7{,}200\). An approximate Monte Carlo standard error is

$$ \sqrt{\frac{0.8426(1-0.8426)}{7{,}200}} = \sqrt{\frac{0.13262524}{7{,}200}} \approx 0.0043. $$

This standard error describes simulation error in estimating the posterior probability, not uncertainty about the treatment effect itself. The estimated probability is 0.0426 above the threshold, much larger than this approximate Monte Carlo standard error. The rule is met on the stated computation, subject to the adequacy of the model, prior, and simulation.

Margin Choice and Decision Sensitivity

The probability inside the region depends on both the posterior distribution and the margin. A larger \(\Delta\) includes more possible treatment differences, so its posterior probability cannot decrease as \(\Delta\) increases. That mathematical property does not make a wider margin clinically preferable: the margin defines what counts as practically unimportant. It should be justified and, for a confirmatory decision, specified before examining the trial’s treatment-effect results.

Worked Example: A Decision That Changes With the Margin

Suppose a hypothetical trial has posterior \(\delta\mid y\sim N(0.20,0.25^2)\) points on a continuous outcome, with treatment minus control as the contrast. The prespecified probability threshold is \(0.90\). First consider a clinically justified margin of \(\Delta=0.50\) points. The standardized boundaries are

$$ \frac{0.50-0.20}{0.25}=1.20, \qquad \frac{-0.50-0.20}{0.25}=-2.80. $$

Thus,

$$ \begin{aligned} P(-0.50<\delta<0.50\mid y) &=\Phi(1.20)-\Phi(-2.80)\\ &\approx 0.8849303-0.0025551\\ &\approx 0.8824. \end{aligned} $$

The probability is below \(0.90\), so the criterion is not met with this margin. If, for illustration, the margin were instead \(0.60\) points, the standardized boundaries would be \(1.60\) and \(-3.20\), giving

$$ P(-0.60<\delta<0.60\mid y) =\Phi(1.60)-\Phi(-3.20) \approx 0.9452007-0.0006871 \approx 0.9445. $$

That probability exceeds \(0.90\). The changed decision follows from the wider region, not from new evidence about the treatment effect. This comparison is a sensitivity analysis; it is not a reason to select the wider margin after seeing the result.

Assumptions, Comparisons, and Failure Modes

A valid equivalence decision depends on more than the arithmetic. The treatment effect must represent the intended clinical estimand, including its population, outcome, treatment conditions, and time point. The model and prior must yield a defensible posterior for that estimand. The symmetric region must match the clinical question, and both \(\Delta\) and \(\tau_E\) should be stated in advance for a prespecified decision. If the practical consequences differ in the positive and negative directions, a symmetric region may be inappropriate; the region should reflect the scientific question rather than mathematical convenience.

This rule asks whether the effect is probably small in either direction. A superiority rule instead asks whether the effect is probably beyond a clinically meaningful boundary in a favorable direction. A noninferiority rule focuses on ruling out an unacceptable loss in one direction. These rules answer different questions, even though they all use posterior probabilities and may use clinically motivated margins. A frequentist equivalence procedure, such as two one-sided tests, also has a different inferential framework and decision calibration; its criterion should not be substituted for the posterior probability threshold here.

  • Confusing practical equivalence with exact equality. A continuous posterior assigns probability zero to the single point \(\delta=0\). Equivalence concerns a range of effects, not the probability of exact equality.
  • Choosing the margin after seeing the estimate. Widening \(\Delta\) mechanically raises the probability inside the region and can manufacture a favorable decision. Justify and prespecify the margin.
  • Ignoring Monte Carlo error. If the estimated probability is close to \(\tau_E\), simulation error may affect whether the reported estimate clears the threshold. Check diagnostics and effective sample size, and run further computation when needed.
  • Reporting only “equivalent.” Give the effect scale and direction, the margin, the posterior probability, and the threshold. A decision label without these details is difficult to interpret.
  • Treating the threshold as universal. The value of \(\tau_E\) is a decision convention that should be justified for the intended use; it is not a probability cutoff that is appropriate for every clinical question.

The main limitation is that the rule cannot repair a poorly chosen estimand, an unsuitable model, or a clinically unjustified margin. Its strength is transparency: it turns “small enough to matter little” into a stated range and reports how much posterior probability supports that claim. The posterior probability and the final clinical action should remain distinguishable: the probability is a model-based summary, while the action also depends on the consequences of the available choices.

Key takeaway: A posterior equivalence decision requires a clinically defensible region \((-\Delta,\Delta)\), a posterior probability calculation, and a prespecified threshold \(\tau_E\). Report all three, interpret the probability as conditional on the data, model, and prior, and check computational precision when using posterior draws.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

  1. A hypothetical posterior for a treatment difference in score points is \(N(0,0.25^2)\). For \(\Delta=0.50\) points, calculate the probability inside the region and decide whether a threshold of \(0.95\) is met.
  2. A posterior is \(N(0.40,0.20^2)\) on a continuous outcome measured in units per liter. For \(\Delta=0.50\) units per liter, calculate the equivalence probability and assess a threshold of \(0.90\).
  3. In a draw-based calculation, \(8{,}760\) of \(10{,}000\) posterior draws fall inside the prespecified region. What is the estimated probability, and does it meet a threshold of \(0.90\)?
  4. Why does increasing the equivalence margin tend to increase the posterior probability, and why should the margin not be chosen after examining the trial result?

Solutions. 1. The standardized boundaries are \(2\) and \(-2\), so the probability is \(\Phi(2)-\Phi(-2)\approx0.9545\). It meets the \(0.95\) threshold. 2. The standardized boundaries are \((0.50-0.40)/0.20=0.50\) and \((-0.50-0.40)/0.20=-4.50\). The probability is \(\Phi(0.50)-\Phi(-4.50)\approx0.6915\), so it does not meet \(0.90\). 3. The estimate is \(8{,}760/10{,}000=0.8760\), below \(0.90\), so the estimated criterion is not met. 4. A wider interval contains the narrower interval, so it cannot have less posterior probability. Choosing it after seeing the result can redefine what counts as negligible to obtain a preferred outcome, rather than applying a clinically justified decision rule.

Previous

Next

Back to Tutorials
Clinical Biostats