Posterior Noninferiority Rules With a Prespecified Margin

What a Posterior Noninferiority Rule Says

A noninferiority question asks whether a new treatment’s effect is not unacceptably worse than a comparator. The word “unacceptably” must be made quantitative before the trial is interpreted: a prespecified margin describes the largest loss in treatment effect that would still be considered clinically acceptable. A posterior noninferiority rule assesses how much posterior probability lies on the acceptable side of that margin.

The direction of the treatment effect matters. In this tutorial, $\theta$ is defined so that larger values favor the new treatment. For an endpoint where a larger value is better, $\theta$ can be the treatment-minus-control difference. For an endpoint where a smaller value is better, define or transform the effect so that larger $\theta$ still represents a more favorable treatment effect. This convention keeps the decision rule consistent.

Definition: Let $\theta$ be a treatment effect oriented so that larger values are more favorable, and let $M>0$ be a clinically justified, prespecified noninferiority margin in the same units as $\theta$. The treatment is noninferior at posterior probability threshold $q$ when $$ P(\theta>-M\mid \text{data})\geq q. $$ The event $\theta>-M$ means that the new treatment’s disadvantage, if any, is smaller than the maximum acceptable loss $M$. The threshold $q$ is part of the decision rule; it is not automatically a frequentist significance level.

If the posterior distribution is continuous, using $\theta\geq-M$ instead of $\theta>-M$ gives the same probability because $P(\theta=-M\mid\text{data})=0$. If the posterior has a point mass at the boundary, the choice of strict or inclusive boundary must instead be specified. The probability is conditional on the model, prior, and observed data; it is not a claim that the treatment is noninferior with certainty.

Formulating the Effect and Margin

For a favorable binary outcome, such as recovery by a specified time, a natural effect is the difference in success probabilities, $\theta=p_T-p_C$. Positive values favor treatment. If up to 10 percentage points fewer successes is acceptable, then $M=0.10$ and noninferiority concerns $p_T-p_C>-0.10$.

For an adverse binary outcome, such as a complication, lower risk is better. One convenient favorable orientation is $\theta=p_C-p_T$, where $p_T$ and $p_C$ are the treatment and control event probabilities. Positive values then represent a reduction in risk. A margin of $M=0.08$ means that an increase in treatment risk of up to 8 percentage points is the largest tolerated loss, so the event of interest remains $\theta>-0.08$.

For a continuous outcome, if larger scores are better, define $\theta=\mu_T-\mu_C$, the difference in population means. A margin of $M=5$ score points gives the event $\mu_T-\mu_C>-5$. If lower scores are better, reverse the subtraction or change the sign so that the favorable direction remains positive. State the endpoint, treatment-effect definition, direction, margin, and posterior threshold together; a margin without these details is ambiguous.

Important assumption: The clinical meaning of the endpoint and estimand must support the proposed margin. The margin is not chosen after seeing the observed treatment effect to make the posterior probability pass a threshold. The analysis also depends on an adequate likelihood, prior, and treatment-effect model for the population and trial data being analyzed.

Worked Examples

Worked Example: Binary Success Outcome

Suppose a hypothetical trial records recovery by day 30. There are 100 participants in each group: 82 recoveries with the new treatment and 78 with control. Define $\theta=p_T-p_C$, so a positive difference favors the new treatment. Use independent $\mathrm{Beta}(1,1)$ priors for $p_T$ and $p_C$, and prespecify $M=0.10$ and $q=0.95$. Assume independent binomial sampling within groups and no clustering or other dependence that would invalidate that likelihood.

By beta-binomial conjugacy, the posterior distributions are $p_T\mid\text{data}\sim\mathrm{Beta}(83,19)$ and $p_C\mid\text{data}\sim\mathrm{Beta}(79,23)$. Their posterior means are $$ E(p_T\mid\text{data})=\frac{83}{102}=0.8137,\qquad E(p_C\mid\text{data})=\frac{79}{102}=0.7745. $$ Thus the posterior mean difference is $0.8137-0.7745=0.03922$, or about 3.92 percentage points in favor of treatment.

For a transparent numerical approximation, approximate each beta posterior by a normal distribution with the same mean and variance, and approximate the difference by a normal distribution. For a $\mathrm{Beta}(a,b)$ variable, the variance is $ab/[(a+b)^2(a+b+1)]$. Therefore, $$ \operatorname{Var}(p_T\mid\text{data})=\frac{83(19)}{102^2(103)}=0.001472,\qquad \operatorname{Var}(p_C\mid\text{data})=\frac{79(23)}{102^2(103)}=0.001696. $$ The independent posteriors give an approximate standard deviation for the difference of $$ \sqrt{0.001472+0.001696}=0.05628. $$ The posterior probability of noninferiority is consequently approximated by $$ P(\theta>-0.10\mid\text{data}) \approx \Phi\!\left(\frac{0.03922-(-0.10)}{0.05628}\right) =\Phi(2.474)\approx 0.9933. $$ Under the specified model and approximation, the posterior probability that the treatment reduces recovery probability by less than 10 percentage points is about 0.993. This exceeds the prespecified threshold 0.95, so the rule declares noninferiority. The conclusion is conditional on the model and prior, and the normal approximation should be checked against posterior simulation when the decision is sensitive to approximation error.

Worked Example: Continuous Outcome

Consider a hypothetical continuous symptom score measured in points, where higher scores indicate better health. Let $\theta=\mu_T-\mu_C$. The observed difference in sample means is $-2.0$ points, with a known standard error of 2.0 points under the assumed sampling model. For illustration, use the likelihood $\bar{x}_T-\bar{x}_C\mid\theta\sim N(\theta,2^2)$ and prior $\theta\sim N(0,10^2)$. Assume the sampling model is appropriate and the prior was selected without using the trial result. Prespecify a margin of $M=5$ points and a threshold $q=0.95$.

The normal-normal update gives posterior precision equal to the sum of prior and likelihood precisions. The posterior variance and mean are $$ V=\left(\frac{1}{10^2}+\frac{1}{2^2}\right)^{-1}=3.846,\qquad m=V\left(\frac{0}{10^2}+\frac{-2.0}{2^2}\right)=-1.923. $$ Thus $\theta\mid\text{data}\sim N(-1.923,3.846)$, where the second argument is the variance and the posterior standard deviation is $\sqrt{3.846}=1.961$ points. The noninferiority probability is $$ P(\theta>-5\mid\text{data}) =1-\Phi\!\left(\frac{-5-(-1.923)}{1.961}\right) =\Phi(1.569)\approx 0.9417. $$ The posterior probability that the new treatment’s mean score is no more than 5 points worse than control is approximately 0.942. This falls below the prespecified 0.95 threshold, so this rule does not declare noninferiority. That result is not proof that the treatment is inferior; it says only that the chosen posterior-probability criterion was not met.

Worked Example: Adverse-Event Risk Difference

In a separate hypothetical trial, 12 of 100 treatment participants and 15 of 100 control participants experience an adverse event. Lower event risk is better, so define the favorable effect as $\theta=p_C-p_T$. Use independent $\mathrm{Beta}(0.5,0.5)$ priors for the event probabilities, and prespecify an allowable increase in event risk of $M=0.08$ and a threshold $q=0.95$. The binomial likelihood assumes independent participants within each group.

The posterior distributions are $p_T\mid\text{data}\sim\mathrm{Beta}(12.5,88.5)$ and $p_C\mid\text{data}\sim\mathrm{Beta}(15.5,85.5)$. Their means are $12.5/101=0.1238$ and $15.5/101=0.1535$, so the posterior mean of $\theta=p_C-p_T$ is $0.02970$. The posterior variances are $$ \frac{12.5(88.5)}{101^2(102)}=0.001063,\qquad \frac{15.5(85.5)}{101^2(102)}=0.001273. $$ Using an independent-normal approximation for the two beta posteriors, the difference has standard deviation $\sqrt{0.001063+0.001273}=0.04834$. Therefore, $$ P(\theta>-0.08\mid\text{data}) \approx \Phi\!\left(\frac{0.02970+0.08}{0.04834}\right) =\Phi(2.269)\approx 0.9884. $$ The approximate probability that the treatment’s adverse-event risk is not more than 8 percentage points higher than control is 0.988, which exceeds 0.95. Under this rule, noninferiority is declared. Because the probability is approximated, posterior draws from the beta distributions provide a useful check, especially if the result is close to the decision threshold.

Why the Probability Rule Works

A posterior probability for noninferiority is the posterior mass of a specified region. If the posterior density of $\theta$ is $f(\theta\mid\text{data})$, then $$ P(\theta>-M\mid\text{data}) =\int_{-M}^{\infty} f(\theta\mid\text{data})\,d\theta. $$ The decision follows by comparing this probability with the prespecified threshold $q$. This calculation does not require the posterior to be normal; normality is needed only for the normal approximations used in some examples.

When posterior draws are available, the same probability can be estimated directly. For $S$ draws $\theta^{(1)},\ldots,\theta^{(S)}$, count the draws above the boundary: $$ \widehat{P}=\frac{1}{S}\sum_{s=1}^{S} I\!\left(\theta^{(s)}>-M\right), $$ where $I(\cdot)$ equals 1 when its condition is true and 0 otherwise. For independent draws, the Monte Carlo standard error is approximately $\sqrt{\widehat{P}(1-\widehat{P})/S}$; for correlated Markov chain Monte Carlo draws, effective sample size, not the raw number of draws, should guide Monte Carlo precision. A decision near $q$ warrants particular attention to simulation error.

Assumptions, Diagnostics, and Alternatives

A defensible rule requires more than a numerical comparison. The estimand must match the clinical question, the effect direction must be consistent, and the margin must be justified clinically and fixed before examining the result. The likelihood must reflect the design: for example, a simple binomial model may be inadequate with clustering, substantial missing data, or repeated observations. The prior should be examined for whether it meaningfully affects probability near the noninferiority boundary.

Useful diagnostics include checking model fit and data summaries, inspecting posterior distributions near $-M$, and assessing prior sensitivity. For simulation-based calculations, check chain convergence and effective sample size, then ensure Monte Carlo error is small relative to the distance between the estimated probability and $q$. If reasonable prior choices or model specifications move the result across the threshold, report that instability rather than presenting the decision as robust.

A frequentist noninferiority analysis commonly tests a null hypothesis that the treatment effect is at or below the unfavorable boundary, using a one-sided test or confidence bound. The Bayesian rule instead reports posterior probability above that boundary under a specified model and prior. A credible interval can also summarize posterior uncertainty, but a tail-probability rule directly evaluates the probability needed for this particular decision. These approaches answer related questions but are not interchangeable: a posterior threshold is not a type I error rate, and matching numerical cutoffs does not guarantee matching operating characteristics.

Common pitfall: “The estimated effect is better than the margin, so noninferiority is established.” A point estimate alone does not account for uncertainty. The correction is to calculate $P(\theta>-M\mid\text{data})$ and apply the prespecified threshold, while also checking the model and the precision of that probability.

A posterior noninferiority declaration does not establish superiority, prove that the treatments are identical, or guarantee benefit for every patient. It is limited by the chosen endpoint, estimand, margin, prior, likelihood, and threshold. In particular, a poorly justified margin can yield a mathematically correct probability that does not answer a clinically meaningful question.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

For a higher-is-better endpoint, what event defines noninferiority when the treatment-control difference is $\theta$ and the margin is $M$?
For an adverse event where lower risk is better, define a favorable effect using treatment and control risks. What is the noninferiority event?
A continuous outcome has posterior $\theta\mid\text{data}\sim N(-1,4)$ and $M=3$. Find the posterior noninferiority probability approximately.
Why does an estimated treatment effect above $-M$ not by itself establish noninferiority?
What should be checked if a posterior probability estimate is just above the decision threshold and was calculated from MCMC draws?

Solutions. (1) The event is $\theta>-M$, and the rule declares noninferiority if its posterior probability reaches the prespecified threshold. (2) Define $\theta=p_C-p_T$; then noninferiority is $\theta>-M$, meaning the treatment’s excess event risk is less than $M$. (3) The posterior standard deviation is $2$, so $P(\theta>-3)=1-\Phi((-3+1)/2)=\Phi(1)\approx0.8413$. (4) A point estimate does not account for posterior uncertainty; the probability mass beyond the boundary is what the rule evaluates. (5) Check convergence, effective sample size, and Monte Carlo error; if computational uncertainty could change the decision, obtain more reliable posterior sampling or report the result as uncertain.