Joint Posterior Probability of Multiple Arm Successes

Why Joint Posterior Probabilities Need Dependence

A posterior probability for one treatment arm answers whether that arm meets its own criterion, conditional on the model, prior, and observed data. When interest is in whether several specified arms meet their criteria at the same time, the relevant probability is joint. It is not generally the product of the arms’ separate posterior probabilities. In a shared-control analysis, the treatment-control contrasts can be correlated because they use information from the same control group.

This tutorial focuses on calculating that joint probability. It does not replace the individual success criteria with a ranking comparison: an arm can rank highly among treatments and still fail a clinically specified threshold. The criteria and the dependence structure both matter.

Definition: Let \(\theta_j\) be the treatment-control contrast for specified arm \(j\), and let \(\delta_j\) be its success threshold. For \(k\) specified arms, define the joint success event as \(A=\{\theta_1>\delta_1,\ldots,\theta_k>\delta_k\}\). The joint posterior success probability is \(P(A\mid \text{data})\), calculated under the specified prior and likelihood.

For example, if larger contrasts indicate benefit, \(\delta_j\) might be a clinically meaningful benefit threshold. A success criterion using \(\theta_j\geq\delta_j\) instead of a strict inequality changes the event only where the posterior assigns positive probability to equality. A continuous, nondegenerate normal posterior assigns zero probability to a single boundary point, but a deterministic contrast can lie exactly on that boundary, so the chosen inequality should be stated.

Joint Normal Posterior Calculation

Suppose the posterior distribution of the \(k\) contrasts is multivariate normal:

(\theta_1,\ldots,\theta_k)^\mathsf{T}\mid\text{data}\sim N_k(m,V),

where \(m=(m_1,\ldots,m_k)^\mathsf{T}\) is the vector of posterior means and \(V\) is the posterior covariance matrix. The matrix \(V\) must be symmetric and positive semidefinite. This model is appropriate when the posterior contrasts are jointly normal, as in the relevant normal-normal setting; it should not be assumed merely because marginal means and variances have been estimated. The posterior covariance, not just the separate marginal variances, is needed for a joint probability.

First consider contrasts with positive posterior variances, \(V_{jj}>0\) for every arm being standardized. Let \(s_j=\sqrt{V_{jj}}\), and standardize each contrast and threshold:

Z_j=\frac{\theta_j-m_j}{s_j},\qquad a_j=\frac{\delta_j-m_j}{s_j}.

The standardized vector has a multivariate normal posterior with mean zero and correlation matrix \(R\), whose entries are \(R_{ij}=V_{ij}/(s_i s_j)\). The joint success probability is therefore the multivariate normal upper-tail probability

P(\theta_1>\delta_1,\ldots,\theta_k>\delta_k\mid\text{data}) = P(Z_1>a_1,\ldots,Z_k>a_k).

For two arms with \(|\rho|<1\), where \(\rho=R_{12}\), the conditional distribution is \(Z_2\mid Z_1=z\sim N(\rho z,1-\rho^2)\). Integrating the conditional probability over values of \(Z_1\) that meet its threshold gives

P(Z_1>a_1,Z_2>a_2) = \int_{a_1}^{\infty}\phi(z) \Phi\!\left(\frac{\rho z-a_2}{\sqrt{1-\rho^2}}\right)\,dz.

Here \(\phi\) and \(\Phi\) are the standard normal density and cumulative distribution function. This integral is valid only when \(|\rho|<1\). If \(\rho=1\), \(Z_2=Z_1\) almost surely; if \(\rho=-1\), \(Z_2=-Z_1\) almost surely. Those cases reduce to one-dimensional events, not to the displayed conditional-normal formula.

Important edge case: If \(V_{jj}=0\), then \(\theta_j=m_j\) with posterior probability one. Do not standardize that contrast or form a correlation involving it. For the strict criterion \(\theta_j>\delta_j\), it succeeds deterministically if \(m_j>\delta_j\) and fails deterministically if \(m_j\leq\delta_j\). A deterministic failure makes the joint probability zero; if it succeeds, remove that component and calculate the probability for the remaining contrasts.

Worked Examples

Worked Example: Two Arms With Positive Shared-Control Dependence

Suppose a hypothetical normal-normal analysis gives two specified treatment-control contrasts the posterior distribution

(\theta_1,\theta_2)^\mathsf{T}\mid\text{data} \sim N_2\left[ \begin{pmatrix}1\\1\end{pmatrix}, \begin{pmatrix}1&0.5\\0.5&1\end{pmatrix} \right].

The success threshold for each contrast is 1 unit, and success requires exceeding the threshold. Both posterior standard deviations are 1 unit, so \(a_1=a_2=(1-1)/1=0\), and the posterior correlation is \(\rho=0.5\). Thus, the target probability is \(P(Z_1>0,Z_2>0)\).

For this zero-threshold bivariate normal case, the integral above yields \(P(Z_1>0,Z_2>0)=\frac14+\frac{\arcsin(\rho)}{2\pi}\). A short justification is to differentiate the integral with respect to \(\rho\): its derivative is \(1/(2\pi\sqrt{1-\rho^2})\); at \(\rho=0\), the probability is \(1/4\). Integrating that derivative from 0 to \(\rho\) gives the stated expression. Substituting the posterior correlation,

P(\theta_1>1,\theta_2>1\mid\text{data}) = \frac14+\frac{\arcsin(0.5)}{2\pi} = \frac14+\frac{\pi/6}{2\pi} = \frac13\approx0.3333.

The marginal probability for either arm is \(P(Z_j>0)=0.5\), so multiplying the marginal probabilities would give \(0.25\), not \(0.3333\). The positive dependence makes joint success more likely than that independence calculation suggests. The covariance matrix is valid: its eigenvalues are 0.5 and 1.5, both positive.

Worked Example: Negative Dependence Changes the Joint Probability

Consider a second hypothetical posterior with means 1 unit, variances 1 square unit, and covariance \(-0.5\) square units:

(\theta_1,\theta_2)^\mathsf{T}\mid\text{data} \sim N_2\left[ \begin{pmatrix}1\\1\end{pmatrix}, \begin{pmatrix}1&-0.5\\-0.5&1\end{pmatrix} \right].

Again, each threshold is 1 unit, making \(a_1=a_2=0\), but now \(\rho=-0.5\). The matrix is positive definite, with eigenvalues 0.5 and 1.5, and \(|\rho|<1\), so the bivariate formula applies. Using the same zero-threshold result,

P(\theta_1>1,\theta_2>1\mid\text{data}) = \frac14+\frac{\arcsin(-0.5)}{2\pi} = \frac14-\frac{\pi/6}{2\pi} = \frac16\approx0.1667.

Each arm still has marginal posterior success probability 0.5, whose product is 0.25. Here negative dependence makes simultaneous success less likely than that product. The comparison shows why a calculation based only on marginal probabilities cannot determine the joint probability.

Worked Example: Deterministic and Perfectly Correlated Contrasts

First suppose a hypothetical posterior gives \(\theta_1=2\) units with probability one, and the strict success threshold is 1 unit. For a second contrast, suppose \(\theta_2\mid\text{data}\sim N(0.4,0.5^2)\) units, with threshold 0 units. Since \(2>1\), the first criterion succeeds deterministically. The joint probability is the probability that the second contrast exceeds 0:

P(\theta_1>1,\theta_2>0\mid\text{data}) = P(\theta_2>0\mid\text{data}) = \Phi\!\left(\frac{0.4-0}{0.5}\right) = \Phi(0.8) \approx0.7881.

If instead the deterministic contrast were \(\theta_1=1\) with the same strict threshold, it would fail because \(1\not>1\), and the joint probability would be zero.

For positive-variance contrasts with perfect positive correlation, let \(Z_2=Z_1\) almost surely and use thresholds \(a_1=0\), \(a_2=0.5\). Both criteria hold exactly when \(Z_1>0.5\), so the probability is \(1-\Phi(0.5)\approx0.3085\). For perfect negative correlation, let \(Z_2=-Z_1\), \(a_1=-1\), and \(a_2=-0.5\). Both criteria require \(-1<Z_1<0.5\), giving

P(-1<Z_1<0.5) = \Phi(0.5)-\Phi(-1) \approx0.6915-0.1587 = 0.5328.

These reductions follow directly from the almost-sure relationships between the standardized contrasts. In particular, neither case uses a conditional variance of \(1-\rho^2\) as a denominator.

More Than Two Arms and Numerical Checks

For three or more specified arms, the same event is a multivariate normal upper-tail probability. Except in special cases, it does not reduce to a simple product or a one-dimensional closed form. It can be calculated by multivariate normal integration or estimated from joint posterior draws. When using draws, each draw must retain the dependence among all contrasts. If \(\theta^{(b)}\) is the vector in posterior draw \(b\), an estimate based on \(B\) draws is

\widehat{P}(A\mid\text{data}) = \frac{1}{B}\sum_{b=1}^{B} I\{\theta_1^{(b)}>\delta_1,\ldots,\theta_k^{(b)}>\delta_k\}.

For independent draws, the Monte Carlo standard error is approximately \(\sqrt{\hat q(1-\hat q)/B}\), where \(\hat q\) is the estimated probability. For correlated Markov chain draws, this expression should account for the chain’s effective sample size rather than treating all draws as independent. A small estimated probability may require many effective draws to obtain useful precision.

Before trusting a result, check that the covariance matrix is symmetric and positive semidefinite, that the marginal variances and units are correctly matched to the thresholds, and that standardized correlations lie between \(-1\) and 1. For any joint probability \(q\) and marginal probabilities \(p_j=P(\theta_j>\delta_j\mid\text{data})\), basic bounds are

\max\left(0,\sum_{j=1}^{k}p_j-(k-1)\right) \leq q\leq \min_j p_j.

An answer outside these bounds signals a computational or coding error. For two arms, a direct quadrature result can also be compared with posterior simulation; agreement within simulation error is a useful numerical check.

What the Calculation Does and Does Not Say

The joint probability is conditional on the likelihood, prior, observed data, and stated success thresholds. It measures posterior probability that all the specified inequalities hold simultaneously. It is not obtained by multiplying arm-specific probabilities unless the relevant posterior events are independent. It also does not state the probability that the model, prior, or thresholds are correct.

A ranking probability answers a different question: it compares arms with one another. Joint success compares each specified contrast with its own criterion. Marginal probabilities can describe each arm separately, but they cannot reveal how often several arms succeed together without dependence information. If the contrasts are not jointly normal, use their actual joint posterior distribution or posterior draws rather than imposing a normal approximation without justification.

Common Mistakes and Diagnostics

Multiplying marginal probabilities. This is justified only when the posterior success events are independent. Shared-control contrasts commonly have nonzero posterior covariance.
Standardizing a deterministic contrast. A zero posterior variance makes division by the standard deviation undefined. Check the deterministic criterion directly, then remove that component or set the joint probability to zero if it fails.
Using a nonsingular bivariate formula at \(\rho=\pm1\). The conditional variance is zero at these boundaries. Use the almost-sure relation \(Z_2=Z_1\) or \(Z_2=-Z_1\) and calculate the resulting one-dimensional event.
Ignoring incompatible covariance estimates. A matrix that is not positive semidefinite cannot be a covariance matrix. Recheck the model or numerical calculations before attempting a multivariate probability.
Reporting a simulation estimate without precision. State the number of effective draws and an appropriate Monte Carlo error measure, especially when the estimated joint probability is small.

Key takeaway: Specify each arm’s threshold, obtain the joint posterior covariance of the contrasts, and calculate the probability of their simultaneous success event using that dependence. Handle zero variances and perfect correlations as degenerate cases rather than forcing them into a standardization or integral that is undefined.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

Two standardized contrasts have zero thresholds and posterior correlation 0. What is their joint success probability?
For two zero-threshold standardized contrasts with correlation \(0.8\), calculate the joint probability using the bivariate normal quadrant formula.
A contrast has posterior variance zero, posterior value 0.7 units, and a strict success threshold of 0.7 units. Does it succeed?
Why is the bivariate conditional-normal integral not valid when the posterior correlation is \(-1\)?

Solutions. (1) With correlation zero, the two standard normal contrasts are independent, so the probability is \(0.5\times0.5=0.25\). (2) The probability is \(\frac14+\frac{\arcsin(0.8)}{2\pi}\). Since \(\arcsin(0.8)\approx0.9273\), this is \(0.25+0.1476\approx0.3976\). (3) No. The strict criterion is \(0.7>0.7\), which is false, so this deterministic contrast fails. (4) At correlation \(-1\), the standardized contrasts satisfy \(Z_2=-Z_1\) almost surely and the conditional variance \(1-\rho^2\) is zero. The integral’s denominator is therefore zero; calculate the interval event implied by \(Z_2=-Z_1\) instead.