Computing Treatment-Arm Contrasts From Conjugate Posteriors

From Arm-Specific Posteriors to a Treatment Contrast

After observing trial outcomes, a posterior distribution describes uncertainty about each arm’s response probability. A useful treatment contrast is the difference between the treatment and control response probabilities: it is positive when the treatment response rate is higher. The question “How probable is treatment superiority, given the data and model?” is therefore a posterior probability about that difference—not a comparison of two point estimates.

Consider a two-arm trial with binary response. Let \(Y_T\) and \(Y_C\) be the numbers of responders among \(n_T\) treatment participants and \(n_C\) control participants. Write \(p_T\) and \(p_C\) for the corresponding response probabilities. Assume, conditional on these probabilities, that outcomes are independent Bernoulli observations within and across arms, and that the arm-specific probabilities have independent beta priors. With beta-binomial updating, the posteriors can be written \(p_T\mid\text{data}\sim\operatorname{Beta}(a_T,b_T)\) and \(p_C\mid\text{data}\sim\operatorname{Beta}(a_C,b_C)\), independently, where all four shape parameters are positive.

Independence is important: it allows the joint posterior density to be written as the product of the two beta densities. Independent priors and conditionally independent arm-specific binomial likelihoods give this factorization in the usual two-arm beta-binomial model. It need not hold in a hierarchical model that links the arm probabilities, or when shared uncertain parameters induce posterior dependence. In those settings, the independent-beta calculations below do not apply unchanged.

An Integral for the Probability of Superiority

For positive \(a,b\), the beta density on \(0<x<1\) is \(f(x;a,b)=x^{a-1}(1-x)^{b-1}/B(a,b)\), where \(B(a,b)=\int_0^1 x^{a-1}(1-x)^{b-1}\,dx\) is the beta function. Let \(F(x;a,b)\) denote its cumulative distribution function. Conditioning on the treatment probability gives a direct expression for the desired probability.

The first equality follows by integrating over possible values \(x\) of \(p_T\). Given \(p_T=x\), the event \(p_T>p_C\) is \(p_C<x\), whose posterior probability is \(F(x;a_C,b_C)\). The treatment density supplies the weight for each \(x\). This derivation uses posterior independence; without it, the conditional distribution of \(p_C\) given \(p_T=x\) may depend on \(x\).

For arbitrary positive shape parameters, the integral is well defined and can be evaluated by numerical integration. When the control beta shapes are positive integers, its CDF also has a finite polynomial representation. Specifically, for positive integers \(a_C,b_C\), let \(m=a_C+b_C-1\). Then \(F(x;a_C,b_C)=\sum_{j=a_C}^{m}{m\choose j}x^j(1-x)^{m-j}\). Substitution into the integral yields

This is an exact finite sum when the control shapes are positive integers; the treatment shapes need only be positive. Each term follows by integrating a power of \(x\) times a power of \(1-x\) against the treatment beta density. If the control shapes are not integers, this particular finite-sum identity is unavailable, but the integral expression remains valid.

Worked Calculations With Beta-Function Identities

Numerical Evaluation, Checks, and Alternatives

For general shape parameters, evaluate the integral \(\int_0^1 F(x;a_C,b_C)f(x;a_T,b_T)\,dx\) with a numerical quadrature method. Use the posterior shapes in the density and CDF, not the observed response proportions in their place. Beta densities can be sharply concentrated or unbounded near 0 or 1 when a shape is less than 1, so a quadrature routine should handle endpoint behavior reliably. In practice, adaptive quadrature, adequate tolerance, and, when needed, a change of variables or careful treatment of the endpoints help avoid inaccurate results.

Useful checks include verifying that the answer lies between 0 and 1 and increasing numerical accuracy to see whether reported digits remain stable. Because beta distributions are continuous, the probability of an exact tie is zero. Consequently, under independent posteriors, \(P(p_T>p_C\mid\text{data})+P(p_C>p_T\mid\text{data})=1\). A numerical calculation of both directions should satisfy this identity up to numerical error. If it does not, check the arm labels, the integration limits, and the CDF used in the integrand.

The integral or finite sum directly computes the posterior probability of interest. A posterior simulation approach could instead draw independent pairs from the two beta posteriors and estimate the probability by the fraction of pairs for which the treatment draw is larger. This is convenient to generalize, but the estimate has simulation error; it is not an exact evaluation. A normal approximation to the difference may be simpler in some settings, but it can be poor when the posteriors are skewed or concentrated near the boundaries of the probability scale. When the beta integral is tractable, it avoids that approximation.

The calculation also has limits. It answers a model-based question about the two response probabilities as defined by the analysis. It does not address whether missing outcomes, nonadherence, or other design features undermine the likelihood assumptions; nor does a positive contrast alone establish a clinically important benefit. Such questions require their own modeling choices and clinical interpretation.