Worked Normal-Outcome Comparison of Three Arms

Why a Three-Arm Normal Comparison Needs Care

In a multi-arm trial, several experimental treatments may be compared with one control group. With a continuous outcome, a normal model can give a transparent route from observed arm means to posterior probabilities of clinically important effects. The shared control matters: treatment-versus-control contrasts are not independent in the posterior, even when the arm-specific parameters are.

This tutorial works through a hypothetical example using a higher-is-better outcome measured in outcome units. The calculations assume a known outcome variance to keep the conjugate update explicit. The resulting posterior probabilities support a stated decision rule; they are not frequentist significance levels or guarantees of future performance.

These assumptions are substantive. The normal likelihood describes the outcome distribution within each arm; the common variance assumption says that its spread does not differ by arm. Conditional independence presumes that one participant's outcome does not determine another's. The prior says that, before seeing these data, each arm mean is centered at \(\mu_0\), with variance \(\tau^2\), independently of the other arm means. The known-variance assumption is a simplifying condition, not a claim that real outcome variances are always known.

Conjugate Update for Each Arm

For arm \(j\), the sample mean \(\bar{x}_j\) has likelihood \(\bar{x}_j\mid\theta_j\sim N(\theta_j,\sigma^2/n_j)\). Combining that normal likelihood with the normal prior gives a normal posterior. Its variance and mean are

The posterior mean is a precision-weighted average of the prior mean and sample mean. In particular, the prior contributes precision \(1/\tau^2\), and the sample mean contributes precision \(n_j/\sigma^2\). The posterior variance is smaller than either variance source because the two independent sources of information add precision.

For completeness, the update follows by multiplying the prior density by the sample-mean likelihood. The terms involving \(\theta_j\) in the log density are proportional to

Completing the square identifies the quadratic coefficient as \(1/v_j\) and the center as \(m_j\), giving the stated normal posterior. Under the specified independent priors and conditionally independent arm data, the three arm means remain independent in their joint posterior. That independence does not carry over to contrasts that share \(\theta_C\).

Shared-Control Contrasts and Success Probabilities

The posterior treatment-effect means are differences of arm posterior means. Their variances must account for the control mean appearing in both contrasts. For the example, the independent arm posteriors give

The posterior variance of either treatment-versus-control contrast is the sum of the two arm-mean variances. The covariance between the contrasts is positive because both subtract the same uncertain control mean.

For example, in the covariance expansion, \(\operatorname{Cov}(\theta_A-\theta_C,\theta_B-\theta_C)\), all cross-arm covariance terms are zero, while the shared term \(\operatorname{Var}(\theta_C)\) remains. The two contrasts therefore have posterior correlation \(0.2353/0.4706=0.5000\). Their marginal posterior standard deviation is \(\sqrt{8/17}\approx0.6860\) outcome units.

The contrast of A with B has a different variance calculation: the shared control cancels, so \(\Delta_A-\Delta_B=\theta_A-\theta_B\). In this example its posterior mean is \(m_A-m_B=8/17=0.4706\), and its variance is \(v_A+v_B=8/17\). Hence \(P(\Delta_A>\Delta_B\mid\text{data})=\Phi((8/17)/\sqrt{8/17})=\Phi(0.6860)\approx0.7536\). This probability compares the two experimental arms; it is not the probability that A is clinically successful versus control.

Applying a Prespecified Selection Rule

Posterior probabilities become a decision only after the rule is stated. For this hypothetical illustration, suppose the protocol selects an experimental arm only if its posterior probability of exceeding \(\delta=1.0\) is at least 0.80. If both qualify, select the one with the larger posterior success probability; if neither qualifies, select neither.

Diagnostics, Alternatives, and Limitations

A numerical posterior calculation is only as credible as its model. In a real analysis, examine whether the outcome distribution is reasonably compatible with a normal model, whether arm spreads are similar, and whether outliers or data-quality problems dominate the sample means. Confirm that the stated sample sizes and summaries match the analyzed participants and that the outcome units and direction are consistent with the clinical threshold. With small samples, skewness or heavy tails can make the normal sampling assumption consequential.

The known-variance assumption deserves particular attention. Here it made each sample-mean variance exactly \(\sigma^2/n_j\). If the variance must be estimated, treating an estimated value as known can understate uncertainty, especially with limited data. A model that assigns a probability distribution to the unknown variance, or a suitable computational posterior method, may then be more appropriate. Different prior centers or prior variances can also change the posterior shrinkage and decision probabilities; sensitivity to plausible priors should be checked when the decision is close to its cutoff.

The shared-control dependence is a separate diagnostic for the calculations. Treating \(\Delta_A\) and \(\Delta_B\) as independent would omit their covariance \(v_C\), distort joint probabilities, and give incorrect uncertainty for combinations of decisions. The marginal probabilities above can be calculated from each contrast separately, but a probability involving both success events—for example, the probability that at least one arm succeeds—requires their joint posterior distribution.

A Bayesian analysis reports probabilities under the likelihood and prior, then applies a decision rule such as the one illustrated here. A frequentist multiple-comparison analysis instead evaluates operating characteristics under repeated sampling and may use simultaneous inference or error-rate control. Those approaches answer related but distinct questions. Posterior probabilities do not by themselves establish frequentist familywise error control, and a frequentist p-value is not the posterior probability of a clinically meaningful effect.

The normal known-variance model is useful because its assumptions and calculations are visible, but it may not suit every continuous endpoint. It does not automatically handle unequal variances, strong skewness, informative missingness, or additional covariates. Nor does this single terminal calculation capture every feature of a staged trial; interim decisions and data-dependent selection require their own prespecified design and analysis considerations.