Defining Exchangeability Across Trial Subgroups

Why Exchangeability Matters Across Subgroups

A basket trial or multi-subgroup trial asks whether evidence from one subgroup should inform what is learned about treatment effects in another. The subgroups may share a disease, intervention, outcome, or study protocol, but their treatment effects need not be identical. The modeling question is whether, before seeing the subgroup-specific results, the effects can reasonably be treated as arising from a common distribution.

That idea is exchangeability. It provides a basis for borrowing because evidence about the distribution of effects can be learned across subgroups. It does not say that every subgroup has the same effect, nor does it guarantee that borrowing will be appropriate when subgroup results happen to look similar.

Definition: Let \(\theta_j\) denote the treatment effect for subgroup \(j\), for \(j=1,\ldots,K\). The subgroup effects are exchangeable if their joint probability distribution is unchanged by any permutation of the subgroup labels. Formally, for every permutation \(\pi\), \((\theta_1,\ldots,\theta_K)\) and \((\theta_{\pi(1)},\ldots,\theta_{\pi(K)})\) have the same joint distribution.

In practical terms, exchangeability means that, under the model and the information being conditioned on, the labels themselves do not make one subgroup's effect systematically more likely to be larger or smaller than another's. It is a statement about the joint distribution of unknown effects, not a statement that the realized effects are numerically equal.

A common way to construct an exchangeable model is to suppose that the effects are conditionally independent and identically distributed given shared parameters \(\eta\). For example, a conceptual model might be \(\theta_j\mid\eta\sim N(\mu,\tau^2)\), where \(\eta=(\mu,\tau^2)\) describes the distribution across subgroups. This example illustrates the definition; exchangeability does not require a normal distribution.

p(\theta_1,\ldots,\theta_K) = \int \prod_{j=1}^{K} p(\theta_j\mid\eta)\,p(\eta)\,d\eta

The product inside the integral is unchanged by reordering its factors. Integration over the shared parameters therefore gives a joint distribution invariant to permutations of the subgroup labels. This establishes exchangeability for this model construction. Conditional independence given \(\eta\) is one sufficient construction, not the definition of exchangeability itself; other joint distributions can also be exchangeable.

Exchangeability Is Not Equality

If subgroup effects share a distribution with nonzero spread, individual effects can differ. In the illustrative normal model, \(\mu\) is the distribution's centre and \(\tau\) describes between-subgroup variation. A positive \(\tau\) permits different \(\theta_j\) values, while also connecting them through shared distributional information. By contrast, asserting one common effect for every subgroup would impose \(\theta_1=\cdots=\theta_K\). That is a different and stronger modeling assumption.

The distinction can be stated without choosing a particular model: exchangeability makes each subgroup's position in the joint distribution interchangeable with every other subgroup's position. It does not collapse the joint distribution onto the set where all effects are equal. Nor does it mean that the observed estimates must look alike. Estimates vary both because the underlying effects may differ and because each estimate is subject to sampling uncertainty.

Borrowing is possible because observations from several subgroups can provide information about shared features of the distribution of their effects. That shared information may in turn be relevant to each subgroup's effect. The extent and consequences of this borrowing depend on the full model and the observed data; exchangeability alone does not prescribe a particular amount of borrowing.

Worked Example: Different Estimates Under a Common Exchangeability Assumption

Suppose a hypothetical trial studies the same treatment and 12-week response outcome in three disease subgroups. The response counts are:

Subgroup	Treatment responses	Control responses
A	18 of 30	9 of 30
B	12 of 30	8 of 30
C	9 of 30	11 of 30

For illustration, define each subgroup effect as the treatment-minus-control difference in response proportions. The observed effect estimate in subgroup \(j\) is \(\hat{\theta}_j=\hat{p}_{T,j}-\hat{p}_{C,j}\).

For A, \(\hat{\theta}_A=18/30-9/30=0.60-0.30=0.30\), or a 30 percentage-point difference. For B, \(\hat{\theta}_B=12/30-8/30=0.40-0.2667=0.1333\), or about 13.33 percentage points. For C, \(\hat{\theta}_C=9/30-11/30=0.30-0.3667=-0.0667\), or about −6.67 percentage points.

These estimates differ, including in sign. That fact alone neither proves nor disproves exchangeability: the estimates are noisy observations of subgroup effects, and the effects themselves may vary. If the subgroups, outcome definition, treatment contrast, and trial conditions support treating the effects as drawn from a common distribution, an exchangeable model can represent that assumption while allowing distinct effects. It would be incorrect to interpret the assumption as saying that the three calculated differences must be equal.

What Must Be Comparable

Exchangeability is meaningful only after the effects have been defined. A treatment effect is tied to an estimand: the population, treatment contrast, outcome, time point, and other features that specify what quantity is being compared. Subgroups need not be identical in every respect, but their \(\theta_j\) values must be comparable quantities for a shared distributional assumption to make sense.

Before adopting exchangeability, examine at least the following:

Same treatment contrast: the effects should compare the same intervention with a compatible control or reference condition.
Same outcome and time frame: effects for different endpoints or different follow-up windows are not automatically measurements of one common kind of quantity.
Compatible effect scale: risk differences, risk ratios, and log odds ratios are different scales. Exchangeability on one scale does not imply exchangeability on another.
Relevant trial conditions: differences in eligibility, dose, co-interventions, assessment, or follow-up can make subgroup labels informative about effects.
Subgroup definition: the groups should be defined in a way that supports interpreting their effects as members of the intended collection, rather than as unrelated quantities selected after observing results.

These are substantive checks, not a mechanical test that certifies exchangeability. Similarity of design and clinical context can make the assumption more credible, but cannot establish it with certainty. Conversely, visible differences between estimates do not by themselves invalidate it.

Worked Example: Checking the Effect Scale

Consider two hypothetical subgroups with binary response outcomes. In subgroup A, 20 of 50 controls and 30 of 50 treated participants respond. In subgroup B, 10 of 50 controls and 25 of 50 treated participants respond. Suppose the intended effect is the risk difference.

For A, the control response proportion is \(20/50=0.40\), and the treatment response proportion is \(30/50=0.60\). Thus \(\hat{\theta}_A=0.60-0.40=0.20\), a 20 percentage-point difference.

For B, the corresponding proportions are \(10/50=0.20\) and \(25/50=0.50\). Thus \(\hat{\theta}_B=0.50-0.20=0.30\), a 30 percentage-point difference. Both estimates are risk differences for the same kind of outcome and treatment contrast, so they are at least defined on a directly comparable scale.

This calculation does not establish that the effects are exchangeable; it checks one necessary aspect of comparability. A model might allow the two effects to differ while relating them through a shared distribution. If instead one subgroup's effect were defined as a risk difference and the other's as a log odds ratio, the two numbers could not be treated as values of the same \(\theta_j\) without a deliberate modeling transformation and a clear common target.

Permutation Invariance in a Simple Illustration

A short calculation makes the label-symmetry idea concrete. Suppose three subgroup effects are conditionally independent draws from a normal distribution with fixed mean \(\mu=0.10\) and standard deviation \(\tau=0.15\). Consider the hypothetical effect values \(0.20\), \(-0.05\), and \(0.15\), all measured as risk differences.

p(\theta_1,\theta_2,\theta_3\mid\mu,\tau) = \prod_{j=1}^{3}\frac{1}{\sqrt{2\pi}\tau} \exp\left[-\frac{(\theta_j-\mu)^2}{2\tau^2}\right]

The three deviations from the common mean are \(0.10\), \(-0.15\), and \(0.05\). Their squared sum is \(0.10^2+(-0.15)^2+0.05^2=0.0100+0.0225+0.0025=0.0350\). The joint density is therefore proportional to \(\exp[-0.0350/(2\times0.15^2)] = \exp(-0.7778)\), with the same proportionality constant for every ordering of these values.

If the labels assigned to the three effects are permuted, the squared deviations are merely reordered; their sum remains \(0.0350\). The joint density is unchanged. This calculation illustrates permutation invariance, not evidence that the effects are equal. In fact, the three values in the illustration are visibly distinct.

When Exchangeability Is Implausible

A shared distribution is less defensible when subgroup membership systematically changes what treatment effect should be expected, after accounting for the model's other information. For example, subgroups might differ in disease mechanism, treatment exposure, outcome ascertainment, or baseline risk in ways that modify the treatment effect on the chosen scale. If labels carry such information, treating them as interchangeable can misrepresent the uncertainty and the relationships among effects.

Worked Example: Incompatible Follow-Up Times

Suppose a hypothetical trial reports a response outcome at 12 weeks for subgroup A and at 24 weeks for subgroup B. In A, 15 of 50 treated participants and 10 of 50 controls respond by 12 weeks. In B, 30 of 50 treated participants and 20 of 50 controls respond by 24 weeks.

The observed difference in A is \(15/50-10/50=0.30-0.20=0.10\), or 10 percentage points at 12 weeks. The difference in B is \(30/50-20/50=0.60-0.40=0.20\), or 20 percentage points at 24 weeks.

Although both numbers are risk differences, they refer to different follow-up times. The calculation alone therefore does not define two measurements of the same estimand. Treating them as exchangeable effects without addressing the time mismatch would conflate subgroup differences with differences in outcome definition. A common time point or an explicit model connecting the time-specific outcomes would be needed to make the comparison interpretable.

A practical warning is that exchangeability is an assumption about the model for effects, not a conclusion obtained simply because subgroup estimates are not statistically distinguishable. Small subgroup samples can make substantial differences difficult to detect. Equally, a conflict diagnostic or a visual comparison can flag tension without uniquely identifying its cause. Trial differences, covariate imbalance, and sampling variability may all contribute.

The main alternatives differ in what they assume about relationships across subgroups. A model that treats effects as unrelated does not use a shared distribution to connect them. A model that requires one identical effect imposes equality. Exchangeability occupies neither extreme by definition: it specifies symmetry of the joint distribution, while permitting subgroup-specific effects. Choosing among these assumptions requires substantive knowledge and sensitivity analysis; no single observed pattern settles the choice.

Common Mistakes and a Practical Check

Wrong reasoning: “The effects are exchangeable, so every subgroup has the same treatment effect.” Correction: exchangeability is invariance to relabeling of the joint distribution; it permits variation among effects.
Wrong reasoning: “The observed estimates are similar, so exchangeability is established.” Correction: observed similarity is not proof of the assumption, particularly when subgroup estimates are imprecise.
Wrong reasoning: “One subgroup estimate differs, so exchangeability is impossible.” Correction: differences can arise from genuine between-subgroup variation, sampling error, or both. Investigate their size and context rather than treating any discrepancy as decisive.
Wrong reasoning: “The outcomes are both binary, so their treatment effects are interchangeable.” Correction: outcome timing, treatment contrast, population, and effect scale also determine whether the effects are comparable.

A useful working question is: Would the subgroup labels, by themselves, give a reason to predict different distributions of treatment effects? If the answer is yes, identify the relevant differences and decide whether the model should account for them. If no, exchangeability may be a defensible starting assumption, but its sensitivity should still be assessed.

Key takeaway: Exchangeability says that subgroup treatment effects have a joint distribution symmetric to relabeling. It can support borrowing through shared distributional information, while allowing the effects themselves to differ. Its credibility depends on defining comparable effects and considering whether subgroup identity predicts systematic differences.

Check Your Understanding

Answer each question, then compare with the solutions below.

In one sentence, distinguish exchangeability of \(K\) subgroup effects from equality of those effects.
Three hypothetical subgroups have estimated risk differences of \(0.08\), \(0.12\), and \(0.20\). Does this pattern alone disprove exchangeability? Explain.
A study defines the effect in one subgroup at 8 weeks and another at 16 weeks. What comparability concern should be addressed before assuming exchangeability?
Under a conditionally independent model with a shared distribution, why does permuting the subgroup labels leave the joint distribution unchanged?

Solutions. (1) Exchangeability means the joint distribution is invariant to relabeling, whereas equality requires every effect to have the same value. (2) No. The estimates may differ because the underlying effects vary, because of sampling error, or both; the pattern must be assessed in context and does not by itself settle the assumption. (3) The effects refer to different follow-up times and may not represent the same estimand; a common time point or an explicit model linking the outcomes is needed. (4) The joint distribution is a product of the same subgroup-specific conditional density factors, so a permutation only reorders those factors and does not change their product; integrating over shared parameters preserves that symmetry.