Prior Predictive Checks For Subgroup Outcomes

Why Check Subgroups Together?

A prior can imply plausible outcomes in each subgroup separately but implausible outcomes when subgroups are considered together. For example, the prior may allow a wide range of treatment effects in both subgroups while making the effects nearly identical, or instead allowing large differences between them. These are different clinical assumptions, even if each subgroup’s marginal prior is the same.

A subgroup prior predictive check examines outcomes generated jointly across the trial’s subgroups, using the joint prior and the planned design. It asks whether the resulting subgroup response rates, treatment effects, and differences between subgroup effects are credible before using trial outcomes to update the prior. The check is about the implications of the prior combined with the outcome model; it is not a test of a treatment effect.

Definition: For subgroup \(g\) and treatment arm \(a\), let \(p_{g,a}\) be the response probability, where \(a=0\) denotes control and \(a=1\) denotes treatment. The subgroup treatment effect on the risk-difference scale is \(\delta_g=p_{g,1}-p_{g,0}\). For subgroups A and B, the difference in treatment effects is \(D=\delta_B-\delta_A\). A joint prior specifies a probability distribution for the vector of subgroup probabilities, not merely a separate distribution for each probability.

The scale matters. Here, \(\delta_g\) and \(D\) are measured in probability units, often reported as percentage points. A negative \(\delta_g\) indicates a lower response probability in the treatment arm than in control under this definition. A positive \(D\) means the treatment-minus-control difference is larger in subgroup B than in subgroup A; by itself, it does not establish that treatment is beneficial in either subgroup.

From the Joint Prior to Subgroup Outcomes

For a binary endpoint, suppose the planned numbers assigned to each subgroup-arm cell are fixed at \(n_{g,a}\). Given the probabilities, let \(C_{g,a}\) be the number of responses in that cell. The usual binomial sampling model is

C_{g,a}\mid \boldsymbol{p}\sim\operatorname{Binomial}(n_{g,a},p_{g,a}),

where \(\boldsymbol{p}\) denotes the vector of all subgroup-arm probabilities. The standard model assumes that responses are independent within each cell conditional on its probability, and that the cell counts are conditionally independent given the full probability vector. These assumptions may need modification for clustered outcomes, repeated observations, or other dependence not represented in \(\boldsymbol{p}\).

To generate a prior predictive trial, draw the entire vector \(\boldsymbol{p}\) from its joint prior, then draw counts from the conditional sampling model. Repeating both steps produces joint predictions for all subgroup-arm outcomes. Formally, for possible counts \(c_{g,a}\),

P(C_{g,a}=c_{g,a}\ \text{for all }g,a) = \int \left[ \prod_{g,a} \binom{n_{g,a}}{c_{g,a}} p_{g,a}^{c_{g,a}} (1-p_{g,a})^{n_{g,a}-c_{g,a}} \right] \pi(\boldsymbol{p})\,d\boldsymbol{p},

where \(\pi(\boldsymbol{p})\) is the joint prior density. For discrete priors, the integral is replaced by a sum over possible prior states. The product inside the integral is conditional on the probabilities; integrating over their joint prior carries the prior dependence into the predicted outcomes.

Specify the subgroup estimands.
State which subgroups and treatment arms are being checked, and define the response probabilities and treatment-effect scale. For two subgroups, state whether the key contrast is \(D=\delta_B-\delta_A\).

Use the trial design.
Set the planned cell sizes \(n_{g,a}\), or specify a model for them if subgroup sizes are uncertain. The predicted counts depend on these design quantities.

Generate joint outcomes.
Draw all probabilities together from the joint prior, then generate the subgroup-arm counts under the stated outcome model. Do not independently redraw each marginal if the prior encodes dependence.

Assess clinical coherence.
Inspect the generated subgroup rates and contrasts in light of the intended clinical knowledge and design. Investigate predictions that conflict with those assumptions, distinguishing variation in probabilities from binomial variation in counts.

Worked Example: Checking Rates and Effects in Two Subgroups

Suppose a hypothetical trial has two subgroups, A and B, with 40 participants in each arm of each subgroup. A simple joint prior gives equal probability to four states:

Prior state	\(p_{A,0}\)	\(p_{A,1}\)	\(p_{B,0}\)	\(p_{B,1}\)
1	0.20	0.30	0.40	0.50
2	0.25	0.35	0.45	0.55
3	0.30	0.40	0.50	0.60
4	0.35	0.45	0.55	0.65

In every state, \(\delta_A=0.10\) and \(\delta_B=0.10\), so \(D=0\). The prior predicts a 10-percentage-point treatment-minus-control difference in each subgroup and no difference between those subgroup effects. Its control probabilities also differ systematically: subgroup B’s control response probability is 0.20 higher than subgroup A’s in every state.

Under the prior, the mean response probability in subgroup A’s control arm is \((0.20+0.25+0.30+0.35)/4=0.275\). The mean predicted response count in that cell is therefore \(40(0.275)=11\). In subgroup A’s treatment arm, the mean probability is \(0.375\), giving \(40(0.375)=15\) expected responses. For subgroup B, the corresponding means are \(0.475\) and \(0.575\), giving \(40(0.475)=19\) and \(40(0.575)=23\) expected responses. These averages summarize the prior implications; the simulated counts vary around them. A useful check is whether the rate patterns and the common 10-percentage-point effect match the clinical assumptions the prior was intended to express.

Why Dependence Changes the Subgroup Contrast

The joint prior matters because \(D\) depends on both subgroup effects at once. Its variance is

\operatorname{Var}(D) = \operatorname{Var}(\delta_B) + \operatorname{Var}(\delta_A) - 2\operatorname{Cov}(\delta_A,\delta_B).

This follows by writing \(D=\delta_B-\delta_A\) and applying the variance identity \(\operatorname{Var}(X-Y)=\operatorname{Var}(X)+\operatorname{Var}(Y)-2\operatorname{Cov}(X,Y)\). Thus, positive dependence between subgroup effects tends to reduce the variability of their difference, while negative dependence tends to increase it. This statement concerns the variance; the full distribution of \(D\) also depends on the joint distribution’s shape.

Worked Example: Same Marginal Effects, Different Joint Implications

Consider four equally likely prior states. In both constructions, the marginal values for each subgroup’s treatment effect are exactly \(-0.02,-0.04,-0.06,-0.08\). To keep the probabilities valid, set \(p_{A,0}=p_{B,0}=0.30\) in every state and let \(p_{g,1}=0.30+\delta_g\). All resulting probabilities lie between 0 and 1.

In the aligned construction, pair A’s effects \((-0.02,-0.04,-0.06,-0.08)\) with B’s effects \((-0.02,-0.04,-0.06,-0.08)\), in that order. Then the four values of \(D=\delta_B-\delta_A\) are \(0,0,0,0\).

In the reverse-order construction, keep A’s values unchanged and pair them with B’s values \((-0.08,-0.06,-0.04,-0.02)\). The resulting differences are

(-0.08)-(-0.02)=-0.06,\quad (-0.06)-(-0.04)=-0.02,\quad (-0.04)-(-0.06)=0.02,\quad (-0.02)-(-0.08)=0.06.

The marginal distribution for each subgroup effect is unchanged, but the distribution of the between-subgroup difference is not. In the reverse-order construction, the mean of \(D\) is \(0\) and its variance is \((0.0036+0.0004+0.0004+0.0036)/4=0.002\). In the aligned construction its variance is zero. Looking only at the two marginal priors would miss this difference in the prior’s implications for effect modification.

Predicted Counts Include Sampling Variation

The probabilities drawn from the prior are not the same as the observed proportions in a simulated trial. Even when a prior state fixes a response probability, the count varies according to the binomial model. Conversely, mixing over uncertain probabilities adds another source of variation. For a cell with fixed size \(n\) and prior-distributed probability \(p\), the law of total variance gives

\operatorname{Var}(C) = E\{\operatorname{Var}(C\mid p)\} + \operatorname{Var}\{E(C\mid p)\} = nE\{p(1-p)\}+n^2\operatorname{Var}(p).

The first term is average binomial sampling variance; the second reflects uncertainty in the response probability. This decomposition helps diagnose a mismatch: an unusual simulated count might arise from a clinically implausible probability, ordinary sampling variability, or both.

Worked Example: Separating Probability Uncertainty from Count Variation

Suppose a hypothetical prior assigns equal probability to two joint states. In the first state the subgroup probabilities are \((p_{A,0},p_{A,1},p_{B,0},p_{B,1})=(0.20,0.30,0.50,0.60)\); in the second they are \((0.50,0.60,0.20,0.30)\). Let each cell contain \(n=20\) participants. In both states, \(\delta_A=\delta_B=0.10\), so \(D=0\), although which subgroup has the higher baseline response probability switches.

For subgroup A’s control count, the two possible conditional means are \(20(0.20)=4\) and \(20(0.50)=10\), so its prior predictive mean is \((4+10)/2=7\). The expected conditional binomial variance is \(20\{[0.20(0.80)+0.50(0.50)]/2\}=20(0.205)=4.10\). The variance of the two conditional means is \([(4-7)^2+(10-7)^2]/2=9\). Therefore, the predictive count variance is \(4.10+9=13.10\) responses squared.

For subgroup A’s treatment count, the two probabilities are 0.30 and 0.60. The predictive mean is \(20(0.45)=9\); the expected conditional variance is \(20\{[0.30(0.70)+0.60(0.40)]/2\}=4.50\); and the variance of the conditional means 6 and 12 is 9. Thus its predictive count variance is \(13.50\) responses squared. These calculations show why an observed proportion need not equal either the mean prior probability or one of the prior’s possible probability values. The joint states also induce dependence between subgroup counts: they are conditionally independent given the probabilities, but share uncertainty about which prior state occurred.

Assumptions, Diagnostics, and Common Mistakes

A useful check starts with explicit assumptions: subgroup membership is defined as intended, the arm allocations and cell sizes reflect the trial design, the outcome model represents the endpoint, and the joint prior represents the intended beliefs about subgroup risks and effects. If subgroup sizes are random, or if outcomes are clustered, the predictive model must incorporate that feature rather than silently treating it as fixed or independent.

Inspect generated probability vectors as well as the resulting counts. Check that probabilities respect the endpoint’s constraints, that subgroup-specific rates are clinically coherent, and that the signs and magnitudes of \(\delta_g\) and \(D\) accord with the prior’s intended meaning. Check that simulated counts respect the cell sizes. If the prior implies implausible subgroup patterns, consider whether the cause is the marginal probability distributions, the dependence among probabilities, or the outcome model.

Confusing marginal plausibility with joint plausibility. Plausible priors for A and B separately do not guarantee a plausible distribution for \(D\). Inspect paired draws and the subgroup contrast.
Checking only expected counts. A reasonable mean can conceal a wide or unusual predictive distribution. Examine generated outcomes across the joint prior, not just averages.
Calling every extreme count an implausible prior probability. Counts fluctuate even conditional on \(p\). Separate probability-level implications from binomial sampling variation.
Changing marginal values while comparing dependence. To isolate dependence, preserve the same values and probabilities in each subgroup and change only how the subgroup values are paired.

Subgroup checks complement, rather than replace, checks of marginal event rates and treatment contrasts. Separate marginal checks can describe each subgroup’s rate distribution, but cannot establish how subgroup effects move together. A check based only on pooled outcomes can hide subgroup patterns that offset one another. The posterior predictive approach instead conditions on observed data and assesses predictions after updating; the prior predictive check considered here focuses on implications before that update. Neither approach repairs a misspecified subgroup definition or an unsuitable outcome model.

Key takeaway: Generate subgroup outcomes from the full joint prior and the planned trial model. Assess both the subgroup-specific rates and the contrasts between subgroups; marginal priors alone do not determine the joint clinical implications.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

If \(\delta_A=-0.10\) and \(\delta_B=-0.05\), calculate \(D\) and interpret its sign on the treatment-minus-control scale.
For the four values in the worked example, calculate the reverse-order differences \(D\). Why does this comparison isolate dependence rather than change the marginal priors?
A subgroup has a fixed response probability \(p=0.20\) and 10 participants. What is the expected response count conditional on that probability?
Why can two subgroup priors with identical marginal treatment-effect distributions imply different distributions for \(D\)?

Solutions. 1. \(D=-0.05-(-0.10)=0.05\), or 5 percentage points. The treatment-minus-control difference is 5 percentage points larger in subgroup B than in A; this does not mean either effect is beneficial. 2. The differences are \(-0.06,-0.02,0.02,0.06\). Each subgroup still has the same four marginal effect values; only their pairing across subgroups changes. 3. \(E(C\mid p)=np=10(0.20)=2\) responses. 4. \(D\) is a difference of two dependent effects, and its variance includes \(-2\operatorname{Cov}(\delta_A,\delta_B)\). Different dependence can therefore alter the distribution of \(D\) while leaving each marginal distribution unchanged.