Why Check the Control Arm?
A control-arm prior can imply implausible outcomes even when its parameter values initially seem reasonable. For example, a prior may have a sensible average event rate but also allow substantial probability on rates that are incompatible with the intended population or follow-up period. Conversely, outcomes that look unusual in a small planned trial may arise from ordinary sampling variation rather than an implausible prior. A useful check therefore asks what the prior predicts for the control-arm data at the planned sample size, not only what values it assigns to an unknown parameter.
The reference range for this check should have a clinical or design rationale, such as relevant historical control data, a prespecified range of credible baseline risks, or expert input about the endpoint in the trial population. It must refer to the same outcome definition, population, and follow-up period as the planned control arm. A range chosen only because it makes a proposed prior look acceptable is not an independent benchmark.
The object checked should match the clinical question. For a binary endpoint, it might be the control-arm event count or event rate. For a continuous endpoint, it might be the trial mean or the distribution of individual measurements. These are different quantities: an individual outcome is generally more variable than a trial mean, and a prior distribution for a parameter is not itself the distribution of observed trial data.
Binary Control Outcomes: The Beta-Binomial Check
Suppose each of \(n_C\) control participants has a binary outcome, with event probability \(p_C\), and outcomes are conditionally independent given \(p_C\). Write \(K_C\) for the number of events. Then \(K_C\mid p_C\sim\mathrm{Binomial}(n_C,p_C)\). A common prior is \(p_C\sim\mathrm{Beta}(a,b)\), with \(a,b>0\). Combining this prior with the binomial sampling model gives the beta-binomial prior predictive distribution:
where \(B(\cdot,\cdot)\) is the beta function. The formula integrates over possible values of \(p_C\); it does not plug in only the prior mean. The predictive mean and variance are
The variance includes both binomial sampling variation and uncertainty about the event probability. In particular, it can exceed the variance of a binomial distribution evaluated at the prior mean. A check based only on \(n_C\mu_C\) therefore misses how widely the prior predicts trial outcomes.
Worked Example: A Broad Prior for a Control Event Rate
Consider a hypothetical trial with \(n_C=20\) control participants and a binary event measured over a specified follow-up period. Suppose \(p_C\sim\mathrm{Beta}(1,1)\), a uniform prior on event rates from 0 to 1. Clinical reviewers have justified 10%–30% as a reference range for the control event rate in this population and period. What does the prior predict for the number of events?
For this prior, the beta-binomial probability is the same for every possible count \(k=0,\ldots,20\): $$ P(K_C=k)=\binom{20}{k}\frac{B(1+k,1+20-k)}{B(1,1)} =\frac{1}{21}. $$ The clinically referenced count range is \(0.10(20)=2\) through \(0.30(20)=6\), inclusive. There are five integer counts in that range, so $$ P(2\leq K_C\leq 6)=\frac{5}{21}=0.2381. $$
Thus, under this prior, only about 23.8% of prior-predicted trials have a control event count in the reference range; about 76.2% fall outside it. The prior predictive mean is \(20(1/2)=10\) events, or a 50% event rate, also above the reference range. These results flag a serious mismatch for this hypothetical setting. They do not prove the prior is wrong: the reference range, population, follow-up, and model assumptions should first be checked.
Worked Example: A Plausible Mean Can Hide Substantial Predictive Spread
Suppose a separate hypothetical trial plans \(n_C=40\) control participants and uses \(p_C\sim\mathrm{Beta}(2,18)\). The prior mean event rate is \(\mu_C=2/20=0.10\). Assume a clinically justified reference range of 5%–15% for the same event definition and follow-up period. Calculate the predictive mean, predictive standard deviation, and the probability of observing no events.
The predictive mean count is $$ E(K_C)=40(0.10)=4\text{ events}. $$ Here \(\kappa=20\), so the predictive variance and standard deviation are $$ \operatorname{Var}(K_C) =40(0.10)(0.90)\frac{40+20}{20+1} =\frac{216}{21}=10.29, \qquad SD(K_C)=\sqrt{10.29}=3.207\text{ events}. $$ In rate units, the predictive standard deviation is \(3.207/40=0.0802\), or about 8.02 percentage points. The reference range corresponds to 2–6 events. Although the predictive mean of 4 events lies inside that range, the standard deviation is large relative to its width; the mean alone is not enough to establish that the prior predicts concentrated outcomes there.
The probability of zero events can be obtained from the beta-binomial distribution: $$ P(K_C=0)=\frac{B(2,58)}{B(2,18)} =\frac{18\times19}{58\times59} =0.09994. $$ Thus the prior assigns about a 9.99% probability to a trial with no control events. Since zero is below the reference range, this is a concrete indication that the prior allows an outcome worth clinical review. The check does not imply that a zero-event trial is impossible or invalid.
Continuous Control Outcomes at the Planned Sample Size
For a continuous endpoint, a simple illustrative model is \(Y_{Ci}\mid\mu_C\sim N(\mu_C,\sigma^2)\), independently for \(i=1,\ldots,n_C\), with known \(\sigma\), and prior \(\mu_C\sim N(m_0,s_0^2)\). Under these assumptions the trial mean has prior predictive distribution \(\bar{Y}_C\sim N(m_0,s_0^2+\sigma^2/n_C)\). This follows by writing \(\bar{Y}_C=\mu_C+\epsilon\), where the sampling error \(\epsilon\sim N(0,\sigma^2/n_C)\) is independent of \(\mu_C\). Its variance is therefore the sum of prior variance and sampling variance.
Worked Example: Checking a Planned Control-Arm Mean
In a hypothetical trial, the control endpoint is systolic blood pressure at a specified follow-up visit, measured in mmHg. Assume independent normal outcomes with known \(\sigma=15\) mmHg and prior \(\mu_C\sim N(120,10^2)\), where the prior mean is 120 mmHg and prior standard deviation is 10 mmHg. The planned control sample size is \(n_C=36\). Suppose clinical and historical review supports 110–130 mmHg as a reference range for the trial mean. Find the prior predictive probability that \(\bar{Y}_C\) falls in this range.
The predictive variance of the mean is $$ 10^2+\frac{15^2}{36}=100+6.25=106.25\text{ mmHg}^2, \qquad SD(\bar{Y}_C)=\sqrt{106.25}=10.31\text{ mmHg}. $$ Standardizing the endpoints of the reference range gives $$ P(110\leq\bar{Y}_C\leq130) =\Phi\left(\frac{130-120}{10.31}\right) -\Phi\left(\frac{110-120}{10.31}\right) =\Phi(0.970)-\Phi(-0.970) \approx0.668. $$ So about 66.8% of prior-predicted trial means lie in the stated reference range. The prior predictive central 95% interval is approximately \(120\pm1.96(10.31)=[99.8,140.2]\) mmHg, which extends well beyond it. These calculations flag a possible mismatch between the prior predictive spread and the clinical benchmark, even though the prior mean is centered in the range.
The conclusion depends on the model: outcomes are treated as independent and normal, and \(\sigma\) is treated as known. If those assumptions are not defensible, the predictive calculation should reflect a more suitable sampling model rather than treating this example’s normal formula as universal.
How to Conduct and Interpret the Check
A practical check begins with the clinical outcome, not with a convenient prior formula. Specify the control-arm endpoint, its time point or follow-up window, the analysis population, and the planned \(n_C\). Then establish a reference range in the same units and for the same summary that will be checked. For binary outcomes, translate a rate range into the corresponding integer event counts, taking care with inclusive endpoints and rounding. For continuous outcomes, state whether the benchmark concerns individual measurements, a population mean, or a trial mean.
The reference range is a benchmark for judging implications, not a statistical null hypothesis. A prior predictive outcome outside that range is a reason to investigate the assumptions. It is not, by itself, evidence that the observed data are unlikely, since no data have yet been observed. Likewise, a prior predictive distribution that agrees with a benchmark does not establish that the prior is uniquely correct or that the data model fits future observations.
Control-arm checks should remain distinct from checks of a treatment contrast. In a model with separate control and treatment parameters, inspect the control-arm marginal prior and likelihood to assess control outcomes. A plausible treatment contrast does not guarantee a plausible control event rate or endpoint distribution. Conversely, an implausible control prediction should not be explained away by the treatment-effect prior.
| Check | What it describes | What it cannot establish alone |
|---|---|---|
| Prior parameter summaries | Locations and spread of unknown control parameters | Whether trial outcomes at planned \(n_C\) are plausible |
| Prior predictive check | Outcomes generated from the prior and sampling model at planned \(n_C\) | That the prior or model is uniquely correct |
| Posterior predictive check | Outcomes generated using information in observed data | Whether the prior was clinically sensible before seeing the data |
Common Pitfalls and Limitations
- Checking only the prior mean. A reasonable mean can coexist with a wide predictive distribution or substantial probability on extreme counts. Check dispersion and relevant probabilities as well as location.
- Using the wrong sample size. Predictive trial outcomes depend on \(n_C\). Use the planned control-arm size and account for allocation, follow-up, and the endpoint definition relevant to the analysis.
- Mixing up a parameter and an outcome. A prior range for \(p_C\) concerns uncertainty about an event probability; a range for \(K_C/n_C\) concerns a realized trial rate. They are related but not interchangeable.
- Using an unmatched reference range. A rate from a different population, time horizon, or event definition may not be a meaningful benchmark for the planned control arm.
- Treating a flag as a verdict. A mismatch can arise from the prior, the sampling model, the benchmark, or a mistaken translation into trial outcomes. Investigate which assumption is responsible before changing the model.
Analytic predictive distributions are useful when available; simulation from the prior and sampling model is a general alternative for more complicated designs. Either approach depends on the model and on the clinical justification for the reference range. A check can expose problematic implications, but it cannot supply missing clinical knowledge or guarantee that the future trial will resemble its assumptions.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A binary control endpoint has \(n_C=10\) and prior \(p_C\sim\mathrm{Beta}(1,1)\). What is the prior predictive probability of exactly 4 events?
- For \(p_C\sim\mathrm{Beta}(2,18)\) and \(n_C=40\), state the predictive mean event count and explain why it does not describe the full predictive distribution.
- In a normal model with \(\mu_C\sim N(50,4^2)\), known \(\sigma=6\), and \(n_C=9\), what are the predictive mean and variance of \(\bar{Y}_C\)?
- A clinical reference range is specified for individual endpoint values, but a check compares it with the prior predictive distribution of the trial mean. What is wrong with this comparison?
Solutions. (1) Under the uniform beta prior, the beta-binomial predictive probability is \(1/(n_C+1)\) for each count, so \(P(K_C=4)=1/11=0.09091\). (2) The predictive mean is \(40(2/20)=4\) events. It does not convey predictive spread, which also reflects uncertainty in \(p_C\); for this example the variance is \(10.29\) events squared. (3) The predictive mean is \(50\), and the variance is \(4^2+6^2/9=16+4=20\) outcome-units squared. (4) The benchmark and predictive quantity differ: individual outcomes include individual-level variation, whereas a trial mean averages over participants. Compare like with like by using an individual-outcome predictive distribution or a reference range for the trial mean.