Prior Predictive Quantiles For Clinical Plausibility

Why Prior Predictive Quantiles Matter

A prior predictive check asks what outcomes a trial model and its prior could generate before the trial data are observed. Quantiles give a compact way to describe the range of outcomes the model regards as plausible. In a clinical setting, the summaries should be interpretable: for example, the number of participants with an adverse event, or the mean blood pressure in a planned trial arm.

The purpose is not to decide whether a prior is “right” from a single interval. It is to compare its implied outcomes with clinical knowledge and the planned design. A predictive interval that includes outcomes regarded as impossible or clinically nonsensical is a reason to examine the model, the prior, the outcome definition, and the design assumptions.

Definition: Let $\theta$ denote unknown model quantities, with prior distribution $\pi(\theta)$, and let $T^{\mathrm{rep}}$ be a summary from a hypothetical trial generated under the model. Its prior predictive distribution averages the sampling distribution over the prior: $$ p(t^{\mathrm{rep}})=\int p(t^{\mathrm{rep}}\mid\theta)\pi(\theta)\,d\theta. $$ For a discrete summary with cumulative distribution function $F(t)=P(T^{\mathrm{rep}}\leq t)$, its $q$-quantile is $Q_q=\min\{t:F(t)\geq q\}$. For a continuous summary, $Q_q$ is a value at which $F(Q_q)=q$, when the distribution is continuous and strictly increasing there.

An equal-tailed $100(1-\alpha)\%$ prior predictive interval is formed from the quantiles $Q_{\alpha/2}$ and $Q_{1-\alpha/2}$. Its endpoints describe hypothetical trial summaries, not unknown parameters. For a continuous distribution, the interval has probability $1-\alpha$. For a discrete distribution, its probability is generally at least $1-\alpha$ and can be larger because the quantiles cannot split the probability at an outcome value.

Computing and Interpreting Predictive Quantiles

The summary must be defined before calculating quantiles. Specify the endpoint, the population or arm, the sample size and follow-up period, and the statistic being checked. A count of events among 12 participants is not interchangeable with an event rate in a much larger trial, and a participant-level outcome is not the same summary as a trial-level mean. These choices affect the predictive distribution.

The basic calculation has two stages: average over the prior to obtain the predictive distribution, then find its quantiles or probabilities. If the distribution has a tractable form, calculate them directly. Otherwise, simulate $\theta^{(m)}$ from the prior and then $T^{(m)}$ from the trial model conditional on $\theta^{(m)}$. The empirical quantiles of the resulting summaries approximate the prior predictive quantiles. Simulation accuracy is a separate issue from whether the model and prior are clinically sensible.

Worked Example: Equal-Tailed Interval for an Event Count

Suppose a hypothetical single-arm trial will enroll 12 participants. Let the event probability $p$ have a $\mathrm{Beta}(1,1)$ prior, and, conditional on $p$, let the event count $K$ have a $\mathrm{Binomial}(12,p)$ distribution. Assume participants’ binary outcomes are conditionally independent given $p$, and that the planned sample size and event definition are fixed.

Integrating out $p$ gives a beta-binomial predictive distribution. In this case, each possible count has the same probability:

P(K=k)=\binom{12}{k}\frac{B(k+1,13-k)}{B(1,1)} =\frac{1}{13},\qquad k=0,1,\ldots,12.

Thus $F(0)=1/13$, which is greater than .025, so $Q_{.025}=0$. Also, $F(11)=12/13$, which is less than .975, while $F(12)=1$; hence $Q_{.975}=12$. The equal-tailed 95% interval is 0 to 12 events, and its predictive probability is $13/13=1$. This is greater than .95 because the count is discrete.

The interval is not informative about a narrow range: it includes every possible count. If the clinical team regarded four or more events as outside an acceptable range, then $P(K\geq 4)=9/13\approx .6923$, because counts 4 through 12 account for nine of the 13 equally likely outcomes. That probability flags a substantial conflict with the benchmark under this model. It does not establish that the prior is inappropriate by itself; the benchmark and the model assumptions also need review.

Worked Example: A Binomial Special Case and a Clinical Threshold

Consider a hypothetical trial of 20 participants and a binary adverse event. Suppose the prior places all its probability at $p=.10$, so $p=.10$ is fixed in the predictive calculation. Conditional independence of participants’ outcomes and a fixed sample size give $K\sim\mathrm{Binomial}(20,.10)$. This point-mass prior is a special case of prior prediction; unlike a spread-out prior, it does not represent uncertainty about $p$.

For a binomial count, the individual probabilities are $P(K=k)=\binom{20}{k}(.10)^k(.90)^{20-k}$. The probabilities through $k=5$, rounded to four significant figures, are:

P(K=0,\ldots,5)\approx (.1216,\ .2702,\ .2852,\ .1901,\ .08978,\ .03192).

The first cumulative probability is .1216, so $Q_{.025}=0$. The cumulative probability through 4 is approximately .9568, which is below .975; through 5 it is approximately .9887. Therefore $Q_{.975}=5$, and the equal-tailed 95% interval is 0 to 5 events. Its actual predictive probability is approximately .9887, not exactly .95, because the endpoints are integer counts.

Suppose six or more events is the agreed clinical plausibility threshold for this check. Then $P(K\geq 6)=1-P(K\leq 5)\approx 1-.9887=.0113$. Under the specified model, the chance of six or more events is about .0113. This statement is conditional on the predictive model and its fixed event probability; it is not a probability that a hypothesis is true or a claim about the actual trial outcome.

Continuous Summaries and Predictive Benchmarks

For a continuous summary, prior uncertainty and sampling uncertainty both contribute to the predictive distribution. A simple normal model makes that combination explicit. Suppose, for a fixed planned arm size $n$, individual outcomes are conditionally independent with distribution $N(\mu,\sigma^2)$, where $\sigma$ is known. Let the prior for $\mu$ be $N(m_0,s_0^2)$. The replicate arm mean has conditional distribution $N(\mu,\sigma^2/n)$. Averaging over the prior gives a normal prior predictive distribution with mean $m_0$ and variance $s_0^2+\sigma^2/n$.

Result: Under the normal model just described, $$ \bar{Y}^{\mathrm{rep}}\sim N\left(m_0,\ s_0^2+\frac{\sigma^2}{n}\right). $$ The variance contains $s_0^2$, the prior variance in the unknown mean, and $\sigma^2/n$, the conditional sampling variance of the replicate mean.

Worked Example: Prior Predictive Interval for a Trial Mean

Suppose a hypothetical trial will measure systolic blood pressure in mmHg for 16 participants in an arm. Assume individual outcomes are conditionally independent given the arm mean $\mu$, have known standard deviation $\sigma=12$ mmHg, and follow a normal model. Let the prior for $\mu$ be $N(100,8^2)$, with 100 and 8 expressed in mmHg. These assumptions make the calculation exact; in practice, a fixed known standard deviation and a normal outcome model should be checked for relevance.

The replicate arm mean has predictive mean $100$ mmHg and variance $8^2+12^2/16=64+9=73$ $\mathrm{mmHg}^2$. Its predictive standard deviation is $\sqrt{73}\approx8.544$ mmHg. Using the standard normal 97.5th percentile, approximately 1.96, the equal-tailed 95% interval is:

100\pm1.96\sqrt{73} =100\pm16.75 \approx(83.25,\ 116.75)\text{ mmHg}.

Suppose clinicians specify 80 to 120 mmHg as a broad plausible range for this particular arm mean. The calculated interval lies within those bounds. The predictive probability of falling outside the bounds is $P(\bar{Y}^{\mathrm{rep}}<80)+P(\bar{Y}^{\mathrm{rep}}>120)$. Standardizing gives two equal tails: $2\Phi(-20/\sqrt{73})\approx .0192$. Thus the model assigns probability about .0192 to an arm mean outside the stated range. This is a model-based probability for a hypothetical trial mean, not a probability that the unknown $\mu$ lies outside the range.

Justification, Diagnostics, and Alternatives

The normal predictive result follows by adding independent normal quantities. Write the replicate mean as $\bar{Y}^{\mathrm{rep}}=\mu+\varepsilon$, where, conditional on $\mu$, $\varepsilon\sim N(0,\sigma^2/n)$, independent of the prior draw of $\mu$. Since $\mu\sim N(m_0,s_0^2)$, the sum is normal, with expectation and variance:

E(\bar{Y}^{\mathrm{rep}})=E(\mu)+E(\varepsilon)=m_0, \qquad \operatorname{Var}(\bar{Y}^{\mathrm{rep}}) =\operatorname{Var}(\mu)+E\{\operatorname{Var}(\varepsilon\mid\mu)\} =s_0^2+\frac{\sigma^2}{n}.

This decomposition also explains why checking only a parameter prior can miss important features of trial outcomes: a predictive summary reflects both uncertainty about parameters and sampling variation under the planned design.

A useful diagnostic is to compute both the interval and the probability of crossing a clinically chosen boundary. The interval summarizes central predictive mass, whereas a benchmark probability answers a specific threshold question. The interval endpoints alone may conceal an important probability just outside a clinical limit, and an interval containing the benchmark does not mean every outcome it covers is equally likely or acceptable. Define the benchmark in advance and state the event whose probability is being calculated.

Quantiles may be obtained analytically, from an exact predictive distribution, or by simulation. Exact calculations make discreteness and endpoint probabilities transparent. Simulation is flexible for complicated trial models and summaries, but finite simulation can make estimated quantiles unstable, especially in sparse or irregular distributions. Repeated or sufficiently large simulations and checks of the simulated distribution are useful safeguards; Monte Carlo uncertainty should not be confused with uncertainty represented by the prior predictive distribution itself.

A predictive interval is also different from a credible interval for $\theta$. A credible interval concerns prior probability assigned to parameter values; a prior predictive interval concerns possible replicate trial summaries after accounting for the sampling model. Neither is a frequentist confidence interval. These tools answer different questions, so substituting one for another can lead to incorrect clinical conclusions.

Finally, clinical plausibility is not a purely mathematical property. A range that is appropriate for one endpoint, population, arm, or follow-up period may be inappropriate for another. A mismatch can arise from an overly broad prior, but it can also indicate a misunderstood scale, a wrong sample size, an omitted dependence structure, an unsuitable outcome model, or a benchmark that does not match the defined summary. A plausibility check identifies what to investigate; it does not diagnose the cause automatically.

Common Mistakes

Calling a predictive interval a parameter interval. Its endpoints describe a hypothetical trial summary, not a range for the unknown parameter.
Assuming a discrete equal-tailed interval has exactly its nominal probability. Quantiles for counts cannot divide probability at a single integer; calculate the actual interval probability from the predictive distribution.
Equating “inside the interval” with “clinically acceptable.” Clinical boundaries should be stated explicitly, and their predictive probabilities may be more relevant than central quantiles alone.
Ignoring the design. Changing the sample size, follow-up, endpoint, or analysis summary changes the predictive distribution, even if the prior is unchanged.
Treating a small probability as proof that an outcome cannot occur. A low predictive probability is not zero, and its interpretation depends on the model and the clinical benchmark.

Key takeaway: Prior predictive quantiles summarize the range of trial summaries implied jointly by a prior, a sampling model, and a specified design. Interpret them alongside actual interval probabilities and clinically defined benchmarks, while keeping the outcome and parameter scales distinct.

Check Your Understanding

Answer each question, then compare your reasoning with the brief solutions.

If a discrete predictive distribution has $Q_{.025}=0$ and $Q_{.975}=5$, must the probability from 0 through 5 equal .95?
In the normal example, which two variance contributions make up the predictive variance of the arm mean?
What does $P(K\geq6)\approx.0113$ mean in the binomial worked example?
Name two design or modeling details that should be checked if a predictive interval appears clinically implausible.

Solutions. (1) No. A discrete interval can have probability greater than .95; calculate its probability from the predictive distribution. (2) The prior variance in $\mu$, $8^2=64$, and the conditional sampling variance $\sigma^2/n=12^2/16=9$, both in $\mathrm{mmHg}^2$. (3) Under the specified fixed-$p$ model and planned trial size, the predictive probability of six or more events is about .0113. (4) For example, check the endpoint and scale, planned sample size, follow-up period, dependence assumptions, and whether the sampling model fits the outcome.