Posterior Predictive Checks for Translation Models

Why Check Predictions Against Animal Data?

A translational model may produce plausible human predictions while still failing to reproduce important features of the animal data used to inform it. A reasonable-looking posterior interval for clearance, potency, or another model parameter does not by itself show that the model can reproduce the observed concentration-time or response patterns. Posterior predictive checks address this different question: if the fitted model generated new animal data under the observed study design, would those data resemble what was actually observed?

This check is useful before relying on a model for translation. A mismatch in late concentrations, exposure, or dose-response behavior can point to a weakness in the animal model that is relevant to extrapolation. A good check does not establish that the model is biologically true, nor does it validate a human prediction. It tests whether selected features of the observed animal data are plausible under the fitted model.

The Posterior Predictive Distribution

Let \(y\) denote the observed animal data, \(\theta\) the model parameters, and \(p(y\mid\theta)\) the likelihood under the translational model. After observing \(y\), the posterior distribution \(p(\theta\mid y)\) represents uncertainty about the parameters under the chosen model and prior. The posterior predictive distribution for replicated data \(y^{rep}\) is

This distribution includes two sources of variation: uncertainty about \(\theta\), and variation in new observations conditional on \(\theta\), such as animal-to-animal variability and assay error if those are represented in the model. The prediction must reproduce the relevant design: for example, the same species, dose, sampling times, number of animals, and assay scale. If a check instead simulates a different design, its comparison may not answer the intended question.

In practice, the integral is approximated by simulation. For each posterior draw \(\theta^{(m)}\), simulate a replicated data set \(y^{rep,(m)}\) from the model’s observation distribution. Calculate the same summary or diagnostic quantity for the observed and replicated data. The posterior predictive distribution of that quantity describes how it varies across these replications.

A useful way to see why both sources of variation matter is the variance decomposition for a summary \(T\):

The first term is average variation among replicated observations at fixed parameters; the second is variation in the expected summary across posterior parameter values. A parameter credible interval reflects uncertainty in parameters, whereas a posterior predictive interval for observed data also reflects observation-level variability. These intervals answer different questions.

Choosing a Check That Can Reveal a Relevant Failure

A check is only informative about the feature it examines. For animal PK, candidate summaries include AUC, Cmax, trough concentration, or concentration at a prespecified time. For PD, they might include response at a dose, the number of animals exceeding a response threshold, or a feature of the time course. Examine the full profile as well as aggregate summaries when timing or shape matters: a model can reproduce AUC while systematically missing late concentrations.

For a scalar summary \(T\), an upper-tail posterior predictive area can be estimated from \(M\) replicated data sets as

where \(I\{\cdot\}\) is 1 when the stated condition holds and 0 otherwise. A small upper-tail area means relatively few replicated summaries reached or exceeded the observed value. A corresponding lower-tail area counts replicates at or below the observed value. These are descriptive measures of model-data conflict for the chosen summary; they are not ordinary frequentist p-values, and they do not give the probability that the model is true or false.

The assumption behind the comparison is that the posterior and likelihood are adequate representations for the purpose of the check, and that the replicate process matches the observed sampling and measurement process. The summary should be chosen because it is scientifically relevant, not because it makes the fitted model look favorable. Since the same data inform the posterior and are then checked, posterior predictive checks can have limited power to expose some forms of misspecification. A reassuring check is not proof of adequate extrapolation.

Look for Patterns, Not Just One Number

The same comparison can be made across time, dose, species, or animal-level summaries. Overlay observed data with posterior predictive intervals or display replicated curves alongside the observed profile. A useful diagnostic asks whether a mismatch is isolated or structured. For example, consistently high observed late concentrations may indicate a time-dependent failure that an AUC check can obscure. Systematic departures across dose groups may matter more than one unusual group, though the observed design and sample size affect how much evidence each pattern provides.

Discrepancy statistics can formalize such checks. A discrepancy \(T(y)\) is a function of the data, possibly including a model parameter. Compare \(T(y)\) with \(T(y^{rep})\) computed in the same way for every replicate. A simple standardized comparison is

This summarizes how far the observed statistic is from the predictive mean in predictive standard deviation units. It is not automatically a standard normal test statistic; its interpretation depends on the predictive distribution, and a tail area or plot may communicate a skewed or discrete distribution better. For instance, a count of responders is discrete, so its predictive distribution should be compared directly rather than treated as continuous without justification.

Interpreting Checks and Common Failure Modes

Posterior predictive checks are most useful when their target is explicit and their scope is kept in view. A collection of plots and summaries can reveal problems that one statistic misses, but many checks also create opportunities to notice chance extremes. Prioritize features that matter for the translational question and interpret a collection of checks as a diagnostic review rather than a battery of independent hypothesis tests.

• Checking only a pooled summary. A model can match overall AUC while missing a species-specific or time-specific pattern. Examine relevant strata and profiles.
• Confusing parameter uncertainty with predictive variation. A narrow posterior interval for a parameter does not imply that future animal data will vary narrowly. Use replicated observations for data-level checks.
• Treating a tail area as a model probability. An upper-tail area of 0.01 does not mean there is a 1% probability that the model is correct. It describes how often replicated summaries under the fitted posterior predictive distribution were at least as extreme as the observed summary.
• Overinterpreting a reassuring result. The posterior uses the observed data, so it can produce checks that appear satisfactory even when some model failures are difficult to detect. Lack of an obvious discrepancy is not independent validation.
• Simulating the wrong observation process. Omitting animal variability, assay error, or the actual sampling schedule can make the predictive distribution incomparable to the observed data.

Posterior predictive checks differ from parameter summaries because they examine model-generated data, not just plausible parameter values. They also differ from checks on held-out or external data: the observations being checked contributed to the posterior, whereas an independent validation data set provides an out-of-sample assessment. Residual plots can make local patterns easier to see, while posterior predictive displays put observed features alongside the variation the model expects. These approaches are complementary, not interchangeable.

Finally, a successful animal-data check is a necessary diagnostic step, not a guarantee that cross-species assumptions hold in humans. It shows only that selected observed animal features are reasonably reproduced under the model and its assumptions. The biological relevance of the species, the mapping of parameters, and uncertainty in extrapolation still require separate consideration.