Posterior Predictive Checks for Individual PK Fits

Why Check an Individual PK Fit?

An individual posterior can produce precise forecasts and dose recommendations while still failing to reproduce that person’s observed concentrations. A posterior predictive check asks a direct model-adequacy question: if the fitted model and this individual’s posterior were plausible, could replicated concentration data look like the data actually observed? This is a check of the model as used for that individual, not a guarantee that future forecasts will be calibrated or that the population model is correct.

The check must preserve the observation process. If the observed data are assay measurements at specified times after particular doses, a replicated data set should use those same dose histories and sampling times, then include the measurement-error process specified by the model. Simulating only the smooth concentration curve checks a different object: the latent PK trajectory, not the noisy data that were observed.

Model and Assumptions Behind the Check

Suppose the concentration predicted at time \(t_i\), given the administered doses and parameters, is \(C(t_i;\theta)\). For example, with a lognormal observation model,

Here \(Y_i\) is the measured concentration and \(\sigma\) describes log-scale measurement or residual variability. This model assumes positive concentrations, a suitable lognormal error distribution, and conditionally independent errors if the \(\epsilon_i\) are modeled as independent. Other PK models may use additive error, combined error, correlated errors, or an explicit model for values below quantification limits. The check must simulate using the observation model actually fitted; it should not silently replace that model with lognormal error.

The posterior \(p(\theta\mid y)\) must also represent the intended individual-level inference, including the prior information and population-model information used in updating. Dose amount, dose timing, sampling times, units, and any relevant time-varying covariates must be correctly represented. If concentrations are censored or missing, the replication and comparison need to reflect how those observations entered the likelihood. A check based on the wrong observation schedule or an incorrectly recorded dose history can diagnose a data-processing problem rather than a PK-model problem.

For each posterior draw \(\theta_s\), simulate \(y_s^{\mathrm{rep}}\) conditional on that draw. Retain the observed sampling times and dose history. Then compare a feature, or discrepancy, of the observed and replicated data. A useful discrepancy is chosen to target a particular concern: overall magnitude, peak timing, or within-person variability. No single discrepancy can establish that every aspect of a fit is adequate.

Paired Discrepancies and Their Interpretation

A discrepancy may depend on the parameter draw as well as the data. Write it as \(T(y,\theta)\). The paired posterior predictive tail area for an upper-tail check is

The defining feature is the pairing: for each draw \(s\), compare the replicated discrepancy with the observed discrepancy calculated using the same \(\theta_s\). This matters because the reference prediction can change across posterior draws. Comparing every replicate with one fixed observed discrepancy can produce a different, unpaired calculation.

The Monte Carlo approximation follows directly from the predictive simulation. Each pair \((\theta_s,y_s^{\mathrm{rep}})\) is drawn by first drawing from \(p(\theta\mid y)\) and then from \(p(y^{\mathrm{rep}}\mid\theta_s)\). The indicator is one precisely when the replicated discrepancy is at least as large as its observed counterpart. Averaging these indicators estimates the posterior predictive probability of that event. With \(S\) independent draws, the Monte Carlo standard error of the estimated proportion is approximately \(\sqrt{\hat p_B(1-\hat p_B)/S}\), though dependence in simulation output can make the effective Monte Carlo error larger.

A tail area near zero indicates that the observed discrepancy is unusually large relative to replicated discrepancies; a value near one indicates that it is unusually small for an upper-tail statistic. Neither interpretation is a classical hypothesis-test conclusion. The same data are used to form the posterior and to calculate the observed discrepancy, so posterior predictive tail areas are not generally uniform under a correctly specified model and can be conservative. A value near one-half is not proof of fit, and an extreme value is a prompt to investigate the chosen feature, model, data, and simulation—not an automatic instruction to change a dose.

Checks for Timing and Variability

A timing check can focus on when the measured profile reaches its maximum. With discrete sampling, define the observed peak time as the scheduled sampling time at which the largest measured concentration occurs; use a stated tie rule, such as choosing the earliest tied time. Simulate replicated observations at the same sampling times and calculate the same peak-time summary for each replicate. A histogram or table of replicated peak times shows whether the observed peak timing is compatible with the model’s replicated patterns. This is a descriptive predictive comparison; if a tail area is calculated, the discrepancy and its direction must be defined explicitly.

A variability check can ask whether the model reproduces the scatter of an individual’s observations around the parameter-specific PK curve. For log-scale residuals, one possible discrepancy is their sample standard deviation:

This statistic is in log-concentration units. It targets the spread of deviations, not whether the deviations follow the assumed error distribution or whether they have a time pattern. A model can reproduce the overall spread and still miss a rising or falling trend in residuals; that requires a different, deliberately chosen check.

Practical Workflow, Alternatives, and Failure Modes

A common mistake is to calculate \(T(y,\theta_s)\) for each posterior draw, average those observed discrepancies to one number, and compare every replicated discrepancy with that fixed average. That is not the paired upper-tail calculation defined above: it discards the draw-specific reference. The correction is to compare \(T(y_s^{\mathrm{rep}},\theta_s)\) directly with \(T(y,\theta_s)\) for each \(s\), and then average the indicators. A comparison to a fixed threshold can be useful if that threshold is substantively justified, but it should be described as a different, unpaired tail calculation.

Another failure is simulating only \(C(t_i;\theta_s)\) and comparing it to noisy assay values while claiming to check the observed-data model. That omits residual error and can make replicates look artificially smooth. Conversely, simulating error twice inflates variability. Check whether each layer—latent PK, residual error, censoring, and sampling—has been included exactly as intended. Extreme replicated values or implausible timing can also arise from a few influential posterior draws, so inspect the simulated profiles and verify that the posterior simulation itself is adequate.

Posterior predictive checks complement, rather than replace, the earlier material on posterior predictive concentration forecasts and Bayesian model averaging for individual forecasts. A model-averaged check should simulate a model index from its posterior model probabilities, then draw parameters and replicated observations conditional on that model. Residual diagnostics examine structured patterns in observed errors more directly, while posterior predictive checks compare selected features with data generated under the fitted posterior predictive distribution. Neither a satisfactory check nor a moderate tail area proves predictive calibration; calibration of individual predictive intervals is a separate question.

These checks are limited by the data available for one person. Sparse sampling may make peak timing nearly uninformative, and multiple discrepancies can be inspected until one appears reassuring. Small tail areas may reflect a wrong structural PK assumption, a misspecified error model, an incorrectly entered dose, or an unusual observation; the check alone does not distinguish these causes. Report the statistic, its units or scale, simulation design, and uncertainty from finite simulation. Use the check to guide targeted investigation, not as a stand-alone decision rule.