Visual Predictive Checks for Delayed Concentration-Effect Data

Why Check Simulated Predictions?

A delayed concentration-effect model makes predictions about more than a fitted curve. It predicts distributions of plasma concentrations and pharmacodynamic (PD) observations over time, and it predicts how those observations relate when plasma concentration rises and falls. A curve can pass near many observed points while still missing their variability, the timing of the response, or the separation between rising and falling concentration-effect paths. A visual predictive check (VPC) assesses these features by comparing the observed data with replicated data generated under the fitted model.

For delayed effects, useful views include plasma and PD observations against time, PD observations against plasma concentration, and a display that preserves whether each concentration was observed on the rising or falling part of the profile. The displays complement residual diagnostics: a VPC asks whether the model can generate data with patterns like those observed, not simply whether a single predicted curve lies close to them.

Constructing a VPC for Delayed PK/PD Data

Start by specifying the observation design to reproduce: dose and infusion history, sampling times, covariates, number of subjects, and which plasma and PD measurements are available. Simulate complete datasets under the fitted PK/PD model, including relevant between-subject and residual variability. Preserve the paired plasma and PD values from each simulated subject. The effect-compartment dynamics described in earlier material on effect-site models determine how concentration history contributes to the predicted response; the VPC checks the resulting observable predictions.

At each chosen time or within each prespecified time bin, calculate summaries such as the 10th, 50th, and 90th percentiles of observed plasma or PD values. Calculate the same summaries separately in every simulated dataset. For each summary, the simulated values form a distribution. Its central 90% prediction interval can be formed from the 5th and 95th percentiles across simulated datasets. Plot the observed summary and the interval over time. The observed percentile is a statistic from the actual data; the interval describes the variability of that statistic under repeated datasets generated by the fitted model.

Quantile conventions must be consistent and stated, especially for small datasets. In the examples below, empirical percentiles use the nearest-rank rule: the percentile \(q\) is the ordered observation at rank \(\lceil qn\rceil\), where \(n\) is the number of observations and \(\lceil\ \rceil\) denotes rounding upward. Thus, for 20 values, the nearest-rank median is rank 10, not the average of ranks 10 and 11. A 90% interval across 20 simulated values uses ranks 1 and 19 for its 5th and 95th percentile endpoints. With many simulated datasets, endpoint discreteness is less consequential, but the convention should still be documented.

Time bins should reflect the sampling design and the dynamics of interest. Bins that are too narrow can contain too few observations for stable percentiles; bins that are too wide can hide a peak or a lag. A useful display makes bin boundaries and sample counts clear. If sampling is sparse near the response peak or during washout, no choice of plotting style can supply the missing information.

Checking PD Over Time and Against Concentration

Construct a PD-versus-time VPC as well as a plasma-versus-time VPC. A delayed response can have a later peak than plasma concentration, and its spread may change over time. An observed PD median outside its simulated prediction interval for several consecutive bins, or observed lower and upper percentiles that repeatedly fall outside their corresponding bands, can indicate a mismatch in typical response, variability, or timing. A single isolated point beyond a band is less compelling: prediction intervals are not simultaneous bands, and random variation can place some observed summaries outside them.

A concentration-effect display provides a complementary check. Use the paired plasma concentration and PD measurement from the same subject and time. Define concentration bins before interpreting the pattern; within each bin, compare observed PD percentiles with the distribution of those percentiles across simulations. The simulation must apply the same binning rule to its paired concentration-effect data. Otherwise, differences in which observations enter a bin can be mistaken for differences in the response model.

Pooling all observations into concentration bins can obscure hysteresis because the same plasma concentration may occur during both rising and falling phases. When the data permit, display or summarize those branches separately, while retaining the sequence over time. For an increasing PD endpoint, a conventional effect-site delay often gives a lower effect at a given plasma concentration on the rising branch than on the falling branch; on a concentration-horizontal, effect-vertical plot this is counterclockwise hysteresis. For endpoints whose numerical direction is reversed, interpret the response scale explicitly rather than assigning a loop direction by appearance alone.

Why the Predictive Intervals Work

The VPC's comparison has a direct repeated-sampling interpretation under the fitted model. Let \(T\) be a chosen data summary, such as the observed PD median in a time bin. Simulate \(M\) datasets under the model and calculate \(T_1,\ldots,T_M\) using exactly the same procedure. The simulated values approximate the model's sampling distribution for \(T\), conditional on the design and the parameter values used for simulation.

If the model and parameter values were the data-generating process, and a central 90% interval were calibrated exactly, repeated datasets would place \(T\) outside that interval about 10% of the time. This follows because the lower and upper tails each contain about 5% of the sampling distribution: \(P(T<L)\approx0.05\) and \(P(T>U)\approx0.05\), so \(P(T<L\text{ or }T>U)\approx0.10\). Finite simulation counts and discrete data make the realized rate approximate rather than exact. Moreover, checking many bins and summaries increases the chance that at least one falls outside its interval. Therefore, a VPC is interpreted through the pattern and coherence of discrepancies, not by treating any single crossing as a formal rejection.

Diagnostics, Alternatives, and Limitations

A useful VPC distinguishes several types of mismatch. A consistent vertical offset in PD summaries suggests that typical response predictions are misplaced. Observed percentiles wider than simulated bands can indicate underpredicted variability; narrower percentiles can indicate overpredicted variability. A simulated response peak that systematically occurs too early or too late, or simulated rising and falling branches that are too close together, points to a timing or hysteresis-magnitude mismatch. A plasma VPC that is adequate while the PD VPC is not can help localize the problem to the concentration-effect component, although it does not prove that the PK component is fully correct.

Check sample counts, bin boundaries, and the time ordering of the data before attributing a discrepancy to the model. Unequal numbers of observations across bins, sparse observations near a peak, influential subjects, or an inconsistent rule for assigning branches can create unstable or misleading summaries. Repeating the check with a modestly different, defensible binning scheme can reveal whether a conclusion depends on an arbitrary boundary; it should not be used to search for a preferred result.

A VPC is descriptive, not a parameter-estimation method or a formal test with a single p-value. Residual diagnostics focus on observed-minus-predicted errors and can expose patterns at the observation level; a VPC shows whether replicated datasets reproduce distributions and trajectories. A single fitted curve is simpler to display but omits the variability expected in new data. Simulation-based alternatives can summarize discrepancies numerically or account more explicitly for parameter uncertainty, but they still depend on model assumptions and correct simulation. A conventional VPC using fixed parameter estimates is conditional on those estimates; it does not automatically include uncertainty in them.

A common mistake is to regard every observed percentile as required to lie inside its 90% prediction interval, then declare model failure if any one does not. The interval applies to a particular statistic under repeated simulation, and about 10% of such statistics are expected to fall outside a calibrated 90% interval. The correction is to examine repeated, structured departures across relevant summaries, while considering the number of comparisons, sample size, and uncertainty. Conversely, a plot that looks acceptable is not proof that the model is uniquely identified, that its parameters are precise, or that its mechanistic interpretation is true.