Defining the Predictive-Check Target

What Is the Predictive-Check Target?

A predictive check is only interpretable once its target is clear. A statement such as “the model reproduces concentrations” is incomplete: it does not say which concentrations, which feature of their distribution, or how repeated measurements are combined. The predictive-check target is the precisely defined data summary whose observed value will be compared with the values that the fitted model can generate.

For a population PK data set, let \(\mathbf{y}_{obs}\) denote the observed concentrations and let \(\mathcal{D}\) denote the relevant design: for example, subjects, doses, sampling times, and any stratification used in the summary. Write \(T(\mathbf{y};\mathcal{D})\) for a specified statistic calculated from data \(\mathbf{y}\) under that design. The target has two connected parts: the value \(T(\mathbf{y}_{obs};\mathcal{D})\) calculated from the observed data, and the distribution of \(T(\mathbf{y}^{rep};\mathcal{D})\) when replicate data \(\mathbf{y}^{rep}\) are generated under the fitted model.

For example, “the 75th percentile of concentration at 4 hours” is a target only when the relevant subjects and observations, the time rule for including records, and the quantile convention are stated. The observed data give one 75th-percentile value. Each replicate data set gives another value. The check concerns whether the observed value is plausible relative to the replicate values—not whether it equals one particular simulated value.

Define the Summary Before Comparing It

A population PK model describes concentrations in relation to dose, time, subject characteristics, between-subject variability, and residual variability. A generic representation is

where \(Y_{ij}\) is the concentration for subject \(i\) at observation \(j\), \(t_{ij}\) is the sampling time, \(\theta\) denotes population-model parameters, \(\eta_i\) denotes subject-level random effects, \(\mathbf{x}_i\) denotes relevant subject and dosing information, and \(\epsilon_{ij}\) denotes residual error. This expression is schematic: the specific structural, variability, and residual-error models depend on the analysis.

For a plug-in predictive check, replicate data are generated using the fitted parameter estimate \(\hat{\theta}\), with the model’s random effects and residual error represented according to the fitted model. The target must say which design information is retained and which model-generated quantities vary. In the examples below, the subject and sampling design are retained, and the population predictive replicates draw subject effects and residual error according to the fitted model. Parameter-estimation uncertainty is not added unless it is explicitly part of the predictive procedure.

The statistic must also be identical in concept and calculation for observed and replicate data. If the observed target is a time-specific median, calculate that same time-specific median in every replicate. If the observed target is a contrast between two time-specific medians, calculate the contrast in every replicate rather than comparing it with an unrelated predicted mean.

Why Replicate Summaries Form a Distribution

A replicate data set is random under the model, so applying a fixed summary rule to it produces a random statistic. This can be seen by writing the replicate data as a function \(G\) of the design and model-generated random quantities. With fitted parameters held fixed, the replicate target is \(T(G(\mathcal{D},\eta,\epsilon;\hat{\theta});\mathcal{D})\).

Here \(\mathbb{I}\{\cdot\}\) equals 1 when its condition is true and 0 otherwise. The integral averages that indicator over the model’s distribution for the random quantities used to generate replicates. Thus, the distribution of the target follows from the data-generation model and the chosen summary rule. If the model specifies dependence among random quantities, their joint distribution belongs inside the integral; independence should not be assumed unless the model specifies it.

This also explains why the observed target is not itself a predictive distribution. It is one number, or one vector of numbers. The predictive distribution describes how that same quantity varies over replicate data sets. In practice, a finite collection of replicates approximates the distribution; its summaries should not be mistaken for the exact model-based distribution.

The Design and Data Rules Are Part of the Target

A summary cannot be defined by its formula alone. The data records included in it matter. In PK data, sampling times may differ across subjects; the number of observations at different times may also differ. If the intended target is a 2-hour median and an 8-hour median, pooling all records into one median changes the target. Likewise, applying different inclusion rules to observed and replicate data means the two calculated quantities no longer answer the same question.

A complete target specification should state, as relevant:

• the population or subject set represented by the summary;
• the measurement being summarized, its units, and any transformation;
• the time point, time window, or other rule for selecting records;
• how repeated observations are combined, and how quantiles are defined;
• which design features are retained when generating replicates; and
• how missing or otherwise excluded records are handled in both calculations.

The point is not that every check must use a complicated statistic. It is that the definition must be clear enough that a second analyst could calculate the same summary from the same data and apply the same rule to replicate data.

Individual-Level Summaries Need an Explicit Aggregation Rule

The same subject can contribute several concentrations. A summary across all records gives subjects with more observations more influence; a summary that first calculates one quantity per subject gives each subject one contribution. These are different targets, not alternative calculation tricks for the same target.

Diagnostics, Alternatives, and Common Mistakes

Before interpreting a predictive comparison, check that its target is operationally unambiguous. A useful diagnostic review asks whether observed and replicate calculations use the same units, transformations, time rules, subject weighting, and inclusion criteria. It also checks whether the replicated design matches the design named in the target. If not, an apparent discrepancy may reflect inconsistent processing rather than model behavior.

The target also distinguishes a predictive check from other ways of reviewing fit. Goodness-of-fit plots and residual summaries examine discrepancies at the observation level or through residual patterns. A predictive check instead compares a statistic of the observed data with the distribution of that statistic under the fitted model. A point prediction, such as a typical concentration curve, is another object: comparing an observed quantile directly with a model-predicted mean confounds different summaries. Population-level and conditional predictive checks are alternative predictive targets; the chosen check must state what is held fixed and what is generated, without treating those choices as interchangeable.

• Pitfall: treating a replicate summary as the observed target. “The model predicts a median of 4 mg/L” can hide the fact that simulated medians vary. Correction: distinguish the observed median from the distribution of replicate medians.
• Pitfall: comparing unlike quantities. An observed 75th percentile is not directly comparable to a model-predicted mean. Correction: calculate the same statistic from observed and replicate data.
• Pitfall: ignoring the observation schedule. A pooled quantile can change when subjects contribute different numbers or timings of samples. Correction: define the record-selection and aggregation rules explicitly.
• Pitfall: claiming more than the target addresses. Agreement for one summary does not establish agreement for all features of the data. Correction: state the specific feature assessed and recognize that other targets may reveal different discrepancies.

A target is limited by its definition. A time-specific median can assess central concentration behavior at that time, but it does not fully describe distribution tails, individual trajectories, or behavior at other times. A check is therefore evidence about its stated summary under its stated design—not a universal pass-or-fail statement about the entire model.