Prediction-Corrected Visual Predictive Checks

Why Correct a Visual Predictive Check?

A visual predictive check (VPC) compares summaries of observed concentrations with the corresponding summaries from replicate datasets simulated under a fitted population pharmacokinetic model. When observations in one time bin have substantially different typical predictions, a raw concentration quantile can mix differences in typical concentration with differences in variability or model fit. Prediction correction puts each concentration on a common reference scale within its bin before those summaries are compared.

The correction is a normalization, not a new model fit and not a transformation that removes all effects of study design or patient characteristics. It is useful when the predictions used for normalization vary within a bin and a multiplicative rescaling is a reasonable way to place concentrations on a common scale. The basic VPC concepts of simulated reference envelopes and bin-level summaries are assumed here.

Definition: For observation $i$ in bin $b$, let $C_i$ be the observed concentration, $m_i$ the model-predicted typical concentration at that observation's sampling time and design, and $M_b$ the median of the $m_i$ values in that bin. The prediction-corrected concentration is $$ C_{i,\mathrm{pc}} = C_i \frac{M_b}{m_i}. $$ The ratio $M_b/m_i$ is the prediction-correction factor. It is dimensionless, so $C_{i,\mathrm{pc}}$ has the same concentration units as $C_i$.

Deriving the Correction

The correction can be understood in two steps. First, express the observation relative to its own typical prediction as $C_i/m_i$. This ratio describes how large the observed concentration is compared with the model's typical prediction at that record's design. Second, multiply that relative value by the bin reference prediction $M_b$, which returns the result to concentration units:

\frac{C_i}{m_i} \times M_b = C_i\frac{M_b}{m_i} = C_{i,\mathrm{pc}}.

Thus, a concentration equal to its own prediction becomes $M_b$ after correction. A concentration 20% above its prediction becomes $1.20M_b$. The operation preserves the observation's proportional deviation from its own prediction while expressing that deviation around the common reference prediction for the bin.

For a VPC, the same record-specific factor must be applied to the simulated concentrations corresponding to that record. Write $C^*_{ir}$ for the simulated concentration for record $i$ in replicate dataset $r$. Then

C^*_{ir,\mathrm{pc}} = C^*_{ir}\frac{M_b}{m_i}.

The predictions $m_i$ and reference $M_b$ are determined from the fitted model and the observation design, and are held fixed across simulation replicates. The observed and simulated values are therefore transformed using the same factor for each record. After correction, calculate the chosen quantiles within each bin for the observed data and for every simulated replicate; use the replicate quantiles to form the simulation-based reference envelope.

Conditions for a meaningful correction: Predictions used as divisors must be positive and finite. They should represent typical model predictions at the actual sampling times and relevant design information, not predictions estimated from the observed concentrations themselves. The multiplicative rescaling should also be scientifically reasonable over the range of predictions in the bin. Apply the identical record-specific factors to observed and simulated values.

Worked Calculations

Worked Example: Correcting Observed Concentrations

Suppose a hypothetical time bin contains five observations. Concentrations and predictions are in mg/L. The model predictions are $1, 2, 3, 4,$ and $8$ mg/L, so their median is $M_b=3$ mg/L. The observed concentrations are $0.8, 2.4, 4.5, 3.2,$ and $12.0$ mg/L.

For the first record, the correction factor is $3/1=3$, giving $0.8\times3=2.4$ mg/L. For the second, the factor is $3/2=1.5$, giving $2.4\times1.5=3.6$ mg/L. For the third, the factor is $3/3=1$, giving $4.5$ mg/L. For the fourth, the factor is $3/4=0.75$, giving $3.2\times0.75=2.4$ mg/L. For the fifth, the factor is $3/8=0.375$, giving $12.0\times0.375=4.5$ mg/L.

The corrected values are therefore $2.4, 3.6, 4.5, 2.4,$ and $4.5$ mg/L. Their sample median is $3.6$ mg/L. The uncorrected observations had a median of $3.2$ mg/L; the two medians differ because each record was rescaled by a different factor. Prediction correction is not generally equivalent to multiplying an uncorrected bin quantile by one constant.

Worked Example: Correcting a Simulated Replicate

Use the same five records and prediction-based factors from the preceding calculation: $3, 1.5, 1, 0.75,$ and $0.375$. Suppose one hypothetical simulated replicate produces concentrations of $1.0, 2.0, 3.6, 4.8,$ and $7.2$ mg/L at those records.

Apply the factors record by record. The corrected replicate concentrations are $1.0\times3=3.0$, $2.0\times1.5=3.0$, $3.6\times1=3.6$, $4.8\times0.75=3.6$, and $7.2\times0.375=2.7$ mg/L. In sorted order they are $2.7, 3.0, 3.0, 3.6,$ and $3.6$ mg/L, with median $3.0$ mg/L.

This is one replicate's corrected median for the bin. Repeat the calculation for every simulated replicate to obtain the distribution of corrected medians and its reference envelope. Do not compare the observed corrected median with an envelope built from uncorrected simulations: the two summaries are on different scales.

Reading a Prediction-Corrected VPC

A prediction-corrected VPC is read by comparing corrected observed quantiles with the corresponding simulation-based reference envelopes, separately for each bin. The correction changes the values used to calculate those quantiles; it does not change the basic meaning of a VPC envelope. For example, if the observed corrected median repeatedly lies above the upper edge of the simulated median envelope, the observed central concentrations are higher than the model's simulated central concentrations in those bins. A single point outside an envelope may occur by chance; a repeated, coherent pattern is more informative than an isolated deviation.

Worked Example: Interpreting Corrected Quantiles

Consider a hypothetical prediction-corrected VPC with concentrations in mg/L. The table gives observed corrected 10th, 50th, and 90th percentiles. For each percentile, the simulation range is the lower-to-upper edge of its reference envelope.

Bin	Observed corrected 10th	10th-percentile envelope	Observed corrected 50th	Median envelope	Observed corrected 90th	90th-percentile envelope
A	1.2	0.9–1.5	3.6	3.1–3.9	6.8	6.0–7.4
B	1.5	1.0–1.6	5.2	4.0–4.8	8.1	7.0–9.0

In bin A, all three observed quantiles fall within their respective envelopes. This is compatible with the model's simulated quantiles for that bin, although it does not prove that the model is correct. In bin B, the observed 10th and 90th percentiles are within their envelopes, but the observed median of $5.2$ mg/L is above the median envelope's upper edge of $4.8$ mg/L. The central part of the observed corrected distribution is therefore higher than the simulated central part in that bin. Because the outer quantiles do not show the same clear departure, the pattern is not simply that every part of the distribution is shifted upward. This graphical finding motivates model and data review; it is not, by itself, a formal test or a causal explanation.

Diagnostics, Alternatives, and Limitations

Before interpreting the plot, check the calculation and the conditions that make it meaningful. Inspect the prediction values in each bin and the factors $M_b/m_i$. Very small positive predictions can produce very large factors, amplifying small observed or simulated values and making the corrected plot sensitive to a few records. Predictions that are zero, negative, or non-finite cannot be used in this ratio. Also verify that predictions correspond to the correct sampling records and that observed and simulated data received the same factors.

Check the scale. Corrected values should retain the original concentration units. A factor such as $M_b/m_i$ has no units.
Check prediction variation. If predictions differ greatly within a bin, the correction may be dominated by a small number of large or small factors. Review whether the binning and prediction definition are suitable.
Check the simulation workflow. Each simulated replicate must use the same record-specific factors as the observations. Recomputing a different reference or factor for each replicate changes the comparison.
Check the pattern, not only isolated points. Repeated departures across bins or quantiles are more suggestive of systematic misfit than one point outside an envelope.

An uncorrected VPC is often preferable when typical predictions are nearly constant within each bin: in that case the correction factors are close to one, and correction adds little. The uncorrected plot also retains the concentrations on their original scale without a normalization step. Conversely, where typical predictions vary materially within bins, the corrected VPC can make quantile comparisons less entangled with those differences. These approaches answer related but not identical graphical questions, so the chosen scale should be stated when reporting results.

Prediction correction has important limits. It does not guarantee that all covariate or design differences have been removed, and it does not make a misspecified model adequate. It can alter distributional shape because different observations receive different multipliers. A correction based on predictions that are poorly calibrated, unstable, or inappropriate for the target comparison can obscure rather than clarify misfit. Finally, the corrected VPC remains a diagnostic summary: it does not establish that the model is true, and an in-envelope result does not establish that every aspect of the model is adequate.

Common Mistake

Pitfall: “The observed concentrations were corrected, so the simulated concentrations can stay on their original scale.” This compares corrected observed quantiles with uncorrected simulated quantiles, which are not comparable. Apply $M_b/m_i$ to the corresponding simulated record as well, then calculate the simulated quantiles and envelope from those corrected values.

Establish the prediction scale.
For every record, obtain the positive typical prediction $m_i$ at its sampling time and design.

Set a bin reference.
Calculate $M_b$, the median of the predictions in bin $b$, and compute $M_b/m_i$ for each record.

Correct both data sources.
Multiply each observed and corresponding simulated concentration by that same record-specific factor.

Compare like with like.
Calculate observed corrected quantiles and replicate corrected quantiles, then compare the observations with the simulation-based envelopes.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

In one bin, $M_b=4$ mg/L, $m_i=2$ mg/L, and $C_i=3$ mg/L. What is the corrected concentration?
For the same record in a simulated replicate, $C^*_{ir}=2.5$ mg/L. What is its corrected simulated concentration?
If every prediction in a bin equals the bin median prediction, what happens to the corrected concentrations?
An observed corrected median is within its simulation envelope in one bin. Does this prove that the model is correct? Explain.

Solutions. (1) $C_{i,\mathrm{pc}}=3(4/2)=6$ mg/L. (2) Apply the same factor: $C^*_{ir,\mathrm{pc}}=2.5(4/2)=5$ mg/L. (3) Each factor is $M_b/m_i=1$, so corrected values equal the original concentrations. (4) No. It indicates that this particular summary is compatible with the simulated envelope in that bin, but a VPC is a diagnostic and cannot establish that the full model is correct.