What a pcVPC Pattern Can Reveal
A prediction-corrected visual predictive check (pcVPC) compares observed outcomes with simulated outcomes after rescaling them to a common prediction reference. The goal is to make observations collected under different typical predictions more comparable, for example when dose, covariates, or sampling times imply different expected outcome levels. The correction is a rescaling, not a general repair for model misspecification: a time trend in the corrected observations can still reveal residual temporal bias, and their spread can still reveal a mismatch in predictive dispersion.
Interpretation starts with the quantities being compared. Within each time bin, the observed pcVPC typically displays selected quantiles, such as the median and lower and upper quantiles. Simulations generate corresponding quantiles under the fitted model. The observed quantiles are then compared with simulation-based prediction intervals for those quantiles. A prediction interval here describes the range of quantile values expected across replicated data sets under the model and the specified design. It is not a confidence interval for a model parameter.
This multiplicative correction is most directly interpretable when variation is approximately proportional to the prediction: outcomes that are twice as large are expected to have roughly twice the absolute variation. The prediction and reference must be positive, defined consistently, and appropriate for the outcome scale. The simulated data must follow the model and observation design being checked, and the correction must be applied comparably to observed and simulated outcomes. If the outcome has important additive error, a multiplicative rescaling can change its spread in a way that is not justified by the model.
Reading Temporal Bias and Dispersion
A temporal bias is a systematic difference between observed and simulated behavior as time changes. For example, if observed corrected medians repeatedly lie above the simulation intervals early and below them late, the model may predict a time course that is too flat, too steep, or otherwise wrongly shaped. A single point outside an interval is weaker evidence of a systematic pattern than a coherent run across adjacent bins. Sampling variation can produce isolated departures even when the model is adequate.
Dispersion concerns the spread of outcomes around the central trend. When lower and upper observed quantiles are both farther from the median than the corresponding simulated quantiles, observed variability may be greater than predicted. When both are closer, predicted variability may be too large. An asymmetric departure can point to a mismatch in one tail or in the distribution’s shape, rather than a simple overall scale difference. These are diagnostic interpretations, not unique explanations: a pcVPC pattern does not identify which model component caused the discrepancy.
Read the center and spread together. A model can match the median but miss both outer quantiles, indicating plausible central predictions but inadequate dispersion. Conversely, outer quantiles can appear reasonable while the median has a time-dependent offset. Check whether departures persist across bins, whether the relevant bins contain enough observations, and whether the simulation intervals themselves are broad. Bins with few observations can have unstable empirical quantiles; broad simulation intervals can make modest discrepancies hard to distinguish visually.
Worked Example: A Median Trend That Drifts Over Time
Suppose a hypothetical pcVPC has three time bins. The table gives the observed corrected median and the 95% simulation-based prediction interval for that median in each bin.
| Time bin (h) | Observed corrected median | 95% simulation interval |
|---|---|---|
| 0–2 | 12.0 | 9.0–11.0 |
| 2–6 | 10.0 | 9.0–11.0 |
| 6–12 | 7.0 | 8.0–10.0 |
In the first bin, \(12.0>11.0\), so the observed median is 1.0 outcome unit above the interval. In the final bin, \(7.0<8.0\), so it is 1.0 unit below. The middle-bin median is inside its interval. The direction changes from above to below across time, which is more suggestive of a temporal shape discrepancy than a constant overall offset. It could be consistent with a model trend that declines too slowly, but the plot alone does not establish that explanation.
The conclusion is conditional on the data and simulations being comparable: the bins must use the intended observation schedule and the same correction convention, and the displayed intervals must reflect the simulated quantiles. If the first or last bin is very sparse, its observed median may be unstable. Thus the pattern is a diagnostic signal to investigate, not a formal test that proves a particular mechanism is wrong.
Checking the Spread on the Corrected Scale
For a central quantile such as the median and outer quantiles such as the 10th and 90th, compare each observed quantile with its own simulation-based interval. The interval for the 10th percentile is not interchangeable with the interval for the 90th percentile. A useful first check is whether the observed lower and upper quantiles depart in opposite directions from the median, and whether the distance between them is inconsistent with simulated distances.
For example, if the median is well aligned but the observed 10th percentile is below its interval and the observed 90th percentile is above its interval, the observed central 80% spread is likely wider than the model predicts. If both outer quantiles are inside their respective intervals, that is not proof of a perfect distributional fit, but it provides no visible evidence of that particular spread mismatch. Tail behavior, asymmetry, or departures between displayed quantiles may still matter.
Worked Example: Wider Observed Dispersion
In a hypothetical time bin, the observed corrected quantiles are 6, 10, and 14 outcome units at the 10th, 50th, and 90th percentiles. The corresponding simulation-based intervals are 7–9, 9–11, and 11–13.
The observed median is \(10\), which lies in \(9\)–\(11\). The observed 10th percentile is \(6<7\), below its interval, and the observed 90th percentile is \(14>13\), above its interval. The observed central 80% width is \(14-6=8\) units. For comparison, the width between the simulated central quantile values is not determined by those intervals alone; the intervals describe uncertainty in each quantile, not a single simulated pair of endpoints.
The two outer-quantile departures point in opposite directions while the median is aligned, which is consistent with greater observed dispersion than the model predicts in this bin. It does not justify calculating a precise difference in population spread from the interval endpoints. The simulation intervals are not the quantile values themselves, and their endpoints should not be subtracted to estimate the model’s spread.
Why the Correction Can Help—and When It Can Mislead
The correction’s effect can be seen by considering a simple proportional relationship. If an outcome has the form \(Y_i=P_i R_i\), where \(R_i\) is a dimensionless deviation relative to its prediction, then $$ Y_{\mathrm{pc},i} =Y_i\frac{P_{\mathrm{ref},b}}{P_i} =P_iR_i\frac{P_{\mathrm{ref},b}}{P_i} =P_{\mathrm{ref},b}R_i. $$ Under this idealized relationship, the corrected outcome retains the relative deviation \(R_i\) while expressing it at the common reference scale. This explains why correction can reduce differences caused solely by unequal prediction levels. It also shows the limitation: if deviations are additive rather than proportional, the rescaling changes their size differently at different predictions.
A practical check is to compare the uncorrected and corrected displays and examine the predictions used in the correction. The ordinary VPC shows predictive behavior on the original outcome scale; the pcVPC asks whether the pattern remains after rescaling for unequal predictions. If the correction removes a difference that is plausibly due to different typical predictions, that may improve comparability. If it creates a new spread pattern, hides a meaningful absolute-scale difference, or depends heavily on extreme prediction ratios, treat the corrected display cautiously.
Worked Example: Additive Error Can Be Distorted by Correction
Consider two hypothetical groups of observations in one bin. Their typical predictions are \(P_A=10\) and \(P_B=20\), and the common reference prediction is \(P_{\mathrm{ref}}=15\). Suppose the observed values are prediction plus or minus 2 outcome units, representing the same additive deviation in both groups.
For group A, the correction multiplier is \(15/10=1.5\). The values 8 and 12 become \(8(1.5)=12\) and \(12(1.5)=18\), centered at 15 with deviations of \(-3\) and \(+3\). For group B, the multiplier is \(15/20=0.75\). The values 18 and 22 become \(18(0.75)=13.5\) and \(22(0.75)=16.5\), centered at 15 with deviations of \(-1.5\) and \(+1.5\).
The uncorrected additive deviations had the same magnitude, 2 units, in both groups. After correction, their magnitudes differ: 3 units in group A and 1.5 units in group B. Pooling these corrected outcomes can therefore create unequal apparent dispersion even though the original additive error was constant. The correction has standardized the prediction level, but it has not preserved additive error scale. If additive error is important, assess whether the pcVPC’s multiplicative correction is suitable and compare with the original-scale VPC or an appropriately stratified check.
Diagnostics, Alternatives, and Limitations
When a pattern looks concerning, first check whether it is robust to reasonable choices in the display. Sparse or uneven sampling can make bin quantiles noisy; changing bin boundaries can shift which observations contribute to each summary. A pattern that appears only in a bin with very few observations is less persuasive than a consistent departure across well-populated bins. Also check that observed and simulated records use the same time definition, analysis population, and handling of censored or otherwise specially recorded outcomes.
Next inspect the correction itself. Confirm that predictions are positive and that the reference is defined as intended for each bin. Large or highly variable ratios \(P_{\mathrm{ref},b}/P_i\) can amplify or compress individual outcomes substantially. If the observed and simulated data are corrected differently, or if a prediction used in the correction is inappropriate for the target comparison, the resulting pattern may reflect the procedure rather than predictive performance. A useful sensitivity check is to compare the pcVPC with the ordinary VPC and, when design or covariate differences are central, with a stratified VPC.
These displays answer related but distinct questions. An ordinary VPC retains the original outcome scale and can show whether absolute predictions are adequate under the observed design, but unequal typical predictions may make pooled quantiles hard to interpret. A pcVPC rescales to a reference to improve comparison across such differences, but its interpretation depends on the correction scale. A stratified VPC retains separate displays for defined subgroups and can reveal subgroup-specific misfit, though it may divide already limited data. No single display removes the need to assess whether its target and assumptions match the scientific question.
- Do not treat one isolated point outside a band as decisive. Look for coherent departures and consider bin size and simulation variability.
- Do not interpret quantile intervals as parameter confidence intervals. They describe simulated variation in data summaries under the specified model and design.
- Do not assume correction removes all design effects. It rescales by the chosen prediction; it does not automatically correct misspecified covariate effects, time trends, or error structure.
- Do not infer the cause from the shape alone. Similar visual discrepancies can arise from different model components or from unstable summaries.
A pcVPC is most informative when its correction is appropriate for the variability scale, its reference and design are clearly defined, and the pattern is read across the median and spread rather than from one summary alone. It remains a visual predictive diagnostic: it can flag where predictions fail to reproduce observed behavior, but it does not by itself isolate the source of failure or provide a formal measure of independence among repeated observations.
Check Your Understanding
Answer each question, then compare with the brief solutions.
- If the observed median lies above its simulation interval in early bins and below it in later bins, what kind of discrepancy does this suggest, and what should be checked before drawing a conclusion?
- Observed lower and upper quantiles both lie outside their simulation intervals, below and above respectively, while the median is inside. What is the cautious interpretation?
- An observation is 24 units, its prediction is 12 units, and its reference prediction is 9 units. Calculate its prediction-corrected value.
- Why might multiplicative prediction correction distort dispersion when the underlying error is additive?
- What complementary display can help assess whether a pcVPC pattern was introduced or obscured by correction?
Solutions. (1) It suggests a time-dependent bias or shape mismatch, such as a trend that is too shallow or too steep; check bin sizes, simulation intervals, observation design, and the correction before interpreting the pattern. (2) The central quantile alignment alongside outward departures is consistent with greater observed dispersion, but the intervals do not directly quantify a spread difference and the pattern is not proof of its cause. (3) \(24(9/12)=18\) units. (4) The correction multiplies additive deviations by \(P_{\mathrm{ref}}/P_i\), so equal-sized deviations at different predictions become different-sized corrected deviations. (5) Compare with the ordinary VPC on the original outcome scale; a stratified VPC may also help when subgroup differences are relevant.