Why Typical Predictions Matter in a VPC
A visual predictive check (VPC) compares summaries of observed data with the corresponding summaries from replicated data generated under a fitted model. The comparison is usually made within time bins, and may be stratified by a design feature such as dose. A bin can nevertheless contain observations with quite different model-predicted levels. When it does, a pooled median or quantile can reflect the mixture of those levels as well as the model’s ability to describe variability around them.
This issue is especially relevant when different doses, regimens, covariate values, or sampling designs occur in the same bin. A model can describe relative deviations from each observation’s typical prediction consistently while the absolute values vary widely across design groups. Conversely, pooling can conceal differences in those relative deviations. Prediction correction is intended to make such observations more comparable when the model and diagnostic question support that comparison; it is not a general repair for a poor VPC or a misspecified model.
In an NLME model, the predicted response can depend on time, dose, covariates, population parameters, and individual random effects. Setting the random effects to zero defines a reference individual; changing the design or covariates can still change that individual’s prediction. Thus, “typical” does not mean “the same for every record.” A VPC’s simulations should reflect the design features used to produce the observed records, but the typical predictions within a bin can remain unequal.
How Unequal Predictions Affect Pooled Summaries
Suppose two groups have different typical predictions but similar relative variability. The first group may produce responses near 10, while the second produces responses near 100. Pooling them in one bin creates a distribution containing both scales. Its quantiles describe that mixture, not a single group’s spread around its own typical level. The same issue can arise continuously, for example when body size or time since dose changes the prediction within a bin.
This does not automatically make the ordinary VPC invalid. If the replicated data preserve the relevant design and the model is adequate, the simulated summaries also reflect the mixture. The difficulty is interpretive: a wide pooled prediction interval may largely reflect different expected levels rather than large residual or individual variability at a given level. And if observed and simulated design compositions differ, even a comparison of pooled summaries may be misleading.
A useful way to see the scale issue is to consider the ratio \(Y_{ij}/m_{ij}\), when \(m_{ij}\) is positive. This ratio expresses the observation relative to its typical prediction. If the model implies a comparable distribution of relative deviations across design groups, then comparing relative deviations can be more informative than comparing raw values. Prediction correction places those relative deviations on a shared reference scale. The reference is a positive prediction value; its choice affects the corrected units and is a separate practical decision.
Worked Example: A Pooled Bin Combines Two Prediction Levels
Consider a hypothetical bin with three observations from each of two design groups. The typical predictions are 10 units in Group A and 100 units in Group B. Suppose, for illustration, that each group has the same three relative response values: 0.8, 1.0, and 1.2 times its typical prediction.
For Group A, the responses are \(0.8(10)=8\), \(1.0(10)=10\), and \(1.2(10)=12\) units. For Group B, they are \(0.8(100)=80\), \(1.0(100)=100\), and \(1.2(100)=120\) units. The pooled sorted values are 8, 10, 12, 80, 100, and 120.
The pooled median is the average of the third and fourth values: \((12+80)/2=46\) units. Neither group has a typical prediction near 46 units, and each group’s responses are close to its own prediction in the same relative pattern. The pooled median therefore provides little description of either group’s typical level. This example illustrates a mixture problem, not a model failure: a VPC simulation that contains the same two groups would also generate a mixture.
If the diagnostic goal is to compare relative patterns, rescaling each response by its own typical prediction would put both groups on the same relative scale. If the goal is to assess absolute concentrations or another absolute response scale, the raw values and their design context remain important. The appropriate display depends on the question.
What Prediction Correction Assumes
Prediction correction is most useful when unequal typical predictions are a nuisance for the comparison of interest, rather than a feature that should remain visible. The basic rationale is multiplicative: differences in typical level are treated as scale differences, while the relative deviations around that level are regarded as meaningfully comparable. For positive predictions, a generic correction to a common positive scale \(c\) has the form
Here \(Y_{ij}^{*}\) denotes the corrected response and \(c\) is a common reference prediction. The expression makes the principle explicit: retain the response-to-typical-prediction ratio, but express it at one shared scale. It does not imply that all individuals have identical predictions or that their random effects disappear. Nor does it determine the reference value to use.
The correction is most defensible when the response is positive, the typical predictions are well-defined and not close to zero, and the model’s variability is plausibly proportional to its predicted level over the range being combined. The relative response patterns should also be comparable across the groups being pooled. If relative variability, residual behavior, or the underlying relationship differs systematically by group, forcing a common scale can obscure the difference rather than clarify it.
Error structure matters. With proportional error, variability often changes with prediction level, so a relative-scale comparison can be natural. With additive error, the error scale may be roughly constant in the original units; dividing by predictions then gives smaller relative deviations at higher predictions. Combined error includes both components, so the suitability of a purely proportional rescaling depends on which component dominates over the range of interest. The earlier material on VPCs under proportional and combined error is relevant to this assessment.
Worked Example: The Same Relative Pattern on a Common Scale
Suppose two hypothetical observations have typical predictions of 20 and 80 concentration units. Their observed values are 22 and 72 units, respectively. Their relative responses are \(22/20=1.10\) and \(72/80=0.90\). Thus the first is 10% above its typical prediction and the second is 10% below.
For illustration, place both ratios on a common scale of \(c=50\) units. The corrected values are \((22/20)(50)=55\) units and \((72/80)(50)=45\) units. The values now show the relative deviations around the shared 50-unit scale rather than the original prediction levels of 20 and 80 units.
This calculation is informative only if expressing the deviations proportionally is appropriate. It does not show that the groups have identical absolute errors, nor does it establish that their relative distributions are actually the same. Those claims require examination of the model, design, and predictive-check results.
When Correction Can Mislead
A numerical counterexample shows why the error model matters. Suppose responses have additive error with standard deviation 5 units at both typical predictions of 10 and 100 units. On the original scale, the error is equally variable in the two groups. On a scale corrected to \(c=50\), the transformed error standard deviation is \(50(5/10)=25\) units for the lower-prediction group and \(50(5/100)=2.5\) units for the higher-prediction group. The correction has made the groups’ spreads very different, even though their original additive error standard deviations were equal. It would be misleading to treat their corrected distributions as exchangeable merely because they now share a reference prediction.
Other warning signs include typical predictions near zero, negative predictions for an outcome that cannot be negative, strong group-specific trends in relative residuals, or a broad bin spanning a region where the model’s shape changes substantially. In such cases the ratio may be unstable, scientifically awkward, or inadequate to summarize the structure. Prediction correction also cannot fix a wrong structural model, an incorrect residual-error model, poor parameter estimates, or a mismatch between the simulated and observed design.
Before deciding to correct, inspect how typical predictions vary within each proposed bin and across design groups. Ask whether that variation is large relative to the response scale and whether it is associated with a feature that could change the distribution of relative deviations. Then compare the intended interpretation of a raw-scale VPC with that of a relative-scale display.
Decide whether the question concerns absolute response distributions or the pattern of deviations relative to each record’s typical prediction.
Establish that typical predictions are positive and away from zero, and assess whether proportional scaling is consistent with the model’s error structure over that range.
Consider whether relative variability and trends can reasonably be pooled. If not, retain group distinctions rather than rescaling them into apparent comparability.
Use prediction correction when the relative-scale comparison is justified; otherwise consider stratified VPCs or raw-scale summaries that preserve the different prediction levels.
Alternatives, Diagnostics, and Limitations
Stratification is often the clearest alternative when a design feature explains the unequal predictions and each group has enough observations for stable summaries. Separate VPCs by dose, regimen, or a relevant covariate retain the absolute scale and reveal group-specific misfit. Its limitation is reduced precision and visual complexity when there are many groups or sparse data. A raw-scale pooled VPC retains the original units and can be suitable when the design mixture itself is the target, but its marginal quantiles may be difficult to interpret as evidence about within-group variability.
Prediction correction is a compact way to compare relative behavior across unequal levels, but the result depends on the model-based typical predictions and on the assumption that the relative scale is meaningful. A corrected VPC should therefore be read alongside knowledge of the model and, where useful, an uncorrected or stratified view. Agreement between displays can strengthen an interpretation; disagreement is a reason to investigate the scale and grouping rather than to select whichever plot looks more favorable.
A common mistake is to reason, “The predictions differ, so correction is always required.” The correction is not automatic. Unequal predictions can be an intentional feature of the design, and simulations may already reproduce their effect in the ordinary VPC. The better question is whether those level differences obscure the predictive feature being assessed and whether the model supports a shared relative scale. If either answer is no, retain the original scale or stratify.
A second mistake is to interpret a corrected plot as evidence that absolute prediction is accurate. Correction removes record-specific prediction level from the displayed scale by construction. It can help diagnose relative variability or trends, but it cannot by itself demonstrate accurate absolute concentrations, exposure, or response. Both the scale and the diagnostic target should be stated when presenting the result.
Check Your Understanding
Answer each question using the diagnostic target and assumptions, not just the size of the predictions.
- Two groups have typical predictions of 5 and 50 units, but the scientific question concerns absolute response quantiles. Is correction necessarily preferable to a raw-scale VPC? Explain.
- A model has additive error with standard deviation 4 units at typical predictions of 20 and 80 units. What are the error standard deviations after correction to a common scale of 40 units?
- What does a typical prediction represent when the individual random effects are set to zero? Why need it not equal the population mean response?
- A pooled raw-scale VPC is wide, but each dose group has a narrow distribution around its own typical prediction. Give two possible displays or analyses that could clarify the source of the width.
Solutions. (1) No. If absolute response quantiles are the target, retaining the original units is important; stratification may help explain the groups. (2) The transformed standard deviations are \(40(4/20)=8\) units and \(40(4/80)=2\) units, so they are not equal on the corrected scale. (3) It is the model prediction for the specified design and covariates at the typical random-effect value. In a nonlinear model, the prediction at typical random effects need not equal the average prediction over the random-effects distribution. (4) Plot dose-stratified raw-scale VPCs to preserve absolute levels, and consider a corrected VPC if relative deviations are the target and proportional scaling is justified.