What Median and Quantile Trends Show
A visual predictive check (VPC) compares summaries of observed data with the same summaries calculated from replicate datasets simulated under a fitted model. Once the time bins and simulation-based reference intervals have been constructed, the next task is to read the pattern: where does the observed distribution depart from the simulated reference, and which part of the distribution departs? A median trend is most informative about shifts in typical values. Comparing lower and upper quantiles as well as the median helps distinguish a shift from a change in spread or tail behavior.
This interpretation depends on a fair comparison. Observed and simulated data must use the same time bins, quantile definitions, and relevant observation design. Simulations should reflect the model and the design being checked, including the sources of variability represented in that model. The plotted intervals are reference ranges for summaries across simulated replicates; they are not intervals containing a specified proportion of individual concentrations.
For example, a 90% simulation-based interval for a bin's median might be formed from the 5th and 95th percentiles of the simulated medians. If the observed median lies above that interval, the observed typical value in that bin is high relative to the simulated reference. This is not the same as saying that 90% of individual observations should fall inside the interval: the interval describes uncertainty or variability in a quantile summary across simulated datasets.
Read the Median First, Then the Distribution Around It
A median departure is a location signal. If the observed median is above the simulated interval in one time bin, the data in that bin are centered higher than the model commonly predicts under the simulation setup. A run of such departures over adjacent bins can suggest a time-localized bias in the typical profile. A run below the intervals suggests the opposite direction. A single departure is weaker evidence: simulation-based intervals are usually pointwise, so they do not guarantee that every observed point across all bins will fall inside them.
To assess dispersion, compare lower and upper quantiles with the median. A useful descriptive measure is the interquantile range at a selected pair of probabilities. For example, the gap between the 90th and 10th percentiles is
If the observed median is near its simulated reference but the observed lower and upper quantiles are both farther from it than expected, the observed distribution may be more dispersed. If both are closer to the median, dispersion may be too small. If only one tail departs, the discrepancy is not a simple symmetric spread change: it points instead to tail-specific behavior, skewness, or another distributional mismatch. These are diagnostic descriptions, not unique explanations of the underlying model component.
Worked Example: A Localized Median Shift
Suppose a hypothetical VPC has three adjacent time bins. The table gives the observed median concentration and the 90% simulation-based interval for the median in each bin. All concentrations are in mg/L.
| Bin | Observed median | 90% interval for simulated medians |
|---|---|---|
| 1 | 7.8 | 7.2 to 8.3 |
| 2 | 8.6 | 7.4 to 8.2 |
| 3 | 7.1 | 6.6 to 7.8 |
In bin 2, the observed median exceeds the upper endpoint by \(8.6-8.2=0.4\) mg/L. The medians in bins 1 and 3 lie inside their respective intervals. The pattern therefore identifies a high observed median in bin 2, not a persistent upward shift over all three bins. It is reasonable to describe this as a possible localized location mismatch. It is not enough to conclude that the model is wrong: the size and pattern of departures, bin precision, and pointwise nature of the intervals also matter.
Distinguishing Spread from Tail Mismatch
A distribution has more than a center. A median-only comparison can miss important disagreement if the simulated and observed centers agree but their tails do not. Compare several quantile trends, always matching each observed quantile to the simulation interval for that same quantile. A high observed 90th percentile is not interpreted by comparing it with the interval for the simulated median.
The following diagnostic patterns are useful when read across bins rather than as isolated marks:
- Median and outer quantiles shift together: a location shift is plausible, especially if the distances between quantiles remain similar.
- Median agrees, but both tails are too far from it: the observed distribution may be more dispersed than simulated.
- Median agrees, but both tails are too close to it: the simulated distribution may be too dispersed.
- One tail departs while the other and median agree: suspect tail-specific mismatch or a change in asymmetry rather than a simple shift or uniform widening.
- Departures change direction over time: a time-varying pattern may be present; a single overall summary could conceal it.
Worked Example: Wider Observed Distribution with a Similar Median
In a hypothetical time bin, the observed 10th, 50th, and 90th percentiles are 4, 10, and 22 mg/L. The corresponding central simulated quantile trends are 5, 10, and 19 mg/L. The simulation-based 90% reference intervals are 4.7 to 5.3 mg/L for the 10th percentile, 9 to 11 mg/L for the median, and 18 to 20 mg/L for the 90th percentile.
The observed median, 10 mg/L, is inside its interval. The observed 10th percentile is below its interval by \(4.7-4=0.7\) mg/L, while the observed 90th percentile is above its interval by \(22-20=2\) mg/L. The observed 10th-to-90th percentile width is \(22-4=18\) mg/L. The width between the central simulated quantile trends is \(19-5=14\) mg/L. Thus, the observed central location agrees with the model reference, but the observed distribution is wider by \(18-14=4\) mg/L on these summaries. Both tails depart outwards, supporting a dispersion concern rather than a median shift. The unequal tail departures also caution against assuming that the discrepancy is perfectly symmetric.
Worked Example: An Upper-Tail Departure
Consider another hypothetical bin. The observed 10th, 50th, and 90th percentiles are 3.2, 8.0, and 18 mg/L. Their simulation-based 90% intervals are 2.8 to 3.6 mg/L, 7.5 to 8.5 mg/L, and 15 to 17 mg/L, respectively.
The observed 10th percentile and median are inside their intervals. The observed 90th percentile exceeds its upper endpoint by \(18-17=1\) mg/L. The observed 10th-to-90th percentile width is \(18-3.2=14.8\) mg/L, while the central simulated quantile trends have a width of \(16-3.0=13\) mg/L. This pattern is not a general shift, because the median agrees; nor is it a clear symmetric widening, because the lower quantile agrees with its reference. The evidence is specifically for an upper-tail departure in this bin. Possible explanations require further model and data investigation; the VPC pattern alone does not identify a unique cause.
Why the Reference Intervals Are Pointwise
A short justification explains why an observed quantile can occasionally fall outside an interval even when the simulation model is adequate. Suppose, for a fixed bin and quantile, the observed summary and simulated summaries are exchangeable under the model: they have the same distribution under the predictive-check setup. If the central 90% reference interval is calibrated to that distribution, then the observed summary has approximately 90% probability of lying within it. The complementary probability is approximately 10% for that one comparison.
This is a statement about repeated datasets under the specified simulation setup, not a probability that the model is true after seeing the plot. With many bins and quantiles, several pointwise comparisons are made. Even if each interval is well calibrated, the probability that all observed summaries lie inside all intervals is generally less than 90%. Adjacent bins may also be related because measurements from the same individuals contribute over time. Therefore, do not interpret one interval crossing as a formal global test, and do not count neighboring crossings as independent evidence.
In practice, exact exchangeability may not hold. A common predictive check simulates from estimated parameter values, so the intervals are conditional on those fitted values and may not represent all uncertainty from estimating parameters. The observed data also informed the fit. These features mean a VPC is best read as a model diagnostic under its stated simulation procedure, not as a stand-alone calibrated hypothesis test.
Check the Comparison Before Explaining It
Before assigning a physiological or statistical explanation to a trend, check that the comparison itself is trustworthy. Observed and simulated records should use the same bin boundaries and quantile calculation. Simulation should reproduce the relevant design features, such as the timing and number of observations represented in each bin. If bins contain few observations, observed quantiles can be unstable; a sharp-looking departure may reflect sparse information rather than a robust pattern. Likewise, too few simulated replicates can make the reference limits noisy.
Binning is a compromise: broad bins increase the number of observations used for each summary but can obscure a brief departure; narrow bins give finer time resolution but can make quantiles imprecise. The earlier material on binning time and sparse sampling describes those choices. Here, the practical diagnostic is to ask whether the apparent trend is stable enough to interpret given the data density and simulation variability.
A VPC complements, rather than replaces, other checks. Residual diagnostics can reveal observation-level patterns that are not summarized by a few quantiles; VPC trends show whether the model reproduces the distribution of data summaries under the design. If the median and quantile trends suggest a time-localized mismatch, use the model structure, sampling design, and other diagnostics to investigate it. Do not infer a unique mechanism from the VPC alone.
Check matching bins, quantile definitions, observation design, and simulation setup.
Identify which bins and which quantiles fall outside their corresponding simulation intervals, and note the direction.
Use the median to assess location, both tails to assess spread, and one-tail departures to flag tail-specific mismatch.
Consider bin precision, number of simulations, pointwise coverage, and whether other diagnostics support the interpretation.
Common Mistakes
- Calling the interval an individual prediction range. The interval is for a simulated quantile summary, not for individual observations.
- Calling every excursion model failure. A pointwise interval can be exceeded by chance, especially when many bins and quantiles are inspected.
- Reading a high upper quantile as a median shift. Compare the median itself; a departure in only the upper tail has a different interpretation.
- Using the interval endpoints to calculate typical spread. The endpoints describe uncertainty in a simulated quantile. Compare observed quantile distances with the simulated quantile trends or replicate summaries, not with the interval endpoints as if they were quantiles of individuals.
- Claiming a unique cause. Wider observed quantiles flag dispersion mismatch, but do not by themselves identify whether it arises from structural, random-effects, residual, or design features.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- In one bin, an observed median is 12 mg/L and its simulation-based interval is 10 to 11.5 mg/L. What is the direction and size of the departure from the nearest interval endpoint?
- The observed median is within its interval, but both the observed lower and upper quantiles are farther from it than their simulated central trends. What pattern does this suggest?
- Why does one observed quantile outside its pointwise 90% interval not establish that the model is inadequate?
- The median and lower quantile agree with their intervals, but the observed upper quantile is above its interval. What should you conclude, and what should you avoid concluding?
Solutions. (1) The observed median is high; its distance above the upper endpoint is \(12-11.5=0.5\) mg/L. (2) The observed distribution may be more dispersed than the simulated distribution, because both tails extend farther from a similar center. (3) A pointwise 90% interval has approximately 10% probability of being exceeded for that individual comparison under its calibration assumptions; multiple bins and quantiles create multiple opportunities for an excursion, and the intervals are not a simultaneous global test. (4) The pattern supports an upper-tail departure in that bin. It does not establish a general median shift or identify a unique cause of the tail mismatch.