Why VPC Resolution Depends on Bin Size
A visual predictive check (VPC) compares observed data summaries with corresponding summaries from data simulated under a fitted population pharmacokinetic model. In a time-bin VPC, the observed and simulated quantiles are calculated separately within each bin. The key question here is not how to construct the plot, but whether the bin contains enough information for its quantiles to be interpretable.
A curve can look smooth because its plotted points are connected, because a display has been smoothed, or because summaries from nearby times have been pooled. None of these operations creates additional independent observations in a sparse bin. In particular, a median supported by a handful of observations may be quite sensitive to which individuals happened to be sampled, while a lower-tail quantile may be determined almost entirely by one observation.
For each bin, record at least the number of observed concentration records, the number of distinct individuals contributing those records, and the number of usable observations for each plotted summary. A bin with 40 records from 40 individuals and a bin with 40 records from 8 individuals with repeated measurements do not provide the same amount of independent information. Within-individual measurements can be correlated, so treating every record as independent may overstate how much information the bin contains.
Rank Resolution and Tail Counts
Let \(n\) be the number of usable observations in a bin, and let \(q\) be the quantile of interest, such as \(q=0.10\) for the 10th percentile. The empirical quantile is determined by ordered observations. Its exact value depends on the quantile convention, but the relevant rank is near \(nq\) from the lower end of the ordered data. When \(nq\) is close to zero or one, the lower-tail quantile is governed by the minimum or one of the next-smallest observations, rather than by a well-populated tail.
A simple diagnostic comes from counting observations below the population \(q\)-quantile. If observations are independent draws from a continuous distribution, each has probability \(q\) of falling below that quantile. The count \(K\) below it therefore has a binomial distribution. Its expected value and the probability of seeing none are:
This is a resolution heuristic, not a guarantee that a sample quantile has a particular precision. A modest expected tail count signals that the quantile depends on very few observations. The calculation assumes independent observations from a continuous distribution; repeated records within an individual, ties, censoring, or substantial variation in observation times within a bin can make the simple interpretation inadequate.
Worked Example: A Lower-Tail Quantile in a Small Bin
Suppose a hypothetical VPC has 15 usable observed concentrations in a time bin, and the plotted lower quantile is the 5th percentile. Use the rank heuristic and the binomial calculation to assess its resolution.
The expected number of observations below the true 5th percentile is \(nq=15(0.05)=0.75\). Thus, the lower tail is not expected to contain even one observation on average. Under the independent, continuous-data approximation, the probability of no observation below that quantile is \((1-q)^n=(0.95)^{15}=0.4633\), or about 46.3%.
The empirical 5th percentile will consequently be close to the smallest observed concentrations, with its exact value depending on the quantile convention. A plotted point can still be computed, but it should not be read as a stable estimate of the bin's 5th percentile. The calculation checks the sample-size assumption: if several of the 15 records came from the same few individuals, the independent-observation heuristic would be even less reassuring.
The same reasoning applies to upper tails by replacing \(q\) with \(1-q\) when counting observations above the target quantile. For a median, the rank is near the center of the ordered observations, so it is generally less exposed to a single extreme value than a 5th or 95th percentile. Nevertheless, a small bin can still produce a variable median: only a few central ranks determine it, and the observed concentration distribution may be broad or skewed.
Worked Example: Comparing Bin Sizes for a 10th Percentile
Consider two hypothetical bins with 20 and 100 independent usable observations, respectively. Compare their expected counts below the 10th percentile and the probability that no observation falls below that population quantile.
For the bin with \(n=20\), the expected count is \(20(0.10)=2\). The probability of no observation below the 10th percentile is \((0.90)^{20}=0.1216\), or 12.2%.
For the bin with \(n=100\), the expected count is \(100(0.10)=10\), and the probability of no such observation is \((0.90)^{100}=0.00002656\), or about 0.00266%.
The 100-observation bin gives the lower-tail quantile more rank support under the stated assumptions. It is not automatically precise in every practical sense: observations may be clustered by individual, the bin may combine meaningfully different sampling times, and concentration values may be highly variable. The count comparison identifies a clear resolution difference, not a universal sample-size threshold.
Observed Sampling Variability and Simulation Precision
Two different sources of instability should be kept separate. First, finite-sample variability means the empirical quantile changes when a different study sample is drawn, even if the model and population are unchanged. This is the central issue when deciding whether the actual study's bin supports an informative quantile. Second, Monte Carlo precision concerns whether a simulated reference summary is stable enough given the number of simulated data sets. Increasing the number of simulation replicates can reduce Monte Carlo noise, but it does not add observations to the actual study bin or remove its sampling variability.
A practical assessment examines bin-specific quantiles across simulated data sets generated under the model with the study's design, including its sampling schedule and relevant data handling. Look at whether the simulated quantiles themselves vary substantially from replicate to replicate, especially in the tails. This describes the variability expected from finite samples under the model. It should not be confused with a formal uncertainty band for a VPC quantile or with uncertainty about model parameters.
The simulation setup matters. Replicates intended to reveal sampling variability should reproduce the number of individuals and observations contributing to each bin; otherwise their quantile variability is not comparable to the study's. Also inspect simulation Monte Carlo precision separately: a highly variable set of replicate quantiles can be estimated poorly if too few replicate data sets were generated. The earlier material on Monte Carlo precision in predictive checks addresses the precision of simulation-based summaries; the present concern is whether the study design supports informative bin-wise quantiles.
Worked Example: Smooth Median, Unstable Tails
In a hypothetical bin, 18 observed concentrations have a median of 4.8 mg/L and an empirical 10th percentile of 1.0 mg/L. The model is used to generate 200 replicate data sets with the same bin-specific sample size. Across those replicates, suppose the median quantiles range from 4.2 to 5.5 mg/L, while the 10th-percentile summaries range from 0.2 to 2.6 mg/L.
The median range is \(5.5-4.2=1.3\) mg/L; the lower-quantile range is \(2.6-0.2=2.4\) mg/L. These ranges are descriptive summaries of the replicate results, not formal confidence intervals. They indicate that, under this model and design, the lower-tail summary is much more variable across finite samples than the median.
Now suppose a plotting routine connects the median points from adjacent time bins and applies a smooth display. The resulting median curve could look regular even though each bin has a small sample and the lower-tail quantiles vary widely across replicates. The appropriate conclusion is not that the model's tails are well supported by the plot's appearance. Rather, the bin counts and replicate behavior show that tail interpretation is limited, and the display should not hide that limitation.
Diagnostics, Pooling, and Failure Modes
Before interpreting a VPC quantile, inspect the count and composition of every bin, not merely the total study size. A large study can still have sparse bins when sampling is uneven. For each bin, report the usable record count and, where relevant, the number of distinct individuals. Then compare these counts with the rank location of each target quantile and inspect replicate quantile variability under the study design.
Below-quantification observations need particular care. A recorded below-quantification result is not an ordinary numeric concentration equal to zero or to the quantification limit. The relevant number of observations available to a quantile depends on the analysis and summary definition. Follow the handling used for the VPC and make clear which data contribute; silently substituting a numeric value can distort both ranks and apparent tail support.
A common mistake is to treat a large number of simulated records as if it repaired a small observed bin. For example, generating 1,000 simulations from a study design with 12 records in a bin may stabilize the model's simulated reference calculation, but every replicate still reflects the limitations of a 12-record bin. The correction is to distinguish the number of observed records per bin, the number of simulated data sets, and the number of independent individuals. Each answers a different question.
Pooling adjacent bins can increase the number of records used in a summary, but it changes what is being summarized: concentrations from different times are treated as a combined group. If the concentration distribution changes over time, a pooled quantile may obscure that change rather than resolve the original time-specific quantile. Similarly, smoothing is a display choice, not new information. When pooling or smoothing is used, show or report the underlying bin counts and avoid presenting the result as if each sparse-bin quantile were independently well determined.
| Approach | What it can help with | What it cannot establish |
|---|---|---|
| Inspect bin counts and quantile ranks | Identifies bins where quantiles rely on very few order statistics | Does not by itself quantify every source of uncertainty |
| Simulate replicate data sets with the study design | Shows how bin-wise summaries vary under the model and finite sample size | Does not add information to the observed study |
| Pool adjacent bins | Can provide more records for a summary over a broader time region | Does not preserve a time-specific quantile if distributions vary over time |
| Smooth plotted summaries | Can make broad trends easier to see | Does not improve the underlying quantile resolution |
There is no single count that guarantees an informative quantile in every study. The needed count depends on the quantile, variability, distribution shape, within-individual dependence, and the purpose of the VPC. A bin may support a useful median comparison while offering little basis for a 5th-percentile interpretation. State that distinction rather than labeling the entire VPC simply “adequate” or “inadequate.”
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A bin contains 25 independent observations. For a 10th-percentile summary, calculate the expected number below the true quantile and the probability that none fall below it. What does this suggest?
- Why does increasing the number of model-simulated data sets not increase the resolution of the observed study's bin-specific quantile?
- A bin contains 36 records from 9 individuals, with four records per individual. Why might treating all 36 records as independent be misleading?
- A display smooths a lower-quantile curve across adjacent bins. What information should be checked before interpreting the smooth line?
Solutions. (1) The expected count is \(25(0.10)=2.5\). The probability of none is \((0.90)^{25}=0.07179\), or about 7.18%. The tail quantile has only modest rank support and can be sensitive to the sample. (2) Simulations improve Monte Carlo characterization of model-generated summaries, but the observed quantile is still determined by the original observed records and their ranks. (3) Measurements from the same individual may be correlated; the 36 records do not necessarily provide the independent information of 36 different individuals. Inspect both records and contributing individuals, and consider dependence when interpreting the bin. (4) Check underlying per-bin record and individual counts, the unsmoothed quantiles, and whether the bins represent comparable times. Smoothing can clarify broad patterns but cannot establish that sparse-bin tail quantiles are stable.