Visual Predictive Checks With Sparse Sampling

Why Sparse Sampling Changes the VPC

A visual predictive check (VPC) compares summaries of observed concentrations with the corresponding summaries from datasets simulated under a fitted population pharmacokinetic (PK) model. In a typical VPC, observations are grouped into time bins, and selected concentration quantiles—often the 5th, 50th, and 95th percentiles—are calculated in each bin. Simulated datasets provide a reference for judging whether the observed summaries are compatible with model predictions under the same design.

When participants contribute few concentrations, a bin may contain only a modest number of observations, or its observations may come from very few participants. Either condition can make a displayed quantile unstable. A few points may determine the whole summary, and a curve connecting summaries across bins can appear more temporally detailed than the data justify. The central task is therefore not simply to make a VPC plot, but to assess what each bin can support.

A VPC should reproduce the relevant design features in its simulations, including the observation times and the number of observations per participant. If the observed data are sparse but the simulated data are sampled densely, simulated quantiles may look smooth and stable for a reason that does not apply to the observed summaries. Matching the design makes the comparison meaningful; it does not make a poorly occupied observed bin precise.

Occupancy: Count Observations and Participants

The number of concentrations in a bin is the immediate limit on its empirical quantiles. But concentration records from the same participant are not generally independent: they share that participant’s latent characteristics and may be correlated through the PK trajectory. Thus, ten concentrations from ten participants and ten concentrations from two participants should not be treated as equivalent evidence about population behavior. The latter bin has ten records but only two independently sampled participants contributing them.

There is no universal minimum count that makes every VPC bin reliable. The adequacy of occupancy depends on the quantile being estimated, the variability of concentrations near that quantile, the degree of within-participant dependence, and the purpose of the check. Tail quantiles require particular caution because they are determined by observations near the extremes. A bin with few records may have no meaningful resolution for a 5th or 95th percentile.

Adaptive Binning for Visual Predictive Checks addresses how bin boundaries can be selected to improve occupancy. In sparse designs, however, changing boundaries cannot create information that was not collected. A wider bin may increase its record count while combining concentrations from meaningfully different times. The trade-off is between more stable summaries and less temporal resolution; the binning choice should be evaluated in light of the sampling schedule and the PK changes expected over that interval.

Why Quantiles Become Uncertain

A sample quantile is an order statistic: it is determined by the ranked observed values. With a small bin, changing or removing one observation can change the selected rank substantially. This is especially important for a tail quantile. A sample median may be reasonably central in rank yet still vary considerably if the concentration distribution is broad or the records are strongly clustered by participant.

A short justification shows why even a well-defined quantile can be imprecise. Let \(q_p\) be the population \(p\) quantile, and let \(F(q)\) denote the population cumulative distribution function. For independent observations, the empirical proportion at or below \(q_p\) has approximately binomial sampling variability. Its standard error is approximately \(\sqrt{p(1-p)/n}\). If the density \(f(q_p)\) is positive and changes smoothly near \(q_p\), a small change in cumulative proportion corresponds locally to a quantile change of roughly that amount divided by \(f(q_p)\). Consequently, the large-sample approximation is:

This approximation explains two important features: uncertainty generally decreases as the number of independent observations increases, and it is larger where the concentration distribution is relatively flat near the target quantile (small \(f(q_p)\)). It is not a plug-in guarantee for a sparse VPC bin. The density is usually unknown, the approximation can be poor with very small samples, and repeated observations within participants violate the independent-observation condition if records are counted as independent.

In a VPC, another way to judge the behavior of a quantile is to calculate it in each simulated dataset using the observed design, then examine the distribution of those simulated quantiles. This makes the reference reflect the bin occupancy and sampling schedule. The observed quantile remains one estimate from the actual dataset; the simulated distribution indicates how variable that estimate could be if the fitted model generated data under the design. It does not turn the observed quantile into an exact population value.

Worked Interpretation Across Sparse Bins

This example also illustrates why bin counts should appear in, or alongside, a sparsely sampled VPC. A smooth-looking line can conceal that one segment is based on far fewer participants than another. The plot should make the underlying support visible enough that a reader can distinguish data sparsity from a strong model signal.

Diagnostics, Alternatives, and Limits

Before interpreting a sparse VPC, check that simulation and observation records use the same sampling design and bin definitions. Tabulate both records and distinct participants per bin, and identify bins where a few participants contribute a large share of the records. Examine whether tail quantiles are repeatedly determined by one or two values. A marked change in a quantile after a small boundary adjustment is a warning that the summary is sensitive to bin composition, not proof that one binning choice is correct.

Also consider what a VPC can and cannot show. Its purpose is to compare selected distributional summaries, not to reveal every aspect of a concentration-time profile. In sparse data, a VPC may have inadequate resolution to identify brief changes between scheduled samples, and small bins may provide little evidence about tail behavior. Simulated reference distributions help describe expected sampling variation under the model and design, but they cannot demonstrate that the model is correct or repair a design with little information at a time of interest.

When bins are too sparse to support stable quantiles, reasonable alternatives or complements include:

• Report the bin-level occupancy and use fewer, scientifically defensible bins, while acknowledging the reduced time resolution.
• Show the observed data points, with participant identity or sampling structure made visible where practical, rather than relying only on connected quantile summaries.
• Use a different predictive-check summary that is appropriate to the question and supported by the data, rather than forcing an unstable tail quantile into the plot.
• Interpret the VPC together with other diagnostics and the study design; no single graphical check resolves all model-evaluation questions.

These approaches have their own limitations. Broader bins blur time variation; point displays may be visually crowded; and a different summary answers a different question. A sparse VPC is therefore best treated as a design-aware diagnostic with explicit limits, not as a definitive test. The central discipline is to match the strength of the interpretation to the information available in each bin.

Common Pitfall