Detecting Support Gaps In Generated Populations

Why Support Gaps Need Their Own Diagnostic

A generated virtual population can have a plausible mean, variance, and overall range while still omitting a meaningful part of the target population. For example, its minimum and maximum values may bracket the target values, yet almost no virtual patients may lie in a clinically relevant middle range. Such an omission can matter when model predictions depend on that region or when a planned analysis concerns patients with those characteristics.

A support-gap diagnostic asks a focused question: does the candidate or selected population contain patients in target regions that should be represented? It complements distribution summaries and distance measures rather than replacing them. The word “support” requires care: a finite sample cannot establish every point where an underlying continuous population has positive probability. In practice, gaps are assessed over specified intervals or regions at a resolution chosen for the scientific purpose.

The diagnostic depends on a clear target definition. A target can come from a reference sample, population summaries translated into intervals, or explicit eligibility and physiological constraints. Reference and virtual-population values must use comparable definitions, units, transformations, and inclusion rules. If a reference sample is used to represent a broader target population, it should be sufficiently representative for the intended comparison. Without that condition, the plots describe the samples but may not describe the populations of interest.

Empirical Cumulative Distributions

For a single characteristic, the empirical cumulative distribution function (ECDF) records the fraction of observations at or below each value. Let \(T_1,\ldots,T_{n_T}\) denote values in the target reference and \(V_1,\ldots,V_{n_V}\) values in the virtual population. Then:

Here \(I(\cdot)\) is 1 when its condition is true and 0 otherwise. For an interval \((a,b]\), the empirical fraction in the interval is the ECDF increment \(F(b)-F(a)\). Thus, if the target ECDF rises across an interval but the virtual-population ECDF is flat there, that interval contains target observations and no virtual observations. The interval boundaries should be stated explicitly because ECDFs count observations at or below a boundary.

Density Plots and the Resolution of a Gap

Histograms and density plots can make gaps easier to see across a wider range of values. A histogram displays counts or proportions in chosen bins. A density plot, commonly made by smoothing observations, displays an estimated distribution over the value scale. Both require judgment about resolution. Narrow bins can produce many empty bins in a modest sample, while wide bins can merge a meaningful gap with covered values. Kernel smoothing can draw a continuous-looking curve across a region with no observations; that smooth curve is not itself evidence that virtual patients occupy the region.

A practical check is to plot target and virtual-population values on the same scale, state the bin widths or smoothing bandwidth, and mark important target intervals. Inspect the observed values or bin counts alongside the curve. If a density curve appears to bridge a proposed gap, verify whether any actual observations lie there. Changing bandwidth or bin width is a useful sensitivity check, but it should not replace a scientifically justified region definition.

Range Checks and Multidimensional Regions

A range check compares observed minima and maxima with admissible bounds or target ranges. It is simple and often useful for detecting values outside physiological or design constraints. However, checking only endpoints cannot show how observations are distributed between them. A population can pass a range check and still omit a large interior region. Conversely, a single observation in an interval establishes sample presence, but not adequate representation.

For multiple patient characteristics, checking each marginal range separately is not enough to show that their combinations are covered. A target region may require a particular combination of values. Define that region directly and count patients that satisfy all its conditions. This is a localized diagnostic; it does not by itself establish that the full joint distribution is well represented.

A Practical Diagnostic Sequence

A defensible support-gap assessment follows a sequence that makes the target, resolution, and evidence explicit. The checks below can be applied to candidate populations before selection and to selected populations after filtering or resampling.

Comparisons, Failure Modes, and Limitations

These tools answer related but different questions. An ECDF shows cumulative distribution differences and can localize intervals where one population accumulates observations and another does not. A histogram or density plot gives a visual summary whose apparent resolution depends on bins or smoothing. A range check is a fast boundary check but is weak evidence about interior coverage. A global distance metric summarizes distributional discrepancy in a chosen way, but a single overall value may not identify which target region is empty. Combining regional counts with plots and bounds is often more informative than treating any one diagnostic as sufficient.

• Choosing regions after seeing the result. This can make an ordinary fluctuation look like a meaningful gap. Define important regions from the target question where possible, and label exploratory regions as exploratory.
• Treating a smoothed curve as observed coverage. A density estimate can be positive where there are no observations. Check the raw values or bin counts before concluding that a region is represented.
• Equating one observed patient with adequate coverage. Presence is not the same as adequate representation. Report counts and fractions, and interpret them against the intended use rather than declaring sufficiency from a nonzero count alone.
• Inferring a population-level zero from a sample gap. A finite candidate may omit an interval by chance. The diagnostic establishes what is absent from that generated sample, not the exact probability in an unlimited population.
• Checking only marginal ranges. Plausible values for each trait do not guarantee that target combinations occur together. Count membership in relevant joint regions.

Small samples make observed gaps especially sensitive to sample size and the chosen resolution. Repeating generation can show whether a gap recurs, but a recurring gap may still reflect the generation or selection procedure rather than a property of the target. These plots and counts are diagnostics, not automatic acceptance rules. Their role is to reveal where representation deserves investigation and to make the evidence transparent.