Why Match More Than One Summary?
A virtual population can reproduce a target mean while missing the spread or the clinically relevant tails of a predicted outcome. Conversely, matching a few quantiles does not guarantee that the average or variance is right. Matching means, variances, and quantiles makes these trade-offs explicit: each summary captures a different feature, and none alone establishes that two full distributions are alike.
The summaries must refer to the same quantity, units, time point, and population definition. For example, a target for drug exposure at week 4 in a subgroup should not be compared with predictions for the full virtual population at week 2. The target values may come from observed data or specified population summaries; their provenance and uncertainty should be recorded. The criteria below describe agreement with those summaries, not proof that the virtual population is representative in every other respect.
Definitions for a Weighted Virtual Population
Suppose a virtual population contains \(m\) patients, with predicted values \(x_1,\ldots,x_m\) for one outcome. Assign each patient a nonnegative weight \(w_i\), with \(\sum_{i=1}^{m}w_i=1\). Equal weights give \(w_i=1/m\). These are descriptive summaries of the weighted population, not sample estimates with a degrees-of-freedom correction.
For equal weights, the mean is the arithmetic average and the variance divides the sum of squared deviations by \(m\), not \(m-1\). The latter divisor is used for the usual sample variance \(s^2\) when estimating a population variance from a sample; it is not the definition of the variance of the finite weighted virtual population itself. Quantile conventions also matter. Here the quantile is the first ordered value at which cumulative weight reaches or exceeds \(p\); interpolated sample quantiles can produce different values.
What Moments and Quantiles Capture
The mean gives the population’s average predicted outcome. The variance describes average squared departure from that mean, and therefore summarizes spread while giving relatively strong influence to distant values. Neither identifies the distribution’s shape: asymmetry, multiple clusters, and particular tail behavior can differ even when the first two moments agree.
A quantile specifies a location in the ordered distribution. For example, \(q_{0.25}\) is the value at or below which at least 25% of the total weight lies under the stated inverse-CDF convention. Selected quantiles can make lower- and upper-tail locations visible and are often less affected by extreme values than the mean or variance. But a finite set of quantiles leaves unreported the behavior between and beyond those points.
Worked Example: Equal Moments, Different Quantiles
Consider two hypothetical equally weighted sets of four virtual-patient predictions for a centered biomarker deviation, measured in mg/L. Set A is \(-1,-1,1,1\); set B is \(-\sqrt{2},0,0,\sqrt{2}\). Negative values are possible because this outcome is a deviation from a reference concentration, not an absolute concentration.
For A, the mean is \((-1-1+1+1)/4=0\) mg/L. Its population variance is \([1+1+1+1]/4=1\) (mg/L)\(^2\). For B, the mean is \((-\sqrt{2}+0+0+\sqrt{2})/4=0\) mg/L, and the variance is \([(2)+0+0+(2)]/4=1\) (mg/L)\(^2\). Thus the two sets have identical means and variances.
For \(p=0.25\), the first ordered value is the first at which cumulative weight reaches 0.25: each observation has weight 0.25. Therefore \(q_{0.25}=-1\) mg/L for A and \(q_{0.25}=-\sqrt{2}\approx-1.414\) mg/L for B. Their lower quartiles differ despite equal means and variances. The example shows why moment matching alone does not determine distributional shape.
Calculating Weighted Summaries
Worked Example: A Weighted Population and Its Target
A hypothetical selected virtual population has four predicted concentrations: 2, 4, 6, and 8 mg/L, with weights 0.10, 0.20, 0.30, and 0.40. Suppose target summaries are mean 5.5 mg/L, variance 3.5 (mg/L)\(^2\), and quantiles \(q_{0.25}=4\), \(q_{0.50}=5.5\), and \(q_{0.75}=7.5\) mg/L.
The virtual-population mean is \(0.10(2)+0.20(4)+0.30(6)+0.40(8)=6.0\) mg/L. Its variance is \(0.10(2-6)^2+0.20(4-6)^2+0.30(6-6)^2+0.40(8-6)^2=1.6+0.8+0+1.6=4.0\) (mg/L)\(^2\).
The cumulative weights at 2, 4, 6, and 8 mg/L are 0.10, 0.30, 0.60, and 1.00. Hence \(q_{0.25}=4\) mg/L, \(q_{0.50}=6\) mg/L, and \(q_{0.75}=8\) mg/L. Compared with the targets, the mean is 0.5 mg/L higher and the variance is 0.5 (mg/L)\(^2\) higher; the lower quartile matches, while the median and upper quartile are each 0.5 mg/L higher. Reporting these separately shows which features fit instead of masking them in one overall statement.
Constructing a Matching Criterion
A practical criterion compares each virtual-population summary with its corresponding target. Since means, variances, and quantiles have different units and scales, raw differences should not simply be added. One approach is to divide each difference by a prespecified scale \(a_j>0\), such as an acceptable discrepancy or a scale reflecting target uncertainty, then sum squared standardized differences.
The scales and weights encode priorities and must be chosen before comparing candidate populations, not adjusted afterward to favor a preferred result. A separate set of prespecified tolerances can instead require each summary to fall within an acceptable range. Either approach should preserve individual discrepancies in reporting: a small combined loss can conceal a consequential mismatch if that component has been assigned a small weight.
Worked Example: Comparing Standardized Matching Losses
Suppose target summaries are mean 10 mg/L, variance 4 (mg/L)\(^2\), \(q_{0.25}=8\) mg/L, and \(q_{0.75}=12\) mg/L. For this hypothetical comparison, use scales of 1 mg/L for the mean, 2 (mg/L)\(^2\) for the variance, and 1 mg/L for each quantile, with all \(\lambda_j=1\).
Candidate A has summaries 9 mg/L, 4 (mg/L)\(^2\), 8 mg/L, and 12 mg/L. Its loss is \([ (9-10)/1]^2+[(4-4)/2]^2+[(8-8)/1]^2+[(12-12)/1]^2=1+0+0+0=1\).
Candidate B has summaries 10 mg/L, 8 (mg/L)\(^2\), 8 mg/L, and 12 mg/L. Its loss is \([ (10-10)/1]^2+[(8-4)/2]^2+[(8-8)/1]^2+[(12-12)/1]^2=0+4+0+0=4\). Under this stated criterion, A has the smaller loss. It nevertheless misses the target mean by 1 mg/L, so whether A is acceptable depends on a prespecified tolerance or scientific priority; the loss alone does not make that decision.
Why the Variance Formula Behaves as It Does
A useful identity explains why the mean is the center that minimizes weighted squared deviations. For any proposed center \(a\), write \(x_i-a=(x_i-\mu_V)+(\mu_V-a)\), square, and weight the terms. The cross-term sums to zero because the weighted deviations from the mean sum to zero.
The second equality uses \(\sum_iw_i(x_i-\mu_V)=0\) and \(\sum_iw_i=1\). Since \((\mu_V-a)^2\ge0\), the weighted squared deviations are minimized at \(a=\mu_V\). This clarifies why variance is tied to squared distance from the mean; it does not imply that variance alone describes the full distribution.
Choosing Summaries and Diagnosing Failure
Moment criteria are useful when the average response and overall spread are primary targets and their scales are meaningful. Quantile criteria are useful when specified locations in the distribution matter, for instance when a lower or upper portion of predicted exposure is relevant. A combined criterion can address both, provided it does not treat a few matched numbers as a complete description of the population.
Before interpreting a match, check that the candidate and target use compatible units, population definitions, time points, and quantile conventions. Check the weight normalization and confirm that no unintended zero or negative weights are present. Display the component differences as well as any combined loss, and inspect the ordered predictions or a plot of the distribution. These checks can reveal asymmetry, clusters, or tail behavior hidden by the selected summaries.
A common mistake is to conclude that matching the mean and variance means the distributions match. The first worked example disproves that claim: its two sets have equal moments but different lower quartiles. The correction is to say only that the specified moments match, then evaluate relevant quantiles or other prespecified distributional features. The converse mistake is to assume that matching several quantiles guarantees the mean, variance, or behavior beyond the most extreme selected quantile; it does not.
Targets may themselves be uncertain or based on limited observations. In that case, a discrepancy should be interpreted relative to target uncertainty rather than as a precise biological difference. Also, weighted summaries can look well matched even when only a small number of virtual patients carry most of the total weight. Report the weights and inspect how much population weight supports the summaries. These are diagnostics for the matching exercise, not substitutes for assessing whether the virtual population is appropriate for its intended use.
| Approach | What it emphasizes | What it can miss |
|---|---|---|
| Mean and variance | Average and squared spread around that average | Different shapes, asymmetry, or quantile locations |
| Selected quantiles | Locations at specified cumulative probabilities | Behavior between selected points and beyond the outer quantiles |
| Combined criterion | Agreement on a prespecified set of moments and quantiles | Any feature omitted or given insufficient weight |
These approaches are complements rather than interchangeable guarantees. The appropriate summaries depend on the scientific target; additional distribution-level comparisons may be useful when the full shape matters, but matching selected summaries should remain clearly distinguished from matching a distribution.
Check Your Understanding
Answer each question, then compare with the solutions below.
- For equally weighted values 1, 3, and 5 mg/L, calculate the mean, population variance, and inverse-CDF quantiles at 0.25, 0.50, and 0.75.
- For equally weighted values 0, 0, 0, and 12 mg/L, calculate the mean, population variance, and \(q_{0.75}\). What does this show about that quantile?
- A candidate matches a target mean and both specified quartiles but has twice the target variance. Is it accurate to say that the candidate matches the target distribution? Explain.
- Name two checks to make before interpreting a weighted virtual population’s summary discrepancies.
Solutions. 1. The mean is \((1+3+5)/3=3\) mg/L. The variance is \([(1-3)^2+(3-3)^2+(5-3)^2]/3=8/3\approx2.667\) (mg/L)\(^2\). Cumulative weights are \(1/3,2/3,1\), so \(q_{0.25}=1\), \(q_{0.50}=3\), and \(q_{0.75}=5\) mg/L. 2. The mean is \(12/4=3\) mg/L, and the variance is \([9+9+9+81]/4=27\) (mg/L)\(^2\). Cumulative weight at 0 mg/L is already 0.75, so \(q_{0.75}=0\) mg/L. A quantile identifies a cumulative location under the convention; it does not summarize the magnitude of values above it. 3. No. The candidate matches only the stated mean and quartiles; its variance differs, and other shape features may differ as well. Report the variance discrepancy and assess it against prespecified criteria. 4. Examples include checking that targets and predictions use the same units, outcome, time point, and population definition; verifying that weights are nonnegative and sum to one; and inspecting whether a few patients carry most of the weight.