Interpreting Marginal And Joint Fit Diagnostics

Why Read the Diagnostics Together?

A generated virtual population can match several target means and still represent the wrong population. Its marginal distributions may look plausible while relationships between parameters are distorted; common subgroups may be well represented while a clinically important rare subgroup is nearly absent. A diagnostic panel is useful because it places these different kinds of evidence side by side. The task is not to find one universally good score, but to decide whether the population is adequate for its intended use and whether any mismatch matters for that use.

The interpretation depends on the target being checked, the population being evaluated, and the rules specified for judging fit. Targets may be reference summaries, clinically justified ranges, or prespecified constraints. A plot or residual has no universal pass threshold: a small discrepancy may matter greatly for a decision-critical phenotype, while a larger discrepancy in an irrelevant region may not affect the intended analysis.

Interpret a panel in context. First confirm that each displayed quantity refers to the intended population, variables, units, and eligibility denominator. Then check marginal shapes and summaries, joint structure, subgroup representation, and target residuals. Finally, decide which discrepancies could affect the model outputs or decisions for which the virtual population will be used.

What Each Panel Component Can Tell You

Marginal plots can reveal differences that a mean or standard deviation alone misses: a shifted tail, a support gap, an extra mode, or a concentration of values at a boundary. Histograms and density displays depend on bin widths and smoothing choices, so small visual differences should not be treated as exact measurements. Quantile plots can make distributional shifts easier to locate, but they still examine one variable at a time.

Correlation summaries provide a compact view of pairwise linear association. They do not describe every aspect of a joint distribution: equal correlations do not imply equal conditional distributions, tail dependence, or higher-order structure. Check that the same variables, transformations, and eligible observations are used for the target and generated summaries. A correlation computed in the full population cannot be directly interpreted as a subgroup correlation.

Subgroup counts show how many generated individuals satisfy each phenotype definition. Read each count with its denominator and definition. A count of 10 has a different meaning in a population of 100 than in a population of 1,000. Also inspect important intersections: adequate counts for two phenotypes separately do not guarantee adequate representation of individuals who have both.

Target residuals make the direction and scale of discrepancies explicit. A residual in the original units is directly interpretable, but residuals for different quantities cannot be compared just by their numerical magnitudes. Standardization can help, provided the scaling quantity has a justified interpretation, such as a prespecified tolerance. It should not be mistaken for an inferential standard error unless the sampling model and purpose actually justify that interpretation.

Residual Scaling: A Short Justification

Suppose a target summary is \(T\), the corresponding generated summary is \(G\), and a positive, prespecified tolerance is \(\tau\), expressed in the same units as \(G\) and \(T\). The signed residual and its tolerance-scaled version are

The ratio \(r_{\tau}\) is dimensionless. If \(r_{\tau}=1\), the generated summary is one tolerance above its target; if \(r_{\tau}=-1\), it is one tolerance below. This follows directly because \(G-T=r_{\tau}\tau\). The sign preserves direction, and the magnitude expresses discrepancy in units of the chosen tolerance. This scaling makes quantities with different units easier to review, but it does not make them equally important or establish that any particular value is acceptable. Tolerances should be set from the target's relevance and intended use, not selected after seeing the discrepancies.

Subgroups: Counts, Denominators, and Rarity

A subgroup diagnostic is interpretable only when its membership rule and denominator are explicit. For a subgroup defined among all eligible generated individuals, its observed proportion is the subgroup count divided by the eligible count. If the target is a proportion \(p\), the expected count at generated size \(n\) is \(np\); this is a useful scale for checking representation, not automatically a requirement that the realized count equal \(np\). Eligibility restrictions change the denominator, and overlapping phenotypes mean subgroup counts need not add to the population total.

Rare subgroups deserve particular attention. A small absolute count may make it difficult to characterize within-subgroup variability or support the intended simulation, even if the overall population is large. Conversely, a count difference should not be judged without considering how the population was constructed: a deliberately constrained finite population is not the same as a random sample from a source population. A count-based uncertainty calculation may be a helpful reference under an appropriate sampling interpretation, but it does not by itself certify the generated population.

Joint Fit and Conditional Structure

A marginal match is not evidence of a joint match. If two parameters each have the correct distribution, their generated correlation can still be too weak, too strong, or reversed. This matters when model behavior depends on combinations of parameters—for example, when high exposure and low sensitivity jointly identify a response phenotype. Pairwise correlations are useful alerts, but they do not replace inspection of important conditional or multivariable patterns.

From a Panel to an Adequacy Decision

A defensible decision uses the panel as a set of complementary checks rather than collapsing everything into one number. A practical review can proceed as follows:

This approach differs from relying on a single distribution-distance score, which can summarize selected discrepancies but may obscure their location or clinical relevance. It also differs from treating a plot as a formal hypothesis test: plots help diagnose patterns, while formal inference requires a defined sampling model and addresses a different question. The earlier material on distance metrics provides tools for quantitative comparisons; interpretation still requires identifying what each discrepancy means for the target use.

Common Mistakes and Limitations

• “The means match, so the population fits.” Matching means does not rule out tail, support, subgroup, or dependence problems. Review the rest of the panel.
• “The overall score is good, so every important feature is good.” A combined score can average away a severe mismatch in a decision-critical subgroup or relationship. Keep critical checks visible and apply their criteria separately.
• “A standardized residual is a z-score.” A residual divided by a tolerance is in tolerance units, not necessarily standard errors. Do not attach a probability or significance interpretation without a justified sampling distribution.
• “The subgroup count is close, so it is sufficient.” Closeness to a target proportion does not guarantee enough individuals for the planned within-subgroup analysis. Check the absolute count and intended use.
• “Matching correlations proves the joint distributions match.” Correlation is only one feature of dependence. Inspect relevant conditional patterns and combinations as well.

Diagnostics are limited by the targets and features they include. A panel cannot reveal a mismatch in an unreported variable, an unexamined subgroup, or a joint feature that its summaries do not capture. Adequacy is therefore conditional on the intended use and the diagnostic coverage. Documenting that scope makes the conclusion more useful than an unqualified statement that a virtual population “fits.”