Why Stratify a Goodness-of-Fit Plot?
A pooled goodness-of-fit plot can look reassuring even when the model behaves differently across occasions, dose levels, studies, or sampling patterns. For example, positive residuals in one study and negative residuals in another can cancel when all observations are combined. Stratification means displaying observations in groups defined by a relevant feature so that these differences can be seen.
The aim is not to create as many panels as possible. Each split should answer a diagnostic question: does the model show a different pattern after the first versus later doses? Does fit differ between studies? Are observations collected under sparse and intensive sampling schedules behaving similarly? A useful display makes that comparison visible while leaving enough data in each group to interpret its pattern.
Let \(r_{ij}\) denote a selected residual for subject \(i\) at observation \(j\). Depending on the diagnostic question, the vertical axis might show a weighted residual, while the horizontal axis might show time since dose, population prediction, or another relevant quantity. Observed concentrations and population predictions can also be displayed against one another. The key decision here is which observations to compare separately, not which residual definition or axis transformation to use.
Stratification is descriptive. A visible difference between panels is a reason to investigate, not by itself proof that a dose or study characteristic caused the difference. Observations from the same subject may be correlated, and a visual plot does not automatically account for that dependence or provide a formal test.
Choosing Useful Strata
Begin with a diagnostic question and choose a variable that can address it. Occasion can reveal whether the fit changes between dosing periods. Dose categories can help assess whether a pattern differs across dose levels. Study can expose differences in protocols, populations, or data handling. Sampling intensity can help show whether the display looks different for subjects with many versus few observations.
Define groups in a way that is interpretable and reproducible. For occasion, specify whether the categories refer to the first dose, a later dose, or a particular dosing interval. For dose, state the units and cut points. For sampling intensity, specify how observations or sampling schedules are counted and how categories are formed. Avoid categories chosen only after looking for a split that makes a pattern appear striking; such data-driven choices can exaggerate apparent contrasts.
A stratification variable can also be entangled with other features. If one study mostly contributes low-dose observations and another mostly contributes high-dose observations, a difference between study panels does not isolate a study effect from dose. Keep the relevant design context visible, and avoid interpreting a panel contrast as a clean comparison when group composition differs substantially.
- Use a common residual definition. Do not change how the vertical-axis quantity is calculated from panel to panel without making that distinction explicit.
- Keep axes comparable. Use the same scale and reference line across panels when comparing the size or direction of patterns.
- Show group sizes. Counts help readers judge whether a panel is informative or dominated by a few observations.
- Preserve the pooled view. A stratified display supplements, rather than replaces, an overall diagnostic.
How Pooled Patterns Can Conceal Stratum-Specific Bias
For a simple summary, suppose stratum \(g\) has \(n_g\) residuals and their arithmetic mean is \(\bar{r}_g\). The pooled mean of all residuals is the weighted average of the stratum means, with weights proportional to group size:
This follows directly by adding the residuals within each group. The total residual sum is \(\sum_g n_g\bar{r}_g\), and dividing by the total number of observations, \(\sum_g n_g\), gives the pooled mean. Thus, the pooled summary can be near zero because positive and negative group means offset, not because every group is well centered. Unequal group sizes also mean that the pooled display gives more influence to groups with more observations.
The argument applies to this arithmetic summary, not to every visual feature of a plot. A zero mean does not rule out curvature, changing spread, time-dependent structure, or extreme values. Stratified plots should therefore be read for patterns as well as simple group averages.
Worked Example: Opposing Patterns Across Occasions
In a hypothetical repeated-dose analysis, eight conditional weighted residuals are grouped by occasion. The four residuals from occasion 1 are \(-1, 0, 1, 0\); those from occasion 2 are \(1, 2, 1, 2\). These illustrative values are used only to show how grouping changes interpretation.
Occasion 1 mean. The sum is \(-1+0+1+0=0\), so \(\bar{r}_1=0/4=0\).
Occasion 2 mean. The sum is \(1+2+1+2=6\), so \(\bar{r}_2=6/4=1.50\).
Pooled mean. The sum across all eight residuals is \(0+6=6\), so \(\bar{r}_{\mathrm{pooled}}=6/8=0.750\). Equivalently, \((4\times0+4\times1.50)/8=0.750\).
A pooled plot suggests a positive average residual, but it does not show that the positive pattern is concentrated in occasion 2. The stratified view distinguishes a group centered at zero from a group with a positive mean. Before interpreting this as a model problem, check that occasions were classified correctly and that the same residual definition and scale were used in both panels. The means alone do not establish the full shape of either residual distribution.
Worked Example: Study-Specific Patterns That Cancel
Consider a hypothetical diagnostic summary with six residuals from each of two studies. In Study A, the values are \(-1.2, -0.8, -1.0, -1.1, -0.9, -1.0\). In Study B, they are \(1.2, 0.8, 1.0, 1.1, 0.9, 1.0\). The studies have equal numbers of observations in this example.
Study A mean. Its sum is \(-1.2-0.8-1.0-1.1-0.9-1.0=-6.0\), giving \(\bar{r}_A=-6.0/6=-1.00\).
Study B mean. Its sum is \(1.2+0.8+1.0+1.1+0.9+1.0=6.0\), giving \(\bar{r}_B=6.0/6=1.00\).
Pooled mean. The combined sum is \(-6.0+6.0=0\), and the pooled mean is \(0/12=0\). The overall mean is exactly zero, despite substantial directional differences between the two study groups.
A pooled display could therefore conceal the contrast. In the study-stratified display, both panels should retain the same vertical scale and the same zero reference line, making the opposite directions easy to compare. Next, investigate the study designs, data processing, and composition of the groups. The plot does not show which of these explains the contrast, and it does not establish that study membership caused it.
Balancing Detail Against Sparse Panels
Every added split reduces the number of observations available within at least some panels. A plot divided simultaneously by study, dose category, occasion, and sampling intensity may produce many panels with only a few points each. Such panels can suggest unstable patterns, hide the overall distribution, or make comparisons difficult because the reader must repeatedly search for corresponding groups.
There is no universal minimum count that makes a panel adequate. Its usefulness depends on how observations are distributed across the diagnostic axis, whether observations repeat within subjects, and what kind of pattern is being assessed. Report the counts, inspect whether points cover the relevant range, and treat conclusions from very small groups as uncertain. A large count alone does not guarantee useful coverage if observations cluster in only a narrow part of the axis.
Worked Example: Choosing a Sampling-Intensity Display
A hypothetical analysis contains 30 subjects: 2 have sparse sampling, 8 have moderate sampling, and 20 have intensive sampling. Their total observation counts are 6, 32, and 160, respectively. The corresponding mean residuals are \(0.20\), \(-0.10\), and \(0.05\).
Check the pooled mean. Weighting each group mean by its number of observations gives \[ \bar{r}_{\mathrm{pooled}} = \frac{6(0.20)+32(-0.10)+160(0.05)}{6+32+160} = \frac{1.2-3.2+8.0}{198} = \frac{6.0}{198} \approx 0.0303. \]
The pooled mean is close to zero, and the group means are also relatively close to zero in this example. But the sparse-sampling panel contains observations from only two subjects. Its point pattern cannot support the same level of confidence as the much larger intensive-sampling panel, especially if those six observations cover only a limited range of time or predictions.
A practical first display could use three panels, report the subject and observation counts in each, and retain common axes. If the sparse panel is too thin to interpret, keep the overall plot prominent and present the sparse group with its limited evidence clearly identified, rather than further subdividing it. Do not let the 160 observations from intensively sampled subjects make the other groups disappear from view.
Diagnostics, Alternatives, and Common Pitfalls
Check whether each intended group is actually represented as expected. Verify category definitions, sample counts, and subject membership; confirm that observations have not been assigned to multiple or incorrect groups unintentionally. Then compare the panels on common axes, including the relevant reference line. Look for changes in location, spread, range, or the relationship between residuals and the horizontal-axis quantity. A panel with few points may have no visible pattern simply because it contains too little information.
Faceting is often useful when the goal is to compare shapes across groups. Coloring or symbol-coding groups in one shared plot can preserve a direct view of overlap and make an overall distribution easier to see, but many colors or symbols can become hard to distinguish. Separate plots can make individual patterns legible, yet comparison is unreliable if their axes differ. A pooled plot is compact and provides an overall view, but can mask group-specific behavior. These displays answer complementary questions; none is a universal substitute for the others.
For a continuous variable such as time or a covariate, arbitrary cut points can discard information and create groups that depend on the chosen thresholds. If the diagnostic question concerns a continuous trend, retain an appropriate continuous-axis display as well. Do not use stratification as a substitute for examining that relationship.
- Pitfall: “The pooled mean is near zero, so there is no bias.” Opposing group means can cancel. Inspect relevant strata and the point patterns, not only the pooled average.
- Pitfall: “Every subgroup needs its own panel.” Excessive splitting creates sparse, difficult-to-compare displays. Choose a small number of diagnostically motivated groupings and show counts.
- Pitfall: “A panel difference proves a group effect.” Group composition, study design, or repeated observations may contribute. Treat a visual contrast as a prompt for investigation, not a causal conclusion.
- Pitfall: “Rescaling each panel improves comparison.” Separate scales can make different magnitudes look alike. Use common axes for direct comparison, or clearly mark the display as intended for within-panel shape only.
Check Your Understanding
Answer each question using the principles of purposeful, comparable stratification.
- Two equally sized study groups have mean residuals of \(-0.6\) and \(0.6\). What is their pooled mean, and why is it potentially misleading?
- A plot stratified by occasion uses a different vertical-axis range in each panel. What comparison does this make difficult?
- A dose-by-study display has several panels with only two observations. Give one reason to avoid strong conclusions from those panels.
- Why should a stratified diagnostic not automatically be interpreted as evidence that the stratifying variable caused a model discrepancy?
Solutions. (1) With equal group sizes, the pooled mean is \((-0.6+0.6)/2=0\). It is potentially misleading because the opposing group means cancel, concealing both directional patterns. (2) Different ranges make the magnitudes of residual patterns across occasions difficult to compare directly; common scales are preferable for that purpose. (3) Two points provide little information about the shape or variability of a group’s pattern, and the observations may not cover the diagnostic range. (4) Other group characteristics, design differences, or within-subject dependence may contribute to a visual contrast, so the plot describes a pattern rather than establishing its cause.