Why Stratify Residual Diagnostics?
A residual plot pooled across all observations can look reassuring even when the model systematically overpredicts in one covariate subgroup and underpredicts in another. Stratifying residuals by a categorical covariate, or examining them across useful ranges of a continuous covariate, can expose patterns that disappear in an overall average. In pharmacometrics, such a pattern may motivate closer examination of the model’s covariate structure, but it does not by itself establish a biological mechanism or prove that a covariate effect is missing.
The key is to compare like with like. Groups may differ in the number of observations per subject, their sampling times, or the ranges of other covariates represented. If residuals also vary with time, for example, a group sampled mainly early and another sampled mainly late can have different average residuals even when their residual behavior is identical at each time. A useful stratified plot therefore asks whether residual behavior differs across covariate values at comparable parts of the design.
What a Grouped Residual Plot Can Show
Let \(y_{ij}\) denote the observation for subject \(i\) at observation \(j\), and let \(r_{ij}\) denote a chosen residual for that observation. The residual might be raw, weighted, conditional, or standardized; these quantities are not interchangeable. The earlier material on raw, weighted, and conditional residuals, and on standardizing residuals with prediction variance, provides the distinctions. Choose a residual whose fit target matches the question: a population-level residual is generally more informative about population predictions, whereas an individual-conditional residual addresses fit after subject-specific effects have been used.
Under a correctly specified model and appropriate residual calibration, residuals used to assess the relevant prediction target should show no systematic location pattern with the covariate. For standardized residuals, a center near zero is a useful diagnostic reference. This is not a guarantee that every finite sample will have a mean exactly zero, nor does it mean that an individual-conditional residual plot can diagnose every population-level problem. Conditional fits can absorb some subject-level discrepancies.
For a categorical covariate with level \(g\), a simple descriptive summary is the mean residual among observations in that group:
where \(n_g\) is the number of observations in group \(g\). A plot of individual residuals by group is usually more revealing than the group means alone: it can show skew, changing spread, clusters, or a few extreme points. Because subjects can contribute repeated observations, the plotted points are not necessarily independent. Group means and point clouds are diagnostic summaries, not automatically valid inputs to an ordinary independent-observations test.
For a continuous covariate, a scatterplot can be useful when there are enough observations across its range. Alternatively, define a small number of scientifically interpretable ranges and compare the residual distributions across them. A fitted smooth can help show gradual structure, but it should not be treated as a formal test. Bin boundaries, sparse regions, and uneven subject representation can all change the visual impression.
Worked Example: Comparing a Categorical Covariate
Suppose a hypothetical population-level diagnostic uses standardized residuals for two renal-function categories. The categories have similar sampling-time distributions and comparable representation of other important covariates. Four residuals in the reference category are \(-0.4,\ 0.2,\ 0.1,\ 0.3\); four in the lower-function category are \(0.7,\ 0.8,\ 0.4,\ 0.5\). These values are hypothetical and are stated in standardized residual units.
For the reference category,
For the lower-function category,
The residuals in the lower-function group are consistently more positive, while those in the reference group are centered close to zero. Since the example stipulates similar sampling-time and covariate distributions, this contrast is less readily explained by those design differences and is a reason to investigate subgroup misfit. It does not, on its own, show that renal function caused the discrepancy or identify which part of the model should change.
Continuous Covariates: Use Ranges Carefully
A continuous-covariate display should preserve as much information as the data allow. If there is sufficient coverage, plot residuals against the covariate itself and inspect the direction and shape of any trend. If the plot is too dense or coverage is limited, a few ranges can summarize the pattern. Choose boundaries that have scientific or practical meaning where possible; avoid trying many cut points until one appears to show a difference. A cut point discards information and can make a gradual trend look like an abrupt subgroup effect.
The range boundaries do not ensure that the groups are comparable. For example, the oldest range may contain observations mostly at a different time after dose than the youngest range. If residuals change with time, the binned plot can then show a difference that is actually associated with sampling time. Check covariate overlap and the distribution of relevant design variables within each range before interpreting the residual pattern.
Worked Example: Residuals Across Age Ranges
In a hypothetical diagnostic, age is summarized in three prespecified ranges. The standardized population residuals are \(-0.2,\ 0.1,\ 0.0\) for ages 20–39 years; \(-0.1,\ 0.3,\ 0.2\) for ages 40–59 years; and \(0.5,\ 0.4,\ 0.7\) for ages 60–79 years. Suppose the ranges have adequate covariate coverage and similar sampling-time distributions. Calculate each range’s mean and describe the pattern.
For ages 20–39 years, the sum is \(-0.2+0.1+0.0=-0.1\), so the mean is \(-0.1/3=-0.0333\). For ages 40–59 years, the sum is \(-0.1+0.3+0.2=0.4\), giving \(0.4/3=0.133\). For ages 60–79 years, the sum is \(0.5+0.4+0.7=1.6\), giving \(1.6/3=0.533\). All means are in standardized residual units.
The summaries rise across the ranges, and the older-range residuals are all positive in this small example. This suggests a pattern worth checking in a plot of residuals against age, rather than establishing a sharp change at age 60. The small number of observations per range also means that the displayed pattern is uncertain and could be sensitive to individual observations. Even with similar sampling-time distributions, inspect the underlying points and subject representation before drawing a substantive conclusion.
Why Group Size and Sampling Time Matter
A pooled mean is a weighted average of group means. For two groups \(A\) and \(B\), with \(n_A\) and \(n_B\) observations, respectively,
This identity follows by writing the total residual sum as the sum of the within-group sums: \(\sum r_{ij}=n_A\bar r_A+n_B\bar r_B\), then dividing by the total number of observations. Thus, a large group contributes more to the pooled mean. Conversely, a pooled mean near zero can conceal positive residuals in one group and negative residuals in another. Unequal group sizes affect how strongly each group contributes to the pooled summary; they do not, by themselves, demonstrate that the groups have different model fit.
Sampling time can create a second problem. Suppose the model’s residuals are positive early and negative late in the observation window, with the same time-specific pattern in each covariate group. If one group contributes mostly early observations and another mostly late observations, their overall means can differ even though there is no residual difference between the groups at a given time. This is a composition effect, not sufficient evidence of covariate-related bias.
Worked Example: A Group Contrast Created by Sampling Schedules
Consider hypothetical standardized population residuals with a mean of \(0.4\) at early sampling times and \(-0.2\) at late sampling times in both covariate groups. Group A has 8 early and 2 late observations; Group B has 4 early and 16 late observations. Group A has 10 observations and Group B has 20. Calculate the overall group means.
For Group A, the residual sum is \(8(0.4)+2(-0.2)=3.2-0.4=2.8\), so its mean is \(2.8/10=0.280\). For Group B, the sum is \(4(0.4)+16(-0.2)=1.6-3.2=-1.6\), so its mean is \(-1.6/20=-0.080\). The apparent group contrast is \(0.280-(-0.080)=0.360\) standardized residual units.
Yet the groups have the same residual mean at each sampling-time category: \(0.4\) early and \(-0.2\) late. Their different overall means arise from different time distributions. The pooled mean is \((2.8-1.6)/30=1.2/30=0.040\), which is also close to zero and would hide the within-time residual pattern. A suitable next diagnostic is to compare groups within comparable time ranges, or to display residuals against time separately by group. If the groups have little or no overlap in time, the data may not support a clean visual separation of covariate and time patterns.
A Practical Reading Strategy
Interpret a stratified residual display in a sequence that separates model behavior from data composition. First identify the prediction target and residual type. Then inspect the number of observations and subjects represented in each group or range, the residual center and spread, and the distribution of sampling times. If an overall group contrast appears, check whether it remains when observations are compared over overlapping time ranges and covariate support. A pattern that persists across those comparisons is more suggestive of subgroup misfit, although still not a formal test or a causal conclusion.
State whether the display concerns population-level predictions or individual-conditional fit, and use the same residual definition when comparing groups.
Review subject representation, observation counts, sampling times, and overlap in other relevant covariates before comparing residual centers.
Inspect the point cloud for trends, changing spread, skew, clusters, or a few observations that dominate the summary.
Ask whether the contrast remains at comparable times and within supported covariate ranges. If not, describe the composition difference rather than attributing the pattern to the covariate.
A stratified plot is descriptive. A formal comparison would need a specified estimand and an analysis that accounts for the repeated observations within subjects and the design. A residual pattern can motivate such follow-up, but the plot alone does not supply a p-value, establish a covariate effect, or justify changing the model. Likewise, a lack of a visible pattern is not proof that the model is correct, particularly when groups are small or poorly represented.
Alternatives, Limitations, and Common Pitfalls
| Approach | What it is useful for | Main caution |
|---|---|---|
| Pooled residual plot or summary | Shows overall residual behavior across the data set. | Opposing subgroup patterns can cancel; group and time composition can dominate. |
| Residuals displayed by covariate group or range | Makes subgroup-specific location, spread, and shape easier to see. | Small or unbalanced groups and unequal sampling schedules can mislead. |
| Residuals against a continuous covariate | Preserves the covariate scale and can reveal gradual trends. | Dense points, sparse tails, and over-interpreted smoothing can obscure uncertainty. |
| Formal model comparison or covariate-effect analysis | Can address a specified inferential question using an explicit model. | Requires appropriate assumptions and design-aware inference; it is not supplied by a diagnostic plot. |
Common mistake: “The mean residual is higher in Group A, so the covariate must be causing the model’s bias.” This reasoning ignores how residuals are distributed across times and subjects. The correction is to check the group’s sampling schedule and covariate overlap, then compare residual behavior under comparable design conditions. Even a persistent pattern is evidence of a diagnostic discrepancy, not proof of causation.
- Do not compare only group means. A mean can hide spread, skew, or a pattern driven by a few points.
- Do not treat every observation as an independent subject. Repeated measurements from one subject can make the point count look more informative than the number of subjects warrants.
- Do not create many arbitrary bins. Repeatedly changing cut points invites unstable, selectively noticed patterns.
- Do not infer absence of bias from a pooled mean near zero. Positive and negative subgroup patterns can cancel.
These diagnostics are limited by the information in the design. If covariate groups have little overlap in sampling times or other relevant covariates, the plot may not distinguish their contributions. If a continuous covariate is sparsely observed at its extremes, apparent tail trends may be unstable. Report what the plot shows, how much data support each comparison, and what design differences remain.
Check Your Understanding
Answer each question, then compare with the solutions below.
- Why can a pooled residual mean near zero fail to reveal subgroup misfit?
- Two covariate groups have different overall residual means, but the same mean residual within early and late sampling-time categories. What should you investigate before attributing the contrast to the covariate?
- When might ranges be preferable to a continuous-covariate scatterplot, and what is one limitation of binning?
- Does a persistent positive residual pattern in one covariate group prove that the covariate caused the model discrepancy? Explain.
Solutions. (1) The pooled mean is weighted by group sizes, and positive residuals in one group can cancel negative residuals in another. The overall mean can therefore be near zero despite clear subgroup patterns. (2) Check the groups’ sampling-time distributions, subject representation, and overlap in other relevant covariates. If their time compositions differ, compare residuals within comparable times rather than interpreting the overall contrast as a covariate pattern. (3) Ranges can make a dense or difficult-to-read plot easier to summarize, especially when ranges have a practical interpretation. Binning discards information and can make results depend on the chosen boundaries. (4) No. A persistent pattern is a diagnostic signal that merits investigation; it does not establish causation or identify a unique model change. A formal, design-appropriate analysis would be needed to address a specified inferential question.