Residuals By Study, Site, And Assay

Why Stratify Residuals by Study, Site, or Assay?

A pooled goodness-of-fit plot can look acceptable even when predictions are systematically high in one study and low in another. The same masking can occur across clinical sites or laboratory assays. Stratifying residuals by these labels asks a focused question: does prediction or measurement behavior differ across the groups that generated or processed the observations?

Such a pattern is a diagnostic clue, not a diagnosis. A study difference might reflect a population or protocol difference, but it might also reflect a unit conversion or a different sampling schedule. A site difference could indicate a local measurement process, yet it could arise because sites enrolled different kinds of subjects. An assay difference may signal calibration behavior, while also tracking concentration range or sample handling. The plot helps locate a pattern; contextual checks are needed to explain it.

Choose the Residual and the Grouping Carefully

Use a residual whose scale and prediction level match the question. For population-level model adequacy, a population-based standardized residual such as CWRES is often informative. For how well the model describes observations after subject-specific effects are accounted for, an individual-level residual such as IWRES can be examined as a complementary display. These do not answer identical questions: individual-level residuals condition on estimated individual effects, so apparent improvement in a group pattern there does not by itself rule out a population-level discrepancy.

A residual based on observed minus predicted values is positive when the observation exceeds its prediction and negative when it falls below. Standardization puts residuals on a scale that is more comparable across observations with different prediction uncertainty, but does not make them independent or remove model-estimation effects. Follow the conventions for the selected residual and retain its definition in the analysis record. Do not compare group means from one residual type with means from another as though they were the same quantity.

The group label must refer to the observation’s actual provenance. If a subject contributes observations to multiple studies, use the study recorded for each observation rather than assuming one study per subject. Similarly, an assay can change across samples or visits. Document how missing, unknown, or multiple labels are handled. A plot with an unclear grouping rule cannot support a reliable interpretation.

Before interpreting the displays, check that the residual is defined on the intended scale, the prediction and observation correspond to the same record, and group labels have been verified. Under an adequate model, standardized residuals should generally be centered near zero without a systematic group-specific pattern, subject to sampling variation, dependence, and the residual’s reference behavior. This is a diagnostic expectation, not a guarantee that every group average equals zero.

What to Look for in the Displays

A useful first display facets residual-versus-prediction or residual-versus-time plots by group, with a common vertical scale and a horizontal reference at zero. A complementary group summary can show the number of observations and a measure of center and spread. Showing the observations matters: a mean alone can hide asymmetry, outliers, or a time-dependent pattern within a group. Keep axes comparable across panels; separate scales can make a group difference look larger or smaller than it is.

• Location shifts: one group is predominantly above or below zero. Check whether this persists across the range of predictions or time, rather than being driven by a narrow region.
• Different spread: a group has a wider or narrower residual distribution. Check group size and the prediction-variance scaling before interpreting this as a difference in residual error.
• Shape or trend differences: curvature or a time-related sequence appears in one group but not others. The group label may be associated with a specific design or data range that should be examined.
• Isolated extremes: a few observations dominate the display. Verify the records and assess whether the pattern is broadly shared before treating it as a group-level feature.

Unequal group sizes deserve particular attention. A small group can have a volatile mean, while a large group can dominate pooled summaries. Observations within a subject are also not generally independent. Consequently, the number of plotted observations is not necessarily the number of independent pieces of information about a study or site difference.

Distinguishing Study, Site, and Assay Explanations

The labels can be linked. A site may enroll subjects in only one study, or a particular assay may be used only at certain sites. If a residual shift occurs in one study, one cannot attribute it separately to study protocol, participating sites, or assay if these factors never vary independently in the data. Cross-tabulate the labels and examine overlap before assigning an explanation. When the data contain adequate overlap, compare residual patterns within combinations of study, site, and assay; sparse combinations should be identified rather than interpreted as reliable contrasts.

Also check whether the group label is associated with the model’s input range. For example, one assay might be used primarily for low concentrations, or one study might sample at different times. A group-level residual shift could then reflect a prediction problem concentrated in that range rather than a uniform group difference. Faceting by group while retaining a common residual-versus-prediction or residual-versus-time axis helps expose this distinction. This is complementary to examining residuals across dose levels and regimens or other covariate groups: the grouping variable here is data provenance or measurement source.

Why Group Means Are Only a Screening Tool

A short calculation explains why group averages are useful but not definitive. Suppose, for illustration, a group contains \(m\) independent standardized residuals with mean zero and variance one. Then the group average has expected value zero and variance \(1/m\):

The variance result follows from independence, which makes all covariance terms zero. In NLME data, residuals from the same subject may be dependent, and standardization and parameter estimation can affect their behavior. In general, \[ \operatorname{Var}(\bar R_G)=\frac{1}{m^2}\left(\sum_{k=1}^{m}\operatorname{Var}(R_k)+2\sum_{k<\ell}\operatorname{Cov}(R_k,R_\ell)\right). \] Thus, a small group’s mean is usually less precise, but the simple \(1/m\) calculation is not a universal uncertainty formula for these diagnostics. Treat visual or numerical group summaries as prompts for investigation, not as formal significance tests.

Follow-Up, Alternatives, and Limitations

When a group pattern appears, use a staged review rather than immediately changing the model:

A pooled residual plot is compact and useful for an overall summary, but it can conceal opposing group patterns. A covariate-stratified plot addresses grouping by measured subject characteristics, while study-, site-, and assay-stratification focuses on design or measurement provenance. Model-based comparisons can account for covariates and clustering, but they require a defensible specification and enough independent information at the relevant grouping level. A graph alone does not supply that specification or justify a formal test.

These displays also have limits. With few observations, group-specific shapes and means are unstable; with many groups, chance variation can create striking-looking panels. A group difference may be real for the observed data but not generalize to other sites or studies. Residuals are themselves model-dependent, so the diagnostic cannot identify a unique cause. In practice, report the residual definition, group counts, relevant overlap, and the main pattern, and separate what the display shows from what remains a hypothesis.