Unequal Sampling And Subject-Weighted Summaries

Why Unequal Sampling Changes a Residual Summary

A pooled residual plot gives one point to every observation. When subjects contribute very different numbers of observations, a subject with many measurements contributes many points, while a subject with few measurements contributes only a few. The resulting display can be useful, but it does not automatically describe the typical subject equally well.

This distinction matters in nonlinear mixed-effects diagnostics because observation schedules may differ across individuals. A subject may have more observations because of a planned sampling schedule, missed visits in other subjects, or a design that samples some profiles more densely. The number of plotted points is therefore not necessarily the number of subjects represented equally.

The question is not whether observation weighting is always wrong or subject weighting is always right. The question is which population the summary is intended to describe: a randomly selected observation from the collected data, or a randomly selected subject whose observations are summarized first.

Residuals must be on a comparable scale for a pooled summary to be interpretable. For instance, the examples below use standardized residuals, which are dimensionless. If residuals are on a raw concentration scale, the summaries retain concentration units, and comparisons across settings with substantially different scales require care. The weighting calculations themselves do not require residuals to be independent. However, within-subject dependence means that a large number of observations should not be mistaken for an equally large number of independent subjects.

Two Ways to Average the Same Residuals

For subject \(i\), define the subject-specific mean residual as \(\bar r_i=(1/n_i)\sum_{j=1}^{n_i}r_{ij}\). The pooled mean gives each observation equal weight, whereas the equal-subject mean averages the \(m\) subject-specific means.

The pooled mean is a weighted average of subject means, with weights \(n_i/N\). A subject contributing 20% of all observations receives 20% of the pooled mean's weight. In the subject-weighted mean, every subject receives weight \(1/m\), regardless of their number of observations.

The same distinction applies to other summaries. For example, the observation-weighted mean absolute residual averages \(|r_{ij}|\) over all observations. The subject-weighted mean absolute residual first averages \(|r_{ij}|\) within each subject and then averages those subject-specific values. These answer different questions: average absolute deviation per observation versus average within-subject absolute deviation for a subject chosen equally from the sampled subjects.

Absolute Residuals and Residual Spread

A mean residual can be close to zero because positive and negative residuals cancel, even when residuals are often far from zero. Absolute residuals or squared residuals can summarize magnitude without that cancellation. Their observation-weighted and subject-weighted versions still differ in exactly the same way: compute across all observations for the former, or compute within each subject and average across subjects for the latter.

For squared residuals, one useful subject-level measure is the root mean square (RMS) residual for each subject, \(\sqrt{(1/n_i)\sum_j r_{ij}^2}\). Averaging these subject-specific RMS values gives every subject equal weight. A different subject-weighted summary is the square root of the average subject-specific mean square. These are not generally equal, so the definition of the chosen summary should be reported.

A pooled variance also reflects both variation within subjects and differences between subject means. This can be shown by writing each residual's deviation from the pooled mean as \(r_{ij}-\bar r_{\mathrm{obs}}=(r_{ij}-\bar r_i)+(\bar r_i-\bar r_{\mathrm{obs}})\). On squaring and summing, the cross-product for subject \(i\) is zero because \(\sum_j(r_{ij}-\bar r_i)=0\). Therefore:

The first term is within-subject variation. The second is variation among subject means, with each subject's contribution multiplied by its observation count. This identity explains why a pooled spread is not simply a typical within-subject spread: it also incorporates differences in subject means, and subjects with more observations have greater weight in that between-subject component.

Choose the Weighting to Match the Diagnostic Target

If the diagnostic question concerns a randomly selected recorded observation under the actual sampling scheme, observation weighting is natural. If it concerns the typical sampled subject, summarize within subjects first and weight subjects equally. If the intended target is a different population, neither unweighted approach necessarily matches it; a sampling or analysis design may require prespecified weights. These choices concern the summary's target, not a claim that one weighting reveals the model's truth.

Consider a third hypothetical data set: subject P contributes four residuals, all \(0.5\); subject Q contributes one residual, \(-1\); and subject R contributes one residual, \(-1\). The pooled mean is \([4(0.5)-1-1]/6=0\), whereas the equal-subject mean is \((0.5-1-1)/3=-0.5\). The pooled mean absolute residual is \([4(0.5)+1+1]/6=0.667\); the equal-subject mean absolute residual is \((0.5+1+1)/3=0.833\). Here dense sampling of P changes both the direction summary and the average magnitude relative to an equal-subject summary.

Neither calculation establishes that the model is biased or mis-specified. It identifies how the observed residual summary depends on sampling intensity. A useful diagnostic report can show a pooled display for observation-level patterns alongside subject-level summaries or per-subject displays. When numbers of observations vary greatly, report the distribution of \(n_i\) as context and check whether the main conclusion changes under subject weighting.

Diagnostics, Limitations, and Common Mistakes

Unequal sampling is a warning to inspect weighting, not by itself proof of a problem. Check whether a small number of subjects supply a large fraction of observations; whether high-count subjects have systematically different residual means or magnitudes; and whether the sampling schedule is related to covariates, response history, or study procedures relevant to interpretation. A pooled pattern dominated by a few subjects should be described as such rather than presented as if every subject contributed equally.

A common mistake is to say, “There are many residual points, so the pooled mean represents the typical subject precisely.” The correction is that the pooled mean targets an observation-weighted average, and multiple observations from one subject do not create additional subjects. Another mistake is to conclude that subject weighting is always superior. It is not: if the intended target is the recorded-observation distribution, equal-subject weighting changes that target.

Subject-level summaries also have limitations. A mean or RMS can hide time-ordered structure, changing residual spread, or a brief but important pattern within an individual. Subjects with only a few observations may have unstable subject-specific summaries. Consequently, an equal-subject aggregate should complement, not replace, inspection of individual profiles and residual patterns. Unequal sampling can also arise through missingness or design choices; weighting alone does not correct selection effects or establish why observations are missing.