Tutorials › Nonlinear Mixed-Effects Models (NLME) › Worked Diagnosis Of Dose-Dependent Residual Trends

Goodness-of-Fit Diagnostics and Their Pitfalls · Tutorial 571 of 797

Worked Diagnosis Of Dose-Dependent Residual Trends

Use dose-stratified residuals alongside prediction, time, and individual-fit displays to identify patterns that may reflect model structure, error variance, or sampling design.

Intermediate 12 min read

What You'll Learn

  • Define and interpret dose-stratified residual patterns in an NLME model.
  • Calculate standardized residuals from observed values, individual predictions, and model-implied error scales.
  • Separate a residual mean trend from a residual spread that changes with prediction or dose.
  • Use time-stratified comparisons and individual overlays to investigate apparent dose effects.
  • Recognize how unequal sampling can distort pooled residual summaries.
  • Identify limitations of residual plots and choose targeted follow-up diagnostics.

Why Examine Residuals by Dose?

A pooled residual plot can look acceptable even when a model fits some dose levels poorly. Conversely, a pooled trend across dose groups can arise because the groups were sampled at different times or with different numbers of observations. The diagnostic question is therefore not simply whether residuals vary with dose, but whether a dose-related pattern remains after considering the model prediction, time, and the distribution of observations across individuals.

This tutorial uses residuals as model-checking quantities, not as stand-alone tests of a dose effect. A visible pattern can suggest a missing structural feature, such as an incorrect dose scaling assumption, but it does not establish the cause. The pattern must be checked against the model’s error variance, sampling design, and individual profiles.

Definition: Let \(Y_{ij}\) be the observation for individual \(i\) at time \(t_{ij}\), after dose \(D_i\). A basic NLME observation model is \(Y_{ij}=f(t_{ij},D_i;\theta,\eta_i)+\epsilon_{ij}\), where \(f\) is the structural model, \(\theta\) denotes population parameters, \(\eta_i\) denotes individual random effects, and \(\epsilon_{ij}\) is residual error. A residual is the observed value minus a specified model prediction. A standardized residual divides that difference by an appropriate model-implied residual standard deviation.

The prediction used must be named. A population prediction, often denoted PRED, uses typical random effects, whereas an individual prediction, often denoted IPRED, conditions on estimates of the individual’s random effects. Residuals based on these predictions answer different diagnostic questions; they should not be silently mixed in a dose comparison. The examples below use IPRED and explicitly specified residual standard deviations.

The basic assumptions behind interpreting a residual mean pattern are that the prediction targets the relevant conditional mean and that the residual error has conditional mean zero. For a standardized residual, the scale used for division must also be appropriate to the assumed residual variance. These conditions are approximate in practice because parameters and individual effects are estimated from the same data. Thus, residual plots are diagnostic evidence, not independent observations or formal hypothesis tests.

What the Pattern Can Mean

If the conditional mean is correctly specified, then, at a given time, dose, and individual effect, the expected difference between observation and prediction is zero. Dividing that difference by a positive residual standard deviation preserves a zero expectation. A systematic positive or negative residual mean in a dose stratum can therefore signal a mean-model problem, provided time and sampling composition do not explain it.

$$ E\!\left[Y_{ij}-f(t_{ij},D_i;\theta,\eta_i)\mid t_{ij},D_i,\eta_i\right]=0 $$

A variance problem has a different primary signature: residuals remain centered near zero, but their spread changes with prediction or dose. For example, an additive error model assumes a roughly constant residual standard deviation on the observation scale. A proportional error model allows the standard deviation to increase with the prediction. If the proportional component is omitted, high-prediction observations may have much larger residuals in raw units even when the mean model is adequate.

Dose and time are often related in the data display because concentration changes over time and dose changes the concentration scale. Dose strata should therefore be compared at similar times or phases of the profile whenever the sampling design permits. Individual overlays add another check: a pattern repeated within many individuals is more persuasive evidence of systematic misspecification than a pooled pattern driven by a few heavily sampled individuals.

Worked Example: A Residual Mean That Increases Across Dose Groups

Suppose a hypothetical study has one observation per individual in each of three dose groups. For each observation, a standardized residual is calculated as \(r=(Y-\mathrm{IPRED})/s\), where \(s\) is the model-implied residual standard deviation. The residuals are:

Dose (mg)Standardized residualsMean residual
50-0.3, 0.2, 0.1, -0.1-0.025
1000.1, 0.4, 0.3, 0.20.250
2000.7, 0.9, 0.5, 0.90.750

At 50 mg, the sum is \(-0.3+0.2+0.1-0.1=-0.1\), so the mean is \(-0.1/4=-0.025\). At 100 mg, the sum is \(1.0\), giving \(1.0/4=0.250\). At 200 mg, the sum is \(3.0\), giving \(3.0/4=0.750\).

The group means rise with dose, while the residuals are mostly positive in the highest-dose group. In this hypothetical diagnostic, the model tends to underpredict the observations at 200 mg. Because each group has four observations, the overall observation-weighted mean is \((-0.1+1.0+3.0)/12=0.325\). These summaries identify a pattern; they do not prove that dose is its cause. Before proposing a structural change, inspect residuals against time within each dose group and examine individual overlays. A high-dose group sampled mainly at a different profile phase could produce a similar pooled pattern.

Separate Mean Misfit from Variance Misfit

A useful first distinction is whether dose strata differ in residual center, residual spread, or both. The residual-versus-prediction display helps because dose and prediction may be strongly related: if spread grows with prediction, an apparent dose-specific spread may simply reflect an inappropriate error model. Conversely, a persistent shift in the residual center across dose at comparable times is more consistent with a mean-model concern than with variance misspecification alone.

Worked Example: Larger Raw Residuals Without a Dose-Related Mean Shift

Consider hypothetical observations with predictions of 1 mg/L at low dose and 5 mg/L at high dose. Their observed-minus-IPRED residuals are \(-0.1\) and \(0.1\) mg/L at low dose, and \(-1\) and \(1\) mg/L at high dose. Each group is centered at zero, but the high-dose residuals have much greater spread in raw units.

Under an additive error model with residual standard deviation \(0.2\) mg/L at all predictions, the standardized residuals at low dose are \(-0.1/0.2=-0.5\) and \(0.1/0.2=0.5\). At high dose they are \(-1/0.2=-5\) and \(1/0.2=5\). The high-dose values would be unusually large relative to this additive-error assumption.

Now suppose the intended error model is proportional, with residual standard deviation \(s=0.2\times\mathrm{IPRED}\). At prediction 1 mg/L, \(s=0.2\) mg/L; at prediction 5 mg/L, \(s=1.0\) mg/L. The standardized residuals are then \(-0.5,0.5\) at low dose and \(-1/1=-1,1/1=1\) at high dose. The centers remain zero, and the apparent difference in raw spread is much less pronounced on the proportional scale.

This comparison suggests checking the error model rather than immediately adding a dose effect to the structural mean. The exact conclusion depends on the assumed error distribution and the full data, but the arithmetic shows why residual scale matters.

Account for Time and Unequal Sampling

A dose-stratified summary weights observations, not individuals, unless it is explicitly constructed otherwise. If one dose group contributes many more observations, or if its observations occur at different times, its pooled residual summary can be dominated by that sampling pattern. This is especially important with repeated measurements: observations from the same individual are not independent pieces of evidence.

Worked Example: An Apparent Dose Trend Created by Unequal Time Sampling

Suppose a hypothetical model has the same time-related residual pattern in both dose groups: early observations have standardized residual \(+1\), and late observations have residual \(-1\). The low-dose group contributes 8 early and 2 late observations. The high-dose group contributes 2 early and 8 late observations.

The low-dose observation-weighted mean is \((8\times1+2\times(-1))/10=(8-2)/10=0.6\). The high-dose mean is \((2\times1+8\times(-1))/10=(2-8)/10=-0.6\). A pooled comparison therefore appears to show a strong decline in residuals as dose increases.

But the time-specific means are identical across doses: both groups have mean \(+1\) among early observations and \(-1\) among late observations. If each time phase is given equal weight within each dose, each dose mean is \((1+(-1))/2=0\). In this constructed example, the apparent dose pattern is entirely due to different proportions of early and late samples, not a dose-specific residual pattern.

A practical follow-up is to compare dose groups within overlapping time ranges and inspect individual profiles. If there is little overlap in time between dose groups, the data may not separate a dose pattern from a time pattern reliably; a plot cannot recover information that the design did not collect.

A Focused Diagnostic Sequence

Use the following sequence to avoid attributing every dose-stratified pattern to the structural model. The sequence is a way to organize evidence, not an automatic decision rule.

1
Specify the residual and prediction.
Record whether the display uses PRED or IPRED, raw or standardized residuals, and which residual variance is used. Comparisons are not interpretable if scales change silently across dose groups.
2
Compare centers and spreads.
Ask separately whether residual means shift with dose and whether their spread changes. A center shift points toward a possible mean-model problem; a spread change with centered residuals points toward an error-variance issue.
3
Condition the comparison on time and prediction.
Look for dose patterns within comparable profile phases and inspect residuals against prediction. This helps distinguish a dose association from a time-composition or scale effect.
4
Inspect individual overlays and sampling.
Check whether many individuals show the same discrepancy, or whether a few people with dense sampling dominate the pooled display. Note dose groups with little overlap in time or sparse observations.
5
Form a targeted model check.
Only after these checks, identify a plausible structural or residual-error feature to investigate. Refit or compare candidate models on matched diagnostic scales so that visual differences are not caused by changed axes or residual definitions.

For example, a high-dose positive residual center that persists at comparable times across many individual overlays is a reason to revisit the mean structure. Depending on the pharmacology and design, a dose-related change in bioavailability or a departure from assumed dose proportionality might be plausible. A high-dose spread increase with no corresponding center shift instead motivates scrutiny of the error variance. These are hypotheses to evaluate, not conclusions supplied by the plot itself.

Limits, Alternatives, and Common Pitfalls

A dose-stratified residual plot is most useful when dose groups have adequate observations and overlap in time. With sparse data, individual residuals are noisy, and group means can be unstable. Individual predictions can also make residuals look less variable because individual effects are estimated from the same observations; this is one reason to interpret IPRED-based residuals alongside population-level displays rather than treating them as a complete assessment.

The main alternative is not a single competing plot, but a set of complementary views: residuals against prediction to assess scale, residuals against time to assess profile-phase patterns, and individual overlays to assess whether discrepancies recur within people. A dose-stratified plot summarizes dose groups directly; the complementary plots help explain what that summary may be conflating. Formal model comparison can quantify a specified candidate change, but a dose-stratified residual pattern alone does not define the candidate model or validate its assumptions.

  • Pitfall: “The high-dose residuals are larger, so the mean model is wrong.” Larger raw residuals may reflect a residual standard deviation that increases with prediction. Check centers and spreads separately, using a scale appropriate to the error model.
  • Pitfall: “The dose-group means differ, so dose caused the pattern.” Unequal time sampling or a few heavily sampled individuals can create the difference. Compare within time ranges and inspect individual overlays.
  • Pitfall: “A flat pooled plot proves the fit is adequate.” Opposing dose-specific or time-specific patterns can cancel when pooled. Stratification is useful precisely because pooled summaries can hide structure.
Key takeaway: Diagnose dose-dependent residual trends by separating residual center from residual spread, then checking prediction scale, time composition, and individual profiles. A pattern is a clue about model adequacy; its cause must be assessed against the data design and competing explanations.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

  1. At one dose, standardized residuals have mean near zero but much greater spread at high predictions than at low predictions. Which feature should be checked first: the mean structure or the residual variance? Why?
  2. A high-dose group has a positive mean residual, but it was sampled mostly early, while a low-dose group was sampled mostly late. What comparison would help assess whether dose itself is associated with the pattern?
  3. Why can a pooled residual summary be dominated by a small number of individuals with many observations?
  4. In a hypothetical dose group, four standardized residuals are \(-0.2, 0.1, 0.3,\) and \(0.2\). Calculate their mean and state what it indicates, and what it does not establish.

Solutions. (1) Check the residual variance first: a centered pattern with spread increasing as prediction rises is consistent with a scale mismatch, although other explanations remain possible. (2) Compare dose groups within overlapping time ranges, and inspect residuals against time and individual overlays. If time overlap is absent, dose and time effects may not be distinguishable from these data. (3) An observation-weighted summary gives more influence to individuals contributing more observations; repeated observations from one person do not represent as many independent individuals. (4) The sum is \(-0.2+0.1+0.3+0.2=0.4\), so the mean is \(0.4/4=0.1\), a small positive mean on the standardized scale. It is a descriptive summary of these four residuals, not proof of a systematic dose effect or of adequate fit.