Why the Order of Forecast Errors Matters
A single concentration that differs from its forecast may reflect ordinary assay variation, an unusual observation, or a weakness in the model. A sequence of discrepancies can provide more information. If residuals repeatedly have the same sign, or change direction at a particular stage of a dosing interval, the timing and shape of the pattern may be inconsistent with the model’s account of absorption, distribution, elimination, or dose history.
Residual patterns are diagnostic clues, not direct measurements of the mechanism that caused them. Several different problems can produce similar discrepancies, particularly when sampling is sparse. Interpretation therefore requires the forecast uncertainty, the timing of samples relative to doses, the recorded dosing history, and the model’s assumptions.
A useful diagnostic sequence uses pre-observation, or one-step-ahead, forecasts: forecast the next concentration using the dosing record and information available before that concentration is observed, then calculate its residual. If the new concentration is first used to update the patient-specific estimate, a forecast made afterward is not an independent check of that observation. Sequential updating is useful for prediction, but diagnostic residuals should be associated with forecasts that did not use the value being checked.
Under a correctly specified conditional forecast, residuals should be centered near zero over repeated comparable predictions. Standardized residuals should generally be of a scale consistent with their predictive distributions. These expectations do not require every residual to be small or signs to alternate. Nor does a correctly specified model guarantee independent residuals in every practical dataset: shared uncertainty, repeated measurements, and the forecasting procedure can induce dependence. The diagnostic question is whether the observed ordering is more structured than the model and design would reasonably predict.
Reading a Time-Ordered Pattern
Plot residuals, or standardized predictive residuals, against time since dose and connect observations within a patient in chronological order. Mark dose times, dose amounts, and relevant changes in administration or sampling. The sign convention matters: with \(r_j=C_j-\widehat{C}_j\), negative residuals mean the model forecast too high, while positive residuals mean it forecast too low.
Different mechanisms can suggest different patterns, but the patterns are not unique fingerprints:
- Absorption: after an oral dose, a model that predicts concentrations rising too quickly may forecast too high early (negative residuals), then too low later around or after the observed peak (positive residuals). A shifted or blunted peak can also suggest an absorption-rate or absorption-history problem.
- Distribution: systematic discrepancies concentrated in the early post-dose period, especially a mismatch in the initial decline or curvature, can be consistent with an inadequate description of distribution. The inference is stronger when the samples cover that phase; late samples alone rarely identify it.
- Elimination: after the absorption and distribution phases, residuals that progressively become negative over the interval may indicate that forecasts decline too slowly; progressively positive residuals may indicate they decline too quickly. This interpretation requires appropriate timing and a sufficiently stable dose history.
- Dose history or administration: a sustained discrepancy beginning after a missed, delayed, extra, or incorrectly recorded dose may reflect an incorrect input history rather than a wrong disposition model. An unrecorded dose can create positive residuals; a recorded dose that was not taken can create negative residuals.
A quick justification explains why order is informative. If the forecast distribution at time \(j\), conditional on information available before that observation, has mean \(\widehat{C}_j\) and standard deviation \(s_j\), then \(z_j=(C_j-\widehat{C}_j)/s_j\) is the discrepancy measured in predictive standard-deviation units. Under a well-calibrated forecast, large positive and negative values should not persist systematically at a particular phase without explanation. For example, several early negative values followed by later positive values suggest a phase-specific shape mismatch more than a single isolated error does. This is a diagnostic argument, not a formal test: dependence and small numbers of observations limit how much can be concluded from a visual pattern alone.
Worked Example: A Possible Absorption-Phase Mismatch
Suppose a hypothetical patient has three concentrations after a documented oral dose. Before seeing each concentration, the model forecasts the value and its predictive standard deviation. The values below are illustrative, in mg/L.
| Time after dose (h) | Observed \(C_j\) (mg/L) | Forecast \(\widehat{C}_j\) (mg/L) | \(s_j\) (mg/L) |
|---|---|---|---|
| 1 | 3.0 | 4.0 | 0.50 |
| 2 | 7.0 | 6.0 | 0.50 |
| 4 | 8.0 | 9.0 | 0.50 |
Calculate each residual and standardize it using its predictive standard deviation:
At 1 h, \(r_1=3.0-4.0=-1.0\) mg/L and \(z_1=-1.0/0.50=-2.0\). At 2 h, \(r_2=7.0-6.0=1.0\) mg/L and \(z_2=1.0/0.50=2.0\). At 4 h, \(r_3=8.0-9.0=-1.0\) mg/L and \(z_3=-1.0/0.50=-2.0\).
The signs alternate, so this small sequence does not show a simple early-negative-then-late-positive pattern. It does show discrepancies of two predictive standard deviations at each time under the stated uncertainty. The appropriate conclusion is not that absorption is proven wrong: with only three observations, the pattern could reflect chance, underestimated predictive uncertainty, or another model or data problem. Check the actual sampling and dose times, assay information, and whether the forecast uncertainty is appropriate before assigning a mechanism.
Worked Patterns: Distribution, Elimination, and Dose History
Worked Example: Early Decline Consistent With a Distribution Mismatch
Consider a hypothetical intravenous dose and samples during the early decline. A one-compartment forecast is being checked against concentrations measured before any of those observations were used to update the patient estimate. Assume the dose and sample times are verified and the stated predictive standard deviations include assay and parameter uncertainty.
| Time after dose (h) | Observed (mg/L) | Forecast (mg/L) | Predictive SD (mg/L) |
|---|---|---|---|
| 0.5 | 18.0 | 19.0 | 1.0 |
| 1 | 10.0 | 13.0 | 1.0 |
| 2 | 5.0 | 8.0 | 1.0 |
The residuals are \(18.0-19.0=-1.0\) mg/L, \(10.0-13.0=-3.0\) mg/L, and \(5.0-8.0=-3.0\) mg/L. Dividing by the respective predictive standard deviations gives \(z=-1.0\), \(-3.0\), and \(-3.0\). The increasingly negative early residuals indicate that the forecast declines less rapidly than the observations over this window.
Because these samples are concentrated in the early post-dose phase, a distribution description that is too simple is one possible explanation. This is not a unique diagnosis: incorrect dose or sample times, an unmodeled administration feature, or an unsuitable uncertainty model could also matter. Later samples would help determine whether the discrepancy is mainly an early-phase shape problem or persists into the terminal decline.
Worked Example: A Late-Interval Elimination Pattern
For a hypothetical patient at a stable recorded regimen, suppose late-interval forecasts are made before the corresponding concentrations are observed. The sampling times are verified to be after the main absorption and distribution phases.
| Time after dose (h) | Observed (mg/L) | Forecast (mg/L) | Predictive SD (mg/L) |
|---|---|---|---|
| 8 | 7.2 | 7.0 | 0.40 |
| 12 | 4.7 | 5.5 | 0.40 |
| 16 | 2.8 | 4.3 | 0.40 |
The residuals are \(7.2-7.0=0.2\) mg/L, \(4.7-5.5=-0.8\) mg/L, and \(2.8-4.3=-1.5\) mg/L. The standardized residuals are \(0.2/0.40=0.5\), \(-0.8/0.40=-2.0\), and \(-1.5/0.40=-3.75\). The increasingly negative late residuals mean the model forecasts a slower decline, and hence higher late concentrations, than were observed.
With the stated timing and stable regimen, faster elimination than the model represents is a plausible explanation. Before changing a clearance estimate, verify the dose record, sample times, and uncertainty assumptions. Three values are a clue, not a definitive estimate of an individual’s elimination rate.
Worked Example: A Discrepancy After a Missed Dose
A hypothetical patient is forecast to receive a dose at 8 h, but a later record check shows that the dose was missed. At 12 h, the model forecast, which assumed the dose was taken, is 6.0 mg/L; the observed concentration is 3.5 mg/L. The predictive standard deviation under that forecast is 0.50 mg/L.
The residual is \(3.5-6.0=-2.5\) mg/L, and the standardized residual is \(-2.5/0.50=-5.0\). This large negative discrepancy begins after the dose whose administration was incorrectly assumed. The dose-history error offers a direct explanation for the forecast being too high. Reconstructing the forecast with the missed dose represented appropriately is a useful check; the result should not be interpreted as evidence of unusually rapid elimination without considering this input error.
Diagnostics, Alternatives, and Limitations
A practical review combines a time-ordered plot with checks of the data and the predictive model. Examine residuals against time since dose, dose number, and calendar time where relevant. Keep observed concentrations, forecasts, and predictive uncertainty available: a residual of 1 mg/L has different implications when \(s_j=0.2\) mg/L than when \(s_j=2\) mg/L. Large standardized residuals can indicate an unusual observation, but a run of moderate values in one direction can also reveal a systematic mismatch.
Follow a suspected pattern with targeted checks. Confirm dose amounts and administration times, sample collection times, formulation and route, and whether the observation was obtained during absorption, distribution, or the late decline. Review assay and residual-error assumptions. If the data support it, compare forecasts under scientifically plausible alternative structures or input histories. A change in fit is not by itself proof that the alternative mechanism is true; it should also be plausible and supported by the sampling design.
Time-ordered standardized residual plots are more informative than simply counting observations above and below a forecast. A formal posterior predictive check can compare a chosen pattern statistic—such as a run of late negative residuals—with patterns generated under the model. Such a check asks whether the pattern is unusual under that model; it does not identify the true mechanism automatically. Treating each residual as an independent normal value and applying a simple test may be inappropriate when predictive errors are dependent or when the diagnostic pattern was selected after inspecting the data.
| Approach | What it helps answer | Main limitation |
|---|---|---|
| Time-ordered residual plot | Does discrepancy align with a dose phase or emerge over time? | Visual patterns can be ambiguous, especially with few samples. |
| Isolated outlier review | Is one observation unusually discrepant relative to its uncertainty? | Can miss a sequence of modest, same-direction errors. |
| Posterior predictive check | Is a selected pattern unusual under the forecast model? | Does not by itself identify which structural explanation is correct. |
| Alternative model or dose-history forecast | Does a plausible change address the observed pattern? | Can overfit sparse data and may not distinguish competing explanations. |
Do not confuse this diagnostic use with fitting a model to all available concentrations and then inspecting residuals from that fit: the fitted values may be pulled toward the observations, concealing forecast errors. Also, an isolated large residual is not automatically a structural problem. The earlier material on robust forecasts under concentration outliers addresses how unusual concentrations can affect forecasts; here, the key question is whether discrepancies show a coherent relationship with time and dosing.
Common Mistake and Key Takeaway
Wrong reasoning: “The late residuals are negative, so the patient’s clearance must be too high.” This reverses the direction implied by the residuals. Negative residuals mean observed concentrations are below the forecasts. In a late elimination phase, a model that predicts concentrations that remain too high may be declining too slowly, which is consistent with faster elimination than predicted—but a missed dose, an incorrect dose time, or underestimated uncertainty can produce the same sign.
Correction: State the residual convention, locate the pattern in the dosing interval, and check dose and sample history before proposing a mechanism. A residual sequence supports a hypothesis only to the extent that its timing, data quality, and predictive uncertainty support that interpretation.
Check Your Understanding
Answer each question using the convention \(r_j=C_j-\widehat{C}_j\). Brief solutions follow the questions.
- An observation is 5.0 mg/L, its forecast is 6.0 mg/L, and its predictive standard deviation is 0.50 mg/L. Find \(r_j\) and \(z_j\), and interpret their signs.
- After an oral dose, residuals are negative early and positive near the observed peak. What kind of mismatch might this suggest, and why is the conclusion not definitive?
- Late-interval standardized residuals become increasingly negative. What does this say about the forecasts, and what should be checked before attributing the pattern to elimination?
- Why should a diagnostic forecast be made before the concentration being checked is used to update the patient-specific estimate?
- A large negative residual begins after a dose recorded as taken, but the patient later reports missing it. Which explanation should be investigated first?
Solutions: (1) \(r_j=5.0-6.0=-1.0\) mg/L and \(z_j=-1.0/0.50=-2.0\); the observation is 1.0 mg/L, or two predictive standard deviations, below the forecast. (2) A model predicting absorption too quickly is one possibility: it can forecast too high early and too low later. Sparse samples, timing errors, or another structural problem can create similar signs. (3) Forecasts are increasingly high relative to observations. Check dose and sample times, adherence, and predictive uncertainty before concluding that elimination is faster than represented. (4) Using the observation to update the estimate can pull the subsequent fitted forecast toward that same observation, weakening its value as an independent diagnostic check. (5) Investigate the dose history first: an assumed dose that was not taken can explain an overly high forecast and negative residual without requiring a change in elimination.