Residual and Predictive Checks for TMDD Fits

Why Check TMDD Fits Beyond the Objective Function?

A fitted TMDD model can converge and still fail to describe important features of the data. A single overall measure of fit can hide a concentration bias confined to high doses, or a target prediction that misses recovery after dosing. Residual and predictive checks make these discrepancies visible in relation to time, dose, and the measured analyte.

The checks here are diagnostic, not automatic tests that identify the true mechanism. They ask whether the fitted model reproduces the observed patterns within the uncertainty specified by its observation and variability models. The earlier material on sampling designs is relevant: these checks are most informative when the observations cover the phases and dose range needed to distinguish TMDD processes.

Definition: For observation \(j\), a residual is the observed value minus its model prediction on the scale used for fitting. If \(DV_j\) is the observation and \(PRED_j\) is the corresponding prediction, the raw residual is \(e_j=DV_j-PRED_j\). If the observation model gives a residual standard deviation \(SD_j\), the standardized residual is \(r_j=e_j/SD_j\). For log-scale fitting, both the observation and prediction in these expressions are on the log scale.

A residual is meaningful only when observation and prediction refer to the same quantity: for example, measured free drug compared with predicted free drug, or measured target compared with the prediction corresponding to that target assay. Do not compare a total-drug observation with a free-drug prediction. Also inspect concentration and target residuals separately: a model can fit one outcome while systematically missing the other.

Residual Patterns and Their Interpretation

Under a well-calibrated observation model, residuals should be centered near zero, with spread compatible with the assumed error, and should not show systematic structure against time, dose, predicted concentration, or target level. These are expectations rather than exact rules: fitted residuals are not generally independent standard normal observations, because parameters were estimated from the same data and repeated measurements within a subject are related.

For positive concentrations with proportional error, residuals are often easier to interpret on the log scale. A positive log residual means the observation exceeds its prediction by a multiplicative factor; a negative one means it is below. When the model assumes additive error, residuals on the original scale may be appropriate. The scale must match the fitted observation model, rather than being chosen after seeing which plot looks better.

For TMDD, plot residuals against time after dose and dose level, and, where useful, against predicted concentration or target. Repeated positive residuals during a concentration phase indicate underprediction in that region; repeated negative residuals indicate overprediction. A pattern that changes with dose may be consistent with an inaccurate description of saturation or dose-dependent disposition. A time-dependent pattern in target residuals may indicate that the modeled target turnover or recovery trajectory is not capturing the data. These clues do not uniquely identify the faulty process.

Worked Example: Reading Standardized Concentration Residuals

Suppose a hypothetical fit uses additive error for concentration in mg/L. At three selected observations, the model's predictions and residual standard deviations are as follows.

Time after dose	Observed \(DV\) (mg/L)	Predicted \(PRED\) (mg/L)	\(SD\) (mg/L)
2 h	8.4	8.0	0.8
12 h	4.2	5.0	0.8
48 h	1.8	1.2	0.6

At 2 h, \(e=8.4-8.0=0.4\) mg/L, so \(r=0.4/0.8=0.50\). At 12 h, \(e=4.2-5.0=-0.8\) mg/L and \(r=-0.8/0.8=-1.00\). At 48 h, \(e=1.8-1.2=0.6\) mg/L and \(r=0.6/0.6=1.00\).

The late observation is above prediction, while the 12 h observation is below it. One subject and three residuals are not enough to establish a systematic pattern. If many subjects showed negative residuals around 12 h followed by positive residuals late in the interval, the fitted time course would warrant investigation. The residuals alone would not establish whether the cause was target turnover, complex handling, or another model component.

Prediction-Corrected Visual Predictive Checks

A visual predictive check (VPC) compares observed data summaries with summaries from replicated data generated under the fitted model. For each simulation replicate, the model generates observations with the fitted structural predictions and the specified variability and observation error. Within time bins, one can compare observed percentiles, such as the median and the 10th and 90th percentiles, with the corresponding simulated percentile distributions. A predictive interval for a percentile shows the range in which that percentile would fall across simulated data sets.

In dose-escalation data, pooling dose levels can make a VPC difficult to interpret because the distribution of predicted concentrations may differ substantially by dose. A prediction-corrected VPC rescales each observation relative to its prediction within a time bin, often separately within dose strata. Let \(PRED_j\) be the prediction for observation \(j\), and let \(PRED_{\mathrm{ref}}\) be the median prediction in the relevant bin and stratum. The prediction-corrected observation is

pcDV_j=DV_j\frac{PRED_{\mathrm{ref}}}{PRED_j}.

The correction has a clear justification under a multiplicative prediction relationship: it puts observations at differing prediction levels onto the reference prediction scale. If an observation equals its prediction, then \(pcDV_j=PRED_{\mathrm{ref}}\). If the observation is proportionally above or below its prediction, that relative discrepancy is retained. The procedure is most interpretable when the proportional rescaling is reasonable within the bin; it does not make a fundamentally inappropriate model correct.

A corrected VPC should be read alongside uncorrected and dose-stratified displays. Correction can obscure clinically important dose dependence if applied across doses where the model's nonlinear relationship is itself under evaluation. In practice, use sufficiently populated time bins, show observed and simulated percentiles with uncertainty, and avoid treating a few points outside a predictive interval as decisive evidence.

Worked Example: Calculating and Interpreting Prediction Correction

Consider a hypothetical time bin within one dose stratum. Its median prediction is \(PRED_{\mathrm{ref}}=12.0\) mg/L. Two observations have predictions of 10.0 and 15.0 mg/L and observed concentrations of 11.0 and 18.0 mg/L.

For the first observation, \(pcDV=11.0(12.0/10.0)=11.0(1.20)=13.2\) mg/L. For the second, \(pcDV=18.0(12.0/15.0)=18.0(0.800)=14.4\) mg/L. The two corrected observations can now be compared on the reference prediction scale of 12.0 mg/L, while retaining their proportional deviations from their own predictions.

Suppose the observed median for this bin is 20.0 mg/L and the median of the model's simulated predictive distribution is 14.0 mg/L. The observed median is \(6.0/14.0=0.429\), or about 42.9%, greater than the prediction. Equivalently, the prediction is \(6.0/20.0=0.300\), or 30.0%, below the observed median. Those are different percentage comparisons because they use different denominators. A persistent separation of observed percentiles from simulated intervals would signal poor predictive calibration in this region; the single discrepancy does not identify its cause.

Dose-Stratified Predictions and Mechanistic Clues

Dose-stratified plots retain the dose context that can be blurred by pooling. For each dose group, compare observed concentrations and target measurements over time with model predictions or simulated predictive intervals. Inspect both the central trajectory and the spread: matching medians while repeatedly missing the lower or upper percentiles can point to an inadequate variability or observation model even when the typical prediction looks reasonable.

A practical diagnostic sequence is to check the measured quantity and scale first, then examine residuals over time and dose, then compare predictive distributions by dose. Ask whether the mismatch is mainly an early concentration feature, a late concentration feature, target recovery, or a pattern shared across outcomes. Only after locating the discrepancy should one consider which model assumptions might explain it.

Worked Example: A Dose-Stratified Pattern in Concentration and Target

In a hypothetical study, the model predicts concentration medians of 9.0 mg/L at a low dose and 15.0 mg/L at a high dose at a selected early time. The observed medians are 9.4 and 14.6 mg/L, respectively. At a late time, the predicted medians are 1.0 and 3.0 mg/L, while observed medians are 1.1 and 1.2 mg/L. At that late time, measured target recovery is also consistently slower than predicted in the high-dose group.

At the early time, the discrepancies are small: \(9.4-9.0=0.4\) mg/L at the low dose and \(14.6-15.0=-0.4\) mg/L at the high dose. At the late time, the low-dose difference is \(1.1-1.0=0.1\) mg/L, whereas the high-dose difference is \(1.2-3.0=-1.8\) mg/L. The high-dose model prediction is \(1.8/3.0=0.600\), or 60.0%, greater than the observed median; equivalently, the prediction is \(1.8/1.2=1.50\), or 150% above the observed median.

The late, high-dose overprediction accompanied by slower-than-predicted target recovery suggests a dose- and time-dependent mismatch, potentially involving saturation or target turnover. It does not prove either explanation: other structural or observation-model issues could produce similar patterns. The appropriate next step is to check the underlying observations and model predictions, and then evaluate plausible explanations using suitable diagnostics and evidence.

Assumptions, Failure Modes, and Alternatives

Residual and predictive checks rely on correct alignment between observations and predictions, an observation model appropriate to the measurement scale, and simulations that faithfully implement the fitted model and its variability. The interpretation of a VPC also depends on adequate simulation size and enough observations in the time and dose strata being examined. Sparse bins produce unstable percentiles and can make apparent mismatches misleading.

Residual trends hidden by pooling. A pooled plot may conceal opposite patterns at low and high doses. Inspect dose strata before drawing an overall conclusion.
Wrong scale or analyte. Residuals calculated on a scale inconsistent with the fitted error model, or against a different concentration quantity, are not interpretable as intended.
Overreading one excursion. Predictive intervals describe expected variation across replicated data sets, not a rule that every observation must fall inside them. Look for persistent patterns and uncertainty.
Correction mistaken for a cure. Prediction correction rescales observations; it cannot repair misspecified dose dependence, and may mask it if unlike dose levels are combined.
Mechanism inferred from a plot alone. A residual pattern can motivate investigation of saturation, turnover, or complex handling, but it does not uniquely identify a mechanism.

Compared with a residual plot, a VPC evaluates whether the full simulated distribution resembles the data, including variability and observation error. A dose-stratified prediction display preserves direct dose interpretation but may be visually noisy when groups are small. Prediction correction can improve comparability across different prediction levels, but sacrifices some direct interpretation on the original concentration scale. These views complement rather than replace one another.

Key takeaway: A credible TMDD fit should reproduce concentration and target patterns over time and dose, not merely yield a successful estimation result. Use residuals to locate structured discrepancies, VPCs to assess simulated distributions, and dose-stratified predictions to retain nonlinear dose context. Treat any proposed mechanism as a hypothesis requiring further evaluation.

Check Your Understanding

Answer each question, then compare with the solutions below.

An observed concentration is 7.2 mg/L, its prediction is 8.0 mg/L, and the residual standard deviation is 0.4 mg/L. Calculate the raw and standardized residuals and interpret their signs.
A prediction-correction reference value is 10 mg/L. For an observation of 6 mg/L with prediction 5 mg/L, calculate \(pcDV\).
Observed late-time concentrations are repeatedly below predictions only in the highest dose group, while early-time predictions fit. What can be concluded, and what cannot be concluded?
Why should prediction correction not automatically be applied across dose groups in a TMDD analysis?

Solutions. (1) \(e=7.2-8.0=-0.8\) mg/L and \(r=-0.8/0.4=-2.00\); the observation is below prediction by two residual standard deviations. (2) \(pcDV=6(10/5)=12\) mg/L. (3) The pattern indicates a dose- and time-specific predictive mismatch worth investigating, but does not by itself identify saturation, turnover, complex clearance, or another cause. (4) Different doses may have genuinely different nonlinear predictions; rescaling across them can hide the dose dependence being assessed, and the correction assumes a meaningful proportional relationship within the chosen bin and stratum.