Why Diagnose Residuals at the Interfaces?
An integrated evidence chain can match its overall predictions poorly even when some components are adequate, or appear to match overall while errors at different handoffs offset one another. A useful diagnostic question is therefore not just “How far off is the final prediction?” but “At which interface does a systematic difference first appear?”
The interfaces considered here connect PBPK-predicted exposure to population PK estimates, population PK exposure to an exposure-response (ER) function, and ER predictions to trial outcomes generated under a simulated trial design. A residual is a discrepancy between a quantity and its corresponding model prediction. Its direction and pattern across relevant conditions can suggest where to investigate. A residual alone does not establish a cause: data quality, uncertainty, shared inputs, and differences in how the compared quantities were defined all matter.
Residuals are meaningful only when the compared quantities refer to the same target: for example, the same exposure metric, time window, population, endpoint, and scale. The earlier material on matching exposure metrics is relevant at the first two handoffs. A difference between AUC over one interval and AUC over another is not, by itself, evidence of model error.
What a Pattern Can Tell You
If a model is adequately specified for the comparison being made, residuals should not show a systematic relationship with conditions that the model is meant to handle. In a simple calibrated setting, \(E(e_j)=0\). If the predictive variance is correctly specified, standardized residuals should also be on an appropriate scale: values much larger in magnitude than expected may indicate underrepresented uncertainty, an unusual observation, or model inadequacy. These are expectations over repeated comparable predictions, not guarantees for a small set of residuals.
The structure of a pattern matters more than one isolated value. Repeated positive exposure residuals under a particular formulation may point toward a formulation-specific issue; a monotone sequence across concentration or dose may suggest a missing dependence; a shift confined to one trial arm may prompt an audit of that arm’s implementation. These are diagnostic leads, not unique explanations.
At a handoff, the discrepancy may involve two uncertain quantities. If \(A\) and \(B\) are estimates of the same target, then
This follows by expanding \(\operatorname{Var}(A-B)=E[((A-EA)-(B-EB))^2]\): the two squared terms contribute their variances, and the cross-product contributes minus twice their covariance. Ignoring a nonzero covariance can overstate or understate the uncertainty of the difference. The earlier material on carrying parameter covariance across model handoffs is especially relevant when the components share data or parameters.
Worked Example: A Formulation-Patterned PBPK Exposure Residual
Suppose a PBPK model predicts steady-state \(C_{\max}\) for a hypothetical drug under two formulations. In a separate evaluation set, the observed and predicted values are: formulation A, 100 and 96 ng/mL; formulation B, 120 and 96 ng/mL. For this illustration, assume each prediction has predictive standard deviation 12 ng/mL, and treat the observed values as measured precisely relative to that uncertainty.
For formulation A, \(e_A=100-96=4\) ng/mL and \(r_A=4/12=0.333\). For formulation B, \(e_B=120-96=24\) ng/mL and \(r_B=24/12=2.00\). The prediction is below the observed value in both cases, with a substantially larger standardized residual for formulation B.
The pattern suggests checking whether the model represents formulation B’s absorption behavior and whether the measurements and exposure metric are comparable across formulations. Two hypothetical observations do not establish a formulation effect or prove that the PBPK model is the source: uncertainty in observations, study conditions, and the handoff definition still require review.
At the PBPK-to-population-PK interface, examine residuals across conditions the PBPK model is intended to represent, such as formulation, food condition, dose, or relevant physiology. Concentration-time residuals can reveal a timing or profile mismatch that a single AUC comparison hides. A consistent offset in all conditions may instead reflect a scale difference, a unit conversion, or a common bias. Check the data and metric definitions before modifying a mechanistic component.
Worked Example: Comparing Population PK Exposure Estimates With PBPK Predictions
For three hypothetical subjects, a PBPK handoff predicts AUC values of 20, 22, and 24 mg·h/L. Corresponding population PK estimates are 21, 25, and 29 mg·h/L. Suppose, for this illustration, the estimate and prediction errors are independent and their standard errors are 2.0 and 1.5 mg·h/L, respectively, for each subject.
The residuals, defined as population PK estimate minus PBPK prediction, are 1, 3, and 5 mg·h/L. Under the stated independence assumption, the standard error of each difference is \(SE_{\mathrm{diff}}=\sqrt{2.0^2+1.5^2}=\sqrt{6.25}=2.50\) mg·h/L. The standardized residuals are therefore \(1/2.50=0.400\), \(3/2.50=1.20\), and \(5/2.50=2.00\).
The increasing positive residuals may motivate a check for a covariate or other systematic difference across subjects. They do not demonstrate one: three cases are too few to establish a trend, and the independence assumption matters. If the two estimates share information, their covariance must be included in the variance of the difference. Also inspect how population PK individual estimates were obtained. Sparse data and shrinkage can make individual estimates less informative than their apparent precision suggests; an individual estimate is not error-free truth.
At the population-PK-to-ER interface, check whether the exposure supplied to the ER model is defined on the same scale and time basis as the exposure used to fit or validate that function. A systematic exposure-dependent residual may indicate a mismatch in the exposure metric, an ER function that does not capture the response pattern, or an issue in the exposure estimates. Residuals against estimated exposure can be distorted by exposure estimation error; a pattern in this plot is not automatically an ER defect.
Worked Example: An Exposure-Response Residual in a Validation Cohort
In a hypothetical validation cohort of 100 participants at specified exposure conditions, the ER model predicts a response probability of 0.30. The observed response count is 42. Assume a binomial observation model with independent outcomes and the predicted probability fixed at 0.30 for this calculation.
The predicted count is \(100(0.30)=30\). Its binomial standard deviation is \(\sqrt{np(1-p)}=\sqrt{100(0.30)(0.70)}=\sqrt{21}=4.583\) responses. The count residual is \(42-30=12\), so the standardized residual is \(12/4.583=2.619\). The observed response rate is 0.42, which is 0.12, or 12 percentage points, above the predicted rate.
This conditional discrepancy is a reason to inspect the ER prediction and its inputs, including exposure assignment, endpoint definition, and cohort comparability. The calculation treats the probability as fixed; parameter uncertainty would add predictive uncertainty and could change the standardized value. This result alone does not identify which component is responsible.
Separate ER Mismatch From Trial Implementation
The ER-to-trial-simulation handoff adds a further distinction: a trial outcome can differ from simulation because the ER function is inadequate, because the simulated design or execution does not match the trial, or because of chance variation. Trial-level residuals should therefore be compared with predictions under the trial’s actual design and relevant implementation conditions, not only with a generic ER prediction.
Useful checks include whether dose assignment, adherence, endpoint timing, concomitant medication, rescue treatment, missingness, and analysis population correspond to the simulation inputs. An implementation difference can shift outcomes in a direction that resembles ER model bias. Conversely, documenting an implementation difference does not prove that it explains the residual; the model may also be inadequate.
Worked Example: A Trial-Level Residual After an Adherence Difference
A hypothetical trial simulation predicts a 40% response rate in an active-treatment arm and, across simulated trials of the same size, a standard deviation of 6 percentage points for the observed rate. The actual trial reports 25 responses among 100 participants, or 25%. The simulation assumed 90% adherence, whereas trial records show 65%.
The observed-minus-predicted residual is \(25\%-40\%=-15\) percentage points. The standardized residual is \(-15/6=-2.50\). Thus the observed rate is 2.50 predictive standard deviations below the simulated mean under the stated simulation.
The adherence difference is a concrete implementation mismatch to investigate, including whether it was measured comparably and whether the simulation should represent it. The residual does not prove that lower adherence caused the shortfall. The ER function, other trial conditions, and uncertainty in the predictive distribution remain relevant. Re-simulation under a justified adherence scenario can assess whether that change is compatible with the result, but it is not a substitute for checking the other assumptions.
A Practical Diagnostic Sequence
State the target quantity, units, population, time window, and whether the residual is observed-minus-predicted or predicted-minus-observed.
Include relevant observation, estimation, and predictive uncertainty. Account for covariance when the quantities share data or parameters.
Plot or tabulate residuals by the inputs and conditions that component is intended to handle. Check whether the same pattern carries into later outputs.
Check units, records, endpoint construction, design, and execution. Then evaluate whether a component-level model check is warranted.
For non-normal outcomes, a standardized residual need not follow a standard normal distribution. Use an appropriate predictive distribution for the outcome, or compare the observed value with simulated predictive draws. A useful diagnostic display can combine residuals against key covariates with residuals ordered by time or study. The earlier material on cross-model posterior predictive checks examines predictive compatibility more broadly; residual displays add emphasis on where patterns occur across interfaces.
Residuals are less informative when there are few independent comparisons, when predictions are evaluated on the same data used to fit the component, or when uncertainty is poorly characterized. A single large residual can arise by chance, and multiple searches across subgroups can produce apparently striking patterns. Conversely, residuals near zero do not guarantee that a model is adequate: biases can cancel across conditions, or an aggregate can conceal subgroup structure.
Common Pitfall and Alternatives
Wrong reasoning: “The trial outcome is below the integrated prediction, so the ER function must be too optimistic.” This jumps from a final residual to a unique explanation. Correction: trace the discrepancy through the interfaces, verify that each compared quantity is aligned, and check trial implementation and uncertainty before attributing it to ER. Residual patterns help prioritize investigation; they do not adjudicate competing explanations by themselves.
A single overall fit statistic is compact but can hide direction and subgroup structure. Residual plots preserve that structure but can be noisy and are sensitive to which conditions are examined. Predictive checks assess whether observations are compatible with a model’s predictive distribution, while interface residuals make the location and direction of discrepancies easier to inspect. These approaches complement one another; none removes the need for correct data alignment, plausible uncertainty, and scientific interpretation.
Check Your Understanding
Answer each question, then compare with the solutions below.
- A measured exposure is 18 mg·h/L and its prediction is 20 mg·h/L. What is the signed residual under the observed-minus-predicted convention, and what does its sign mean?
- Two linked estimates have standard errors 3 and 4 units and covariance 6 units squared. Find the standard error of their difference.
- An ER model predicts 50 responses in a cohort, while 56 are observed. If the predictive standard deviation of the count is 3, calculate the standardized residual and state its direction.
- A simulated trial assumes full adherence, but documented adherence is much lower. Does a negative trial-level residual establish ER model bias? Give one appropriate next check.
Solutions. 1. \(18-20=-2\) mg·h/L; the measured exposure is 2 mg·h/L below the prediction. 2. \(\sqrt{3^2+4^2-2(6)}=\sqrt{13}=3.606\) units. 3. \((56-50)/3=2.00\); the observed count is above prediction by two predictive standard deviations. 4. No. The mismatch may affect outcomes but does not prove causation; check the adherence data and evaluate predictions under a simulation that represents the documented adherence, while retaining other component checks.