What Goodness-of-Fit Figures Can Show
Goodness-of-fit (GOF) figures compare observed concentrations with model predictions or show residuals, the differences between observations and predictions. They help reviewers assess whether the fitted population pharmacokinetic (PK) model captures broad features of the data: for example, whether it systematically overpredicts at late times, misses concentration-dependent variability, or fits one subgroup differently from another. These plots are diagnostic evidence, not a verdict on the model and not a substitute for other model evaluation.
A pattern is more informative than an isolated point. A few observations far from the main cloud can occur even when a model is broadly adequate, while a modest but consistent trend can signal a systematic discrepancy. Interpretation also depends on what was plotted, how predictions and residuals were calculated, and which observations entered the figure.
Predictions, Residuals, and Their Reference Patterns
In population PK reporting, \(DV\) commonly denotes the observed dependent variable, such as concentration. A population prediction, often labelled PRED, is calculated from the population-level model without using an individual's estimated random effects. An individual prediction, often labelled IPRED, incorporates estimated individual effects. The two comparisons answer different diagnostic questions: observed values versus PRED show how the population model describes the overall data, while observed values versus IPRED show agreement after individual effects have been estimated.
IPRED plots can appear more favorable because the same individual's data contribute to estimating the individual effect used in the prediction. They therefore should not be treated as independent validation of population-level predictive performance. Report which prediction is shown and interpret the plot accordingly.
In an observed-versus-predicted plot, the reference is the identity line: observed value equals predicted value. Points distributed on both sides of this line without a clear trend are broadly consistent with agreement, allowing for residual variability. A cloud consistently above the line indicates underprediction; one consistently below indicates overprediction. A curved pattern can indicate that agreement changes across the prediction range. When concentrations are displayed on logarithmic axes, the identity line remains the equality reference, but the visual spacing reflects multiplicative rather than additive differences.
A residual-versus-prediction plot makes some departures easier to see. With raw residuals, the reference is zero. Under a suitable model for the conditional mean, residuals should be centered around zero on average, without a systematic pattern against prediction. Their spread need not be constant on the raw scale: concentration data often have variability that changes with the expected concentration. Standardized residuals are intended to account for expected differences in scale, but their interpretation depends on the residual definition and the model's variance assumptions.
Why Centering Is Expected
The basic justification for residual plots follows from the conditional mean model. Let \(Y_i\) be an observation and let \(\widehat{\mu}_i\) be its model-predicted conditional mean. If the fitted mean is adequate for the relevant conditions, then the residual \(e_i=Y_i-\widehat{\mu}_i\) should have conditional expectation near zero:
This expectation motivates checking whether residuals are centered around zero across time, predictions, or subgroups. If a residual is standardized by a correctly specified conditional standard deviation \(\widehat{\sigma}_i\), then \(r_i=e_i/\widehat{\sigma}_i\) has an approximately unit scale when the variance model is appropriate. These are model-based expectations, not guarantees that a finite sample will form a perfectly uniform band. Repeated observations from the same participant may also be correlated, so a plot's points are not necessarily independent evidence.
Worked Example: Reading an Observed-Versus-Predicted Plot
Suppose a hypothetical report shows five concentrations, in mg/L, against population predictions. The pairs \((PRED,DV)\) are \((1.0,1.1)\), \((2.0,1.9)\), \((4.0,4.2)\), \((8.0,7.8)\), and \((12.0,12.3)\). Calculate the raw residuals and describe the visible evidence.
For each point, \(e_i=DV_i-PRED_i\). The residuals are \(1.1-1.0=0.1\), \(1.9-2.0=-0.1\), \(4.2-4.0=0.2\), \(7.8-8.0=-0.2\), and \(12.3-12.0=0.3\) mg/L.
The residuals are small in absolute terms and occur on both sides of zero. The observations are also close to the identity line. These five points show no obvious consistent overprediction or underprediction, but five observations cannot establish that the model is adequate across the full study. In particular, the raw residual sizes should be considered in relation to the expected variability at each concentration.
Look for Structure Across Time, Concentration, and Subgroups
The diagnostic question is not simply whether residuals are near zero overall. A positive pattern in one region and a negative pattern in another can cancel in an overall average. Examine residuals against time after dose, predicted concentration, and clinically or scientifically relevant subgroups when those displays are appropriate to the analysis question. Use the sampling design and data-handling decisions described in the report to interpret what is visible.
| Display | Pattern to inspect | Possible interpretation, not a conclusion by itself |
|---|---|---|
| Observed versus PRED | Consistent displacement from the identity line or curvature | Population-level bias that varies with prediction |
| Residual versus time | Runs of positive or negative values, or a change over time | Time-related structure not captured by the fitted model |
| Residual versus predicted concentration | Trend in residual center or changing spread | Concentration-dependent mean discrepancy or variance mismatch |
| Residual by subgroup | Different centers or spreads across groups | Potentially relevant subgroup structure, possibly confounded by design or other covariates |
Worked Example: A Time-Related Residual Pattern
In a hypothetical residual-versus-time plot, six standardized residuals are \(1.4, 1.1, 0.8, -0.9, -1.2,\) and \(-1.5\) at times 1, 2, 4, 8, 12, and 24 hours after dose, respectively. Calculate the average for the first three and last three observations, then interpret the pattern.
For the first three times, the average is \((1.4+1.1+0.8)/3=3.3/3=1.1\). For the last three, it is \((-0.9-1.2-1.5)/3=-3.6/3=-1.2\). Thus, the early observations tend to be above their predictions and the later observations tend to be below them.
This direction is consistent with early underprediction followed by later overprediction. It is a time-related pattern worth investigating, not proof of a particular structural model defect. The reviewer should consider the number of observations, the distribution of sampling times, participant-level clustering, and whether a few participants drive the pattern. The plot alone does not identify the cause.
Worked Example: Comparing Residuals by Subgroup
A hypothetical plot displays standardized residuals by renal-function group. The four values in the reference group are \(-0.2, 0.3, -0.1,\) and \(0.2\); the four values in a lower-function group are \(1.1, 1.4, 0.9,\) and \(1.2\). Calculate each group's mean residual and state what the comparison supports.
For the reference group, the mean is \((-0.2+0.3-0.1+0.2)/4=0.2/4=0.05\). For the lower-function group, the mean is \((1.1+1.4+0.9+1.2)/4=4.6/4=1.15\). The lower-function observations are, on average, above their predictions, whereas the reference-group values are centered close to zero.
This small hypothetical display suggests possible underprediction in the lower-function group. It does not show that renal function caused the discrepancy, nor that a covariate effect is necessarily the correct remedy. Differences in dose, sampling times, participant composition, or other model inputs could contribute. The number and spread of observations also limit how strongly the pattern can be interpreted.
Choosing and Comparing Diagnostic Displays
No single GOF plot answers every question. Observed-versus-PRED plots provide an overall view of population-level agreement, while residual plots can reveal whether departures vary across a specific axis. Residuals versus time are useful for detecting time-ordered structure; residuals versus prediction help assess concentration-related trends and changing spread; subgroup displays can reveal patterns obscured by pooling. Observed-versus-IPRED plots answer a related but different question, because individual effects have been estimated from the observations.
These displays are complementary rather than competing procedures. A broad cloud around the identity line does not rule out a time trend, and a centered residual cloud overall does not rule out opposing subgroup patterns. Conversely, a visually unusual feature may reflect sparse data, an influential participant, or design imbalance rather than a persistent model deficiency. Where a display raises a concern, inspect the underlying data and relevant analysis decisions before proposing an explanation.
Diagnostics, Failure Modes, and Reporting
Before interpreting a figure, check its labels, scales, reference lines, prediction type, residual definition, and included observations. Confirm that units are clear and that any transformation or axis limits are disclosed. A log scale can compress large values and change the apparent prominence of deviations. Overplotting can hide dense regions; a few extreme observations can dominate the visible range. If below-quantification observations were handled specially, confirm how they are represented rather than treating their plotted values as ordinary measured concentrations.
A residual plot is not a formal hypothesis test. Do not infer a precise error rate, a statistically significant effect, or a model's overall acceptability from visual inspection alone. Nor does an apparent pattern identify a unique cause. Use measured language that distinguishes the observation from its interpretation: “residuals tended to be positive at early times” describes the figure; “the model underpredicts early concentrations” is a reasonable interpretation when supported by the plotted comparison; “the absorption model is wrong” would require additional evidence.
A useful report states what was plotted, identifies the most relevant visible pattern, notes its scale or scope, and gives an appropriately cautious interpretation. If no notable structure is apparent, say so without claiming that the plot proves adequacy. If a pattern is present, describe where it occurs and whether it appears broad or driven by sparse observations. This makes the figure reviewable without turning visual evidence into a stronger claim than it supports.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- An observed concentration is 5.6 mg/L and its prediction is 6.0 mg/L. What is the raw residual, and does it indicate overprediction or underprediction?
- What is the identity-line reference in an observed-versus-predicted plot?
- Why should an observed-versus-IPRED plot not be treated as independent evidence of population-level predictive performance?
- Standardized residuals average 0.0 overall, but are mostly positive early and negative late. What is the appropriate interpretation?
- A subgroup has positive residuals. Does that establish that subgroup membership caused the model discrepancy? Explain.
Solutions. 1. \(e=5.6-6.0=-0.4\) mg/L, so the prediction is higher than the observation: the model overpredicts this observation. 2. The identity line is \(DV=PRED\); points on it have observed values equal to their predictions. 3. IPRED incorporates estimated individual effects, which use information from that individual's observations, so apparent agreement is partly conditional on those data. 4. The overall mean can conceal time-related structure; describe the early positive and late negative pattern and investigate it cautiously. 5. No. Positive residuals indicate observations above predictions in that subgroup, but design differences, other covariates, sparse data, or other factors may contribute; the plot does not establish causation or a unique explanation.