Why inspect goodness of fit?
A population pharmacokinetic model can return plausible parameter estimates and still fail to describe important features of the concentration data. Goodness-of-fit (GOF) diagnostics provide a structured way to look for discrepancies between observed concentrations and model predictions. They are not a single pass-or-fail test: different plots can reveal different problems, and apparent patterns require follow-up rather than automatic model changes.
The central comparison is between an observed concentration and a prediction for that observation. In a population model, a population prediction uses the estimated population parameters and the subject's dosing and covariate information, but not that subject's estimated individual random effects. An individual prediction also uses the subject-specific empirical Bayes estimates (EBEs). Because individual predictions use information from that subject's observations, they often look closer to the data. A close individual-prediction fit does not by itself demonstrate that the population model captures between-subject patterns well.
The expected concentration and the observation error should be understood on the model's scale. For example, a proportional error model typically implies larger absolute scatter at higher predicted concentrations. A plot with a fan-shaped spread is not automatically evidence of a problem if that spread is consistent with the specified error model. The diagnostic question is whether the observed pattern is compatible with the model, including its residual variability assumptions.
Reading concentration-versus-prediction plots
An observed-versus-predicted plot places \(DV\) on the vertical axis and either \(PRED\) or \(IPRED\) on the horizontal axis. A reference line \(DV=PRED\) (or \(DV=IPRED\)) marks exact agreement. Points scattered around that line without a systematic pattern are broadly consistent with adequate prediction at the plotted scale. The line is a visual reference, not a fitted regression that proves model adequacy.
For a plot against population predictions, check whether points tend to fall above or below the reference line over a range of concentrations. A curved pattern can suggest a structural mismatch, such as a concentration-dependent discrepancy. A widening spread may reflect proportional error, but a change in spread beyond what the error model allows may signal misspecified variability. Plotting on a logarithmic concentration scale can help when concentrations span a wide range, but values and any transformations must be interpreted on the displayed scale.
Worked Example: A Concentration-Dependent Pattern
Suppose a hypothetical population PK analysis produces these observed and population-predicted concentrations, all in mg/L: \(PRED=2\), \(DV=2.3\); \(PRED=5\), \(DV=5.6\); \(PRED=10\), \(DV=12.0\); and \(PRED=20\), \(DV=25.0\). The reference line predicts observed values of 2, 5, 10, and 20 mg/L, respectively.
The raw residuals are \(0.3\), \(0.6\), \(2.0\), and \(5.0\) mg/L, calculated as \(DV-PRED\). Each is positive, and the absolute difference grows at higher predictions. The first two observations are close to the reference line, while the last two lie increasingly above it. This pattern raises a question about concentration-dependent bias, but four points do not establish its cause. The analyst should examine the full plot, relevant time patterns, and whether the residual variability model predicts larger absolute scatter at higher concentrations.
Comparing population and individual prediction plots is useful, but they answer different questions. If the individual-prediction plot looks substantially better than the population-prediction plot, that can occur because EBEs use the individual data. It does not automatically mean that the structural model is correct, nor that population predictions are adequate for development decisions. Conversely, poor fit against both prediction types may indicate a broader mismatch, data issue, or error-model problem that needs investigation.
Residual plots and variability
Residual-versus-prediction plots make it easier to see systematic deviations and changes in spread. Raw residuals are in concentration units and are useful for interpreting the size of an error. However, raw residuals are not directly comparable across observations if their expected variability differs. A weighted or standardized residual divides the difference by an appropriate model-based standard deviation.
For an additive-plus-proportional residual model with independent components, one possible observation-error variance is \(\operatorname{Var}(DV_i\mid\text{model})=\sigma_{\mathrm{add}}^2+(\sigma_{\mathrm{prop}}M_i)^2\). The corresponding standard deviation is the square root of this variance. This expression is not universal: the exact variance depends on the model's error specification and scale. Use the variance actually specified in the analysis, not a convenient substitute.
Worked Example: Standardizing a Residual
In a hypothetical analysis, an observation is \(DV=12.0\) mg/L and its population prediction is \(PRED=10.0\) mg/L. The fitted additive-plus-proportional error model has \(\sigma_{\mathrm{add}}=0.5\) mg/L and \(\sigma_{\mathrm{prop}}=0.10\), with independent additive and proportional components.
The model-based standard deviation at this prediction is \(\sqrt{0.5^2+(0.10\times10.0)^2}=\sqrt{0.25+1.00}=\sqrt{1.25}=1.118\) mg/L. The raw residual is \(12.0-10.0=2.0\) mg/L, so the standardized residual is \(2.0/1.118=1.789\). The observation is about 1.79 model-based standard deviations above its population prediction. This value alone does not establish an outlier or model failure; its significance depends on the residual definition, the other observations, and the model assumptions.
A residual-versus-prediction plot should be inspected for a nonzero trend, curvature, unequal spread, and isolated large values. A funnel in standardized residuals may indicate that the variance model is inadequate, although it can also arise from an unsuitable residual definition or an unmodeled data feature. If residuals are standardized correctly, their spread should not show a systematic dependence on prediction that the model fails to anticipate.
Time-stratified diagnostics
A model can fit concentrations overall while missing a particular part of the concentration-time profile. Plotting residuals against time after dose, or summarizing them in prespecified time windows, can reveal such patterns. For instance, residuals that are mostly positive early and negative late may suggest a mismatch in the predicted time course. A consistent trend across time can point toward structural misspecification, whereas a pattern limited to a dosing phase or sampling window may motivate review of that phase, the data, and the model assumptions.
Time windows should be chosen for pharmacological or design reasons and interpreted with the number and distribution of observations in mind. A mean residual near zero for the whole dataset can conceal positive residuals in one interval and negative residuals in another. Conversely, a few observations in a sparse interval can make its average unstable. Show the data distribution or number of observations alongside summaries where practical, and avoid treating arbitrary bin boundaries as biological evidence.
Worked Example: Opposing Residual Patterns Across Time
Consider a hypothetical diagnostic summary using CWRES, with 40 observations in each of two prespecified intervals. The mean CWRES is \(+0.72\) from 0 to 2 hours after dose and \(-0.68\) from 8 to 12 hours after dose.
The early observations tend to be above their model-based predictions, while the later observations tend to be below. The contrast is more informative than an overall mean alone: if the two groups had equal sizes, their combined mean would be \((40\times0.72+40\times(-0.68))/80=0.02\), close to zero. That overall summary could mask the time-dependent pattern. This diagnostic suggests checking whether the predicted absorption and decline phases match the data; it does not identify which model component is responsible or prove that the pattern is systematic beyond this dataset.
Justification and practical diagnostic sequence
The rationale for standardized residuals follows directly from the observation model. Suppose an observation has prediction \(M_i\) and error \(\epsilon_i\), so \(DV_i=M_i+\epsilon_i\), with \(E(\epsilon_i)=0\) and \(\operatorname{Var}(\epsilon_i)=SD_i^2\). Then \(e_i=DV_i-M_i=\epsilon_i\). Therefore, \[ E(e_i)=0,\qquad \operatorname{Var}(e_i)=SD_i^2. \] Dividing by \(SD_i\) gives \(r_i=e_i/SD_i\), so \(E(r_i)=0\) and \(\operatorname{Var}(r_i)=1\) under those assumptions. This explains why standardized residuals with a trend or changing spread can flag a mismatch. In practice, predictions and variance parameters are estimated, observations may be correlated through shared subject effects, and conditional residuals have software-specific adjustments. Thus the derivation is a reference expectation, not a guarantee that every fitted residual plot will behave exactly like independent standard normal draws.
Identify whether the plot uses population or individual predictions, and confirm that observed and predicted concentrations use compatible units and scales.
Inspect observed-versus-predicted plots and residual trends for persistent departures from the reference line or zero.
Assess whether residual spread changes with prediction and whether residual patterns differ across meaningful time intervals.
Review data, timing, dosing records, and subject-level patterns, then assess whether conclusions depend strongly on the observation.
Influential observations, alternatives, and limitations
An observation is influential when it has a substantial effect on fitted parameters or on conclusions, not merely because its residual is large. A large residual can be compatible with the assumed observation variability and have little effect on the fit; a point with a moderate residual can be influential if it occurs in a sparsely sampled, informative region. A practical sensitivity assessment is to compare the original fit with a justified refit excluding the observation or subject, while also checking the source data. Such exclusion is a diagnostic exercise, not a basis for removing valid data without justification.
Worked Example: Assessing Potential Influence
In a hypothetical one-compartment analysis, the full dataset gives estimated clearance \(CL=5.00\) L/h. One subject has a terminal concentration that is poorly matched by the population prediction. After verifying the record, a sensitivity refit excluding that subject gives \(CL=4.20\) L/h.
The absolute change is \(5.00-4.20=0.80\) L/h, and the change relative to the full-data estimate is \(0.80/5.00=0.160\), or 16.0%. This is a material sensitivity signal worth investigating, not proof that the subject is erroneous or that the full-data estimate is invalid. The analyst should consider whether the observation is valid, whether the model explains the subject's profile, and whether the development conclusion changes across defensible analyses.
GOF plots are complementary to, not interchangeable with, other approaches. Numerical summaries can make comparisons concise but may hide opposing patterns that cancel out. Subject-level plots can reveal whether a population pattern is driven by a few individuals, while population-level plots summarize the overall relationship. Formal likelihood-based comparisons address specified model comparisons, not every possible form of misspecification. Simulation-based predictive checks provide a different assessment of whether model-generated data reproduce observed features; that is a distinct approach from the residual and prediction diagnostics emphasized here.
Visual diagnostics have limitations. They can be affected by sparse sampling, correlated observations within subjects, shrinkage of EBEs, and the number of points obscuring local patterns. A plot can suggest where a model is inadequate but usually cannot identify the unique cause. The introductory material on interpreting shrinkage and empirical Bayes estimates is relevant when judging individual-prediction and subject-level plots: strong shrinkage can make EBEs less informative about individual differences. Do not interpret every departure as a reason to add parameters. First check data quality, prediction definitions, residual assumptions, and whether the pattern is stable across subjects and clinically relevant regions.
A common mistake is to call any point far from the identity line an influential outlier and delete it. This conflates residual size with influence and ignores the model's expected error scale. The correction is to assess standardized as well as raw differences, verify the observation, and quantify the effect of a justified sensitivity analysis. Another mistake is to declare a model adequate because the mean residual is near zero. Positive and negative patterns can cancel, so examine prediction, time, and subject structure together.
Check Your Understanding
Answer each question, then compare with the concise solutions below.
- An observation is \(DV=8.0\) mg/L and \(PRED=10.0\) mg/L. What is its raw residual, and what does its sign mean?
- Why can an individual-prediction plot look better than a population-prediction plot without establishing good population-level performance?
- In a time-stratified plot, early residuals are mainly positive and late residuals mainly negative. Give one interpretation and one reason not to overstate it.
- What distinguishes an influential observation from an observation with a large residual?
- For \(SD=0.8\) mg/L and \(DV-PRED=1.2\) mg/L, calculate the simple standardized residual.
Solutions. (1) \(8.0-10.0=-2.0\) mg/L; the observation is 2.0 mg/L below the population prediction. (2) Individual predictions use subject-specific EBEs estimated using that subject's data, so closer agreement does not alone show that population predictions capture between-subject patterns. (3) The pattern may indicate a time-course mismatch; sparse data, random variation, or the selected time intervals could also contribute, so inspect supporting data and subjects. (4) Influence means a substantial effect on fitted parameters or conclusions; residual size alone does not measure that effect. (5) \(1.2/0.8=1.50\), or 1.50 model-based standard deviations above the prediction.