What Goodness-of-Fit Diagnostics Can Show
A fitted model can converge and produce precise parameter estimates yet still describe the observations poorly in a systematic way. Goodness-of-fit (GOF) diagnostics help assess whether discrepancies between observed data and model predictions show patterns that the model has not captured. In a report review, the task is not simply to decide whether a plot looks acceptable. It is to identify what the plot displays, whether the residuals are appropriate for the model, and whether the written interpretation stays within the evidence.
A residual is a discrepancy between an observation and a corresponding model prediction. Its sign and scale depend on how it was constructed. For example, a raw residual may be defined as observed minus predicted, while a standardized or weighted residual also accounts for an estimate of variability. In pharmacometric reports, residual plots may use quantities such as conditional weighted residuals (CWRES). A reviewer should check the figure caption or methods description rather than assume that every plotted residual has the same definition.
For a well-calibrated residual, values should generally fluctuate around zero without a systematic relationship to time, concentration, or prediction. This is a diagnostic expectation, not a requirement that every residual be small or that every plot look perfectly random. Residuals may be correlated, their distributions can be affected by parameter estimation, and sparse observations can make patterns difficult to discern. A reviewer should therefore interpret them in light of the model structure and data design.
Why Patterns Matter
The basic justification is straightforward. Suppose, for illustration, that an observation follows \(Y=\mu+\epsilon\), where the model gives the correct conditional mean \(\mu\), and the residual error has mean zero and standard deviation \(\sigma\). For \(R=(Y-\mu)/\sigma\),
Thus, if a fitted mean or error model is systematically wrong in a way related to time or concentration, residuals may depart from zero or show a trend across that variable. If residual variability is misspecified, their spread may change across the plot. This argument assumes the prediction and scale are appropriate and does not establish that residuals from a fitted, hierarchical model are independent standard normal observations. In repeated-measures and population models, the residual definition and covariance structure matter.
A smooth trend, curve, changing spread, or grouping by time or concentration can be more informative than an isolated large value. Such patterns can suggest a missing structural feature, an inadequate error model, a data-processing issue, or an influential subset of observations. The plot alone generally does not determine which explanation is correct. The report should describe the pattern and its scope before attributing a cause.
Worked Example: Checking for a Time-Related Pattern
A hypothetical report shows six standardized residuals against time after dose. The three early residuals are \(-1.1\), \(-0.8\), and \(-0.4\); the three late residuals are \(0.3\), \(0.9\), and \(1.1\). Calculate the group means and their difference, then state what the pattern supports.
Early mean. The early residuals sum to \(-1.1-0.8-0.4=-2.3\), so their mean is \(-2.3/3=-0.7667\) standardized-residual units.
Late mean. The late residuals sum to \(0.3+0.9+1.1=2.3\), so their mean is \(2.3/3=0.7667\) standardized-residual units.
Difference. Using the unrounded sums, the late-minus-early difference is \(2.3/3-(-2.3/3)=4.6/3=1.5333\), or approximately 1.533 standardized-residual units. The residuals shift from mostly negative early to positive late. That is a descriptive indication of a possible time-related pattern: the model tends to predict above the observations early and below them late, given the residual sign convention.
There are only three residuals in each group, so the means are unstable summaries, not proof of a time effect or a formal test. A reviewer should inspect the plotted observations, the time range, and whether the residuals are appropriate for the model before endorsing a strong claim.
Reviewing Time and Concentration Dependence
Time-related bias can appear as a gradual rise or fall, a curved pattern, or a change around a particular phase of the dosing interval. Concentration-related bias may appear as residuals predominantly below zero at low concentrations and above zero at high concentrations, or as a change in residual spread with concentration. These patterns are clues, not diagnoses. In particular, a relationship with concentration can be hard to separate from a relationship with time when concentration changes systematically over time.
The choice of horizontal axis should match the question. Residuals versus time can reveal temporal structure; residuals versus observed or predicted concentration can reveal concentration-dependent behavior. Plots against both observed and predicted values can be useful, but each has limitations: observed values also contain residual error, while predictions depend on the fitted model. A report should identify which quantity is on each axis and should not describe a trend in one as though it were a trend in another.
Worked Example: Interpreting Concentration-Related Bias
In a hypothetical diagnostic table, standardized residuals for four low-concentration observations are \(0.1\), \(-0.2\), \(0.2\), and \(-0.1\). Residuals for four high-concentration observations are \(0.8\), \(1.0\), \(1.2\), and \(0.9\). Calculate the means and assess the report statement: “The residuals show no concentration-related pattern.”
Low-concentration mean. The sum is \(0.1-0.2+0.2-0.1=0\), so the mean is \(0/4=0\).
High-concentration mean. The sum is \(0.8+1.0+1.2+0.9=3.9\), so the mean is \(3.9/4=0.975\).
The high-concentration residuals are all positive, while the low-concentration residuals are centered at zero. The high-minus-low mean difference is \(0.975-0=0.975\) standardized-residual units. If positive means observed above predicted, this pattern is consistent with underprediction at high concentrations. The report’s statement that there is “no concentration-related pattern” does not match these displayed values.
The example does not establish why the pattern occurs or whether it is consequential for a development decision. The reviewer should check the full plot, the number and distribution of observations, and whether high concentration is confounded with time or another feature of the design. The conclusion should remain descriptive unless additional evidence supports a stronger claim.
Outliers and Influential Observations
An outlying residual is unusually far from zero relative to the scale of the residual being plotted. An influential observation or subject is one whose inclusion materially affects fitted parameters, predictions, or conclusions. These concepts are related but not interchangeable. A large residual may have limited influence if the observation has little leverage on the fit; an observation with a moderate residual may be influential if it is especially informative for a parameter.
A large absolute standardized residual is a prompt for investigation, not an automatic deletion rule. The reviewer can examine the underlying record, units, timing, assay or data-processing flags, and the relevant individual fit. Influence is more directly assessed by comparing the fitted results with and without the observation or subject, using a clearly documented refit or other suitable influence analysis. Any exclusion must be justified independently of whether it improves the fit.
Worked Example: Separating a Large Residual From Influence
A hypothetical population pharmacokinetic analysis has one subject with a standardized residual of \(3.2\); the other displayed residuals have absolute values no greater than \(1.4\). The full-data estimate of typical clearance is \(12.0\) L/h. In a sensitivity refit omitting that subject, the estimate is \(10.8\) L/h. What can a reviewer conclude?
Residual magnitude. The subject’s residual is large relative to the other displayed residuals. Under a simple standardized Gaussian residual interpretation, its squared value is \(3.2^2=10.24\). This indicates a substantial discrepancy for that observation under that scale, but it is not by itself a formal outlier test, particularly when many observations or subjects are inspected.
Change in clearance estimate. The absolute change is \(10.8-12.0=-1.2\) L/h. Relative to the full-data estimate, the change is \((-1.2/12.0)\times100=-10.0\%\). The estimate is 10.0% lower when the subject is omitted, so this sensitivity result indicates that the subject may be influential for typical clearance.
The reviewer should not conclude that the subject should be removed. The analysis should establish whether the record is valid, explain why the sensitivity refit is relevant, and show whether the main decision changes. A large residual and a material refit change raise distinct questions: fit for that subject and dependence of the result on that subject.
Comparing Diagnostic Displays and Reviewing Claims
Different GOF displays answer related but distinct questions. An observed-versus-predicted plot shows overall agreement and can reveal broad departures from the identity line, but it may hide time ordering or subgroup structure. Residual-versus-time and residual-versus-concentration plots are better suited to detecting patterns against those axes. Individual observed-and-predicted profiles show whether a population-level summary conceals poor fits for particular subjects. A residual distribution or quantile plot can reveal asymmetry or unusually heavy tails, but cannot by itself establish that predictions are unbiased across time or concentration.
These displays complement one another; none establishes model validity alone. A reviewer should check that the figure uses the stated residual type, that axis labels and units are clear, and that the plotted population and analysis set agree with the report. The caption and narrative should distinguish “no obvious pattern” from “the model is validated” and should mention visible exceptions rather than relying only on an overall impression.
Confirm the residual definition, sign convention, prediction type, units, and whether points represent observations or subjects.
Inspect for trends, curvature, changing spread, clusters, and unusually large residuals against the relevant axis.
Consider data density, repeated observations, time-concentration relationships, and whether a few subjects dominate the visible pattern.
State what the plot supports, what remains uncertain, and whether further review or a sensitivity analysis is warranted.
Diagnostics can fail to reveal important problems when data are sparse in a region, axes obscure a pattern, or pooled displays average opposing subject-level behaviors. Conversely, random variation can produce a striking point or short run of residuals. Visual checks are not a substitute for examining the data and model specification, and an apparently pattern-free plot is not proof that the model is correct. Formal predictive checks and broader validation evidence address different questions and should not be inferred from GOF plots alone.
Common Review Pitfalls
- Calling any residual near zero evidence of validity. A few small residuals do not establish adequate fit across the full range of time, concentration, or subjects.
- Treating a large residual as grounds for deletion. Investigate data validity and influence; do not remove an observation solely because it conflicts with the model.
- Confusing association with explanation. A residual trend against concentration may also reflect time or design structure. The plot identifies a pattern, not its cause.
- Rounding before calculating a difference. Use unrounded sums or means for arithmetic, then round the final result to the reported precision.
- Overstating a clean-looking plot. “No obvious systematic pattern is visible” is a bounded description; “the model is validated” is a broader claim that the plot alone cannot support.
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions.
- A residual is defined as observed minus predicted. What does a positive residual mean?
- Residual means are \(-0.5\) in an early time window and \(0.4\) in a late window. What is the late-minus-early difference, and what does it describe?
- Why does a large standardized residual not prove that an observation is influential?
- A report says “the model is unbiased at all concentrations” although the residual plot shows a steady increase from negative to positive values as concentration rises. Is the statement supported?
Solutions. 1. The observation is above the prediction; the interpretation is in the units of the raw residual, or in standardized-residual units if standardized. 2. The difference is \(0.4-(-0.5)=0.9\) standardized-residual units. It describes a shift in the group means, not a formal test or proof of a concentration or time effect. 3. Residual magnitude describes discrepancy, whereas influence concerns how much the fitted results change when the observation is omitted or otherwise assessed; the two need not coincide. 4. No. The increasing trend is evidence of a visible concentration-related pattern, so the blanket “unbiased at all concentrations” claim is stronger than the display supports. The plot alone does not establish the cause or its decision impact.