What Goodness-of-Fit Displays Can—and Cannot—Show
Goodness-of-fit (GOF) displays summarize how model predictions compare with observed data. In a pharmacometric analysis, they are useful for asking focused questions: Do predictions track the range of observations? Are residuals centered around zero? Do discrepancies appear to change with time or predicted concentration? A plot can make these patterns visible, but it cannot by itself identify their cause or establish that the model is adequate for every intended use.
Interpretation starts with the plotted quantities. Check the axis labels, units, prediction type, residual definition, and any transformations before describing a pattern. A residual plotted in concentration units has a different scale and meaning from a standardized residual. Likewise, a population prediction and a subject-specific prediction do not provide the same diagnostic comparison.
A population prediction, often labeled PRED, is calculated from population-level model parameters and the relevant covariates, without using an individual-specific estimated random effect. An individual prediction, often labeled IPRED, includes a subject-specific estimate of random effects. Because IPRED uses information from that subject’s observations to estimate those effects, its apparent agreement with the same observations can be better than PRED’s. State which prediction is shown; do not treat the two displays as interchangeable.
Common displays include observed values versus predictions, residuals versus predictions, and residuals versus time. Some analyses show raw residuals; others show residuals standardized or normalized using a model-based estimate of their variability. Interpret the residual scale as defined by the analysis. Do not infer that residuals are standardized merely because they are called residuals.
Reading Observed-versus-Predicted Displays
An observed-versus-predicted plot places predictions on one axis and observations on the other. If both axes use the same units and scale, the identity line \(y_i=\hat{y}_i\) marks exact agreement. Points above it have positive residuals; points below it have negative residuals. The overall spread around the line describes the size of discrepancies, while systematic departures from it may suggest that predictions are biased in some part of the plotted range.
A cloud that follows the identity line is generally more reassuring than a clear curved or consistently displaced pattern, but “close to the line” needs context: the display’s scale, the amount of data, and the model’s intended application all matter. A few points far from the line may merit checking, but are not by themselves proof that the model is wrong. Conversely, a visually close fit in a dense region does not ensure adequate predictions in a sparsely observed region.
Worked Example: Describing Agreement on an Observed-versus-Predicted Plot
Suppose a hypothetical concentration analysis shows four observations, all in mg/L. The pairs of predicted and observed concentrations are \((10,12)\), \((20,21)\), \((30,29)\), and \((40,39)\). Calculate the signed residuals and describe the display.
Using \(r_i=y_i-\hat{y}_i\), the residuals are \(12-10=2\), \(21-20=1\), \(29-30=-1\), and \(39-40=-1\) mg/L. Their mean is
The first two points lie above the identity line and the last two below it. All four predictions are within 2 mg/L of their observations, and the residuals do not show a consistent direction across this small set. A cautious report might say that the four observations were reasonably close to their predictions, with residuals from \(-1\) to \(2\) mg/L. The small sample does not establish that predictions are unbiased more generally.
A pattern can also change across the prediction range. For example, observations consistently above the identity line at low predictions and below it at high predictions suggest that low values may be underpredicted and high values overpredicted. A curved pattern can likewise indicate that discrepancies vary with prediction magnitude. These are descriptions of the observed pattern, not unique explanations: alternative data issues, model structure, or influential observations may also contribute.
Interpreting Residual Displays
A residual-versus-prediction plot makes the direction of discrepancy explicit. If a mean model is appropriate and the residuals are defined on a suitable scale, residuals should have no strong systematic trend in their conditional center. A horizontal reference line at zero helps assess whether residuals are predominantly positive, predominantly negative, or distributed on both sides. In a residual-versus-time plot, a systematic run of positive or negative residuals can indicate time-related mismatch that deserves investigation.
This expectation follows directly from the residual definition. If the model’s conditional mean prediction is correct for a particular set of predictors, then the conditional expected residual is zero:
where \(x_i\) denotes the predictors used for that prediction and \(\hat{y}_i=E(y_i\mid x_i)\) in this justification. This is an expectation under the stated model, not a requirement that a finite set of residuals average exactly to zero in every subgroup. Fitted parameters, repeated observations within subjects, and model-based prediction construction also mean that plotted residuals should not automatically be treated as independent observations.
Worked Example: Recognizing a Time-Related Pattern
In a hypothetical study, a residual-versus-time display uses raw residuals in mg/L. At six successive sampling times, the residuals are \(-3,-2,0,2,3,4\) mg/L. Calculate their mean and describe the visible pattern.
The mean residual is
The sequence rises from negative to positive values, and the last three residuals are positive. This suggests that predictions may be high relative to observations early in the sequence and low relative to observations later in it, if the ordering corresponds to increasing time and other relevant features are comparable. It is not enough to conclude why the pattern occurs. A report could state: “Raw residuals increased across the displayed sampling times, changing from mostly negative to positive; this time-related pattern warrants review.” The display alone does not establish a particular structural error or biological mechanism.
Residuals can also fan out or narrow as predictions increase. Such a pattern concerns the spread, not just the center, of residuals. If raw residual spread grows with predicted concentration, this may be consistent with a residual-error model whose variability increases with the level of the response. It may also reflect other features of the data or model. A standardized-residual display may help assess spread relative to the specified variability model, but only if the standardization and its assumptions are understood.
Worked Example: Describing a Change in Residual Spread
Consider a hypothetical residual-versus-prediction display with raw residuals in mg/L. At lower predictions of 2 and 4 mg/L, residuals are \(-0.2\) and \(0.4\) mg/L. At higher predictions of 8 and 10 mg/L, they are \(-1.2\) and \(1.5\) mg/L. Compare the average absolute residual in the two groups.
For lower predictions, the average absolute residual is
For higher predictions, it is
The two higher-prediction residuals have greater absolute magnitudes than the two lower-prediction residuals. This small hypothetical display is consistent with greater raw residual spread at higher predictions, but four observations cannot establish a general variance pattern. A cautious narrative would describe the visible increase and note that it merits evaluation against the residual-error model, rather than claiming that the model’s variance structure is definitively wrong.
From a Visible Pattern to a Results Narrative
A useful report separates what the display shows from what the pattern might mean. Describe the plot, direction, scale, and relevant range first. Then give a restrained interpretation, using language such as “may indicate,” “is consistent with,” or “warrants further evaluation” when the cause is not established. State whether the display uses population or individual predictions and whether residuals are raw or standardized.
Name the observed quantity, prediction type, residual type, axes, and units.
Report whether points follow the identity line or whether residuals show a directional trend, changing spread, or notable departures.
State the possible implication for model predictions, but do not assign a unique cause from the plot alone.
Consider sparse regions, influential points, dependence among observations, and the model’s intended use before making a broader claim.
For example, “Population predictions tracked observations over the central concentration range, while raw residuals showed a wider spread at higher predictions” is more informative than “The model fit well.” It identifies the prediction type, range, and pattern without declaring overall adequacy. A report should not claim that visual agreement proves the model is correct, that a residual trend proves a specific mechanism, or that a display substitutes for evaluating other relevant evidence.
Diagnostics, Alternatives, and Limitations
Before interpreting an apparent pattern, verify that observations and predictions refer to the same records, times, outcomes, and units. Check whether axes are transformed and whether the residual sign convention is \(y_i-\hat{y}_i\); some software or reporting pipelines may define discrepancies differently. Review the number and distribution of points, because overplotting can hide patterns and sparse regions make visual conclusions uncertain. A striking point should prompt data and model checks, not automatic deletion.
Observed-versus-predicted displays provide a direct visual comparison on the chosen scale. Residual displays make direction, centering, and spread easier to inspect; residuals versus time can expose a temporal pattern that is less obvious when all observations are combined. Neither view replaces the other. A numeric summary such as average residual can compactly describe a center, but may conceal opposing patterns that cancel. A plot can reveal those patterns, but its interpretation is more subjective and sensitive to scale and point density. They are complementary, not competing proofs.
GOF plots are descriptive diagnostics. They do not by themselves quantify uncertainty in the plotted pattern, prove that assumptions hold, establish predictive performance in new data, or show that the model is suitable for every scientific question. Individual predictions may look favorable partly because subject-specific estimates use the observations being compared; population predictions answer a different question. Interpret each display for its actual construction and avoid extending its evidence beyond the data and prediction target shown.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A point has prediction 5 mg/L and observation 3 mg/L. Under \(r_i=y_i-\hat{y}_i\), what is its residual and where is it relative to the identity line?
- A residual-versus-time plot shows mostly negative residuals early and positive residuals late. Give one cautious interpretation and one conclusion the plot alone cannot support.
- Raw residuals are more widely spread at higher predicted concentrations. What feature of the display changes, and what does this pattern not prove?
- Why should a report identify whether an observed-versus-predicted plot uses population predictions or individual predictions?
Solutions. 1. The residual is \(3-5=-2\) mg/L, so the observation is below its prediction and the point lies below the identity line. 2. The pattern may indicate time-related mismatch in predictions; the plot alone cannot establish a particular biological mechanism or structural cause. 3. The apparent residual spread increases with prediction. This does not by itself prove that the residual-error model is wrong, because other features and limited data may contribute. 4. Individual predictions include subject-specific estimates informed by that subject’s observations, so their apparent agreement is not directly equivalent to agreement from population predictions.