Why Read Both Panels?
A goodness-of-fit panel places measured values beside values generated by a fitted model. In pharmacometrics, the observed-versus-individual-prediction and observed-versus-population-prediction panels provide two related but distinct views of model agreement. The first uses subject-specific estimates; the second asks how the population model predicts without those individual effects. Reading them together helps identify systematic disagreement and unusual observations, while keeping clear what a visual diagnostic can establish.
Each point must pair an observed value with the prediction for the same record, using the same units and the same time, dose, and covariate information. A mismatch in records or prediction definitions can create a pattern that looks like model failure. Before interpreting either panel, check the axis labels, prediction definitions, units, and reference line. The point cloud is descriptive evidence about the fitted model and these data; it is not, by itself, a formal test or proof that the model is adequate.
The Identity Line and Departures From It
In either panel, the horizontal axis is the relevant prediction and the vertical axis is the observed value. The reference line \(y=x\), called the identity line, marks exact agreement: a point on it has an observed value equal to its prediction. A point above the line has an observed value greater than predicted; a point below it has an observed value less than predicted. This direction is easy to reverse if the axes are not checked.
For a record with observed value \(y\) and prediction \(x\), define its signed vertical difference as \(d=y-x\). This is a simple way to describe the point's position relative to the identity line. A consistently positive difference indicates underprediction on average in that region; a consistently negative difference indicates overprediction. Scatter around the line indicates disagreement that varies across records. The magnitude of the differences should be judged in context: a difference of 1 mg/L may be important near a target concentration but small at much higher concentrations.
A short justification explains why different patterns matter. Suppose the average observed value at prediction level \(x\) follows a descriptive straight-line relation \(E(Y\mid x)=a+bx\). Then the average vertical difference is
Thus a nonzero \(a\) corresponds to a constant offset when \(b=1\), while \(b\ne1\) makes the average difference change with prediction level. This is a way to understand visual patterns, not a requirement that the data follow a straight-line model. Curvature, clusters, or changing scatter may also be visible. A cloud that roughly follows the identity line is encouraging, but it does not rule out patterns hidden by dense points, a narrow range, or an unsuitable scale.
Worked Example: Reading an Offset in an Individual-Prediction Panel
Suppose a hypothetical individual-prediction panel contains four records, with predictions and observations in mg/L: \((1.0,1.2)\), \((2.0,2.4)\), \((4.0,4.7)\), and \((8.0,9.0)\). Compute the signed differences and describe the pattern.
For the first record, \(d=1.2-1.0=0.2\) mg/L. For the second, \(d=2.4-2.0=0.4\) mg/L. For the third, \(d=4.7-4.0=0.7\) mg/L. For the fourth, \(d=9.0-8.0=1.0\) mg/L. All four points are above the identity line, so the predictions are lower than the corresponding observations for these records.
The mean signed difference is \((0.2+0.4+0.7+1.0)/4=0.575\) mg/L. That summary describes the average vertical offset in this small hypothetical set. The increasing differences also suggest that the discrepancy may grow with the prediction level rather than being a constant offset. Four points are too few to establish a general pattern, and the plot cannot tell us whether the cause is structural-model misspecification, an observation-model issue, influential data, or chance variation.
Why the Two Panels Can Look Different
Population predictions and individual predictions answer different diagnostic questions. A population prediction reflects the typical model prediction for the record's covariates and design, without a subject-specific random effect. It can therefore reveal whether the population model describes the observations across subjects. An individual prediction includes subject-specific information estimated from that individual's data. It can show how closely the model can describe observations after those estimates are used.
Consequently, closer agreement in the individual-prediction panel is not surprising, but it is not independent confirmation of predictive performance. The subject-specific estimates are informed by the observations being plotted, so the panel is conditional on data that contributed to fitting those estimates. If individual predictions also show a clear trend or large discrepancies, that deserves attention; if they look close to the identity line, the population panel still matters. A model can describe individual profiles reasonably after adaptation while its population predictions systematically miss across subjects.
Worked Example: Comparing Population and Individual Predictions
Consider six hypothetical concentration records in mg/L. Their population predictions are \(2, 2, 5, 5, 8, 8\); their individual predictions are \(1.6, 2.5, 4.5, 5.8, 7.0, 9.4\); and their observed values are \(1.4, 2.7, 4.2, 6.1, 6.2, 10.1\). The paired records are in the same order in all three lists.
The population-prediction differences \(y-\mathrm{PRED}\) are \(-0.6, 0.7, -0.8, 1.1, -1.8,\) and \(2.1\) mg/L. Their mean absolute difference is \((0.6+0.7+0.8+1.1+1.8+2.1)/6=1.183\) mg/L. The individual-prediction differences \(y-\mathrm{IPRED}\) are \(-0.2, 0.2, -0.3, 0.3, -0.8,\) and \(0.7\) mg/L, with mean absolute difference \((0.2+0.2+0.3+0.3+0.8+0.7)/6=0.4167\) mg/L.
For these hypothetical records, individual predictions are closer overall by this descriptive summary. The population differences alternate in sign, so the example does not show a uniform overprediction or underprediction. The smaller individual-prediction differences indicate improved agreement after using subject-specific estimates for these data; they do not establish that new individuals will be predicted more accurately. Because observations from the same subject can be related, the six records should not automatically be treated as six independent pieces of evidence.
Dispersion, Unusual Points, and Diagnostics
Dispersion is the vertical spread of points around the identity line. Some spread is expected when observations vary around model predictions. The panel is concerning when the spread is large relative to the response scale or changes markedly across the prediction range. For example, a narrow cloud at low predictions that fans out at high predictions suggests that discrepancies are not equally variable across the range. This visual observation does not identify the correct error model; it flags a feature to investigate with the model, data, and other diagnostics.
An unusual point is one that is far from the general cloud or identity line, or that lies in a sparse region where it can strongly affect the visual impression. Its signed difference and the scale of the response help describe how unusual it is. But unusual is not synonymous with influential. A point may have a large observed-predicted difference without substantially changing fitted parameters. Conversely, a point at an extreme prediction level may have substantial leverage on a fit even if its vertical difference is modest. Influence concerns how much the fitted result changes when an observation or subject is removed or otherwise perturbed; a panel alone cannot establish that change.
Worked Example: A Point That Warrants an Influence Check
A hypothetical population-prediction panel has five records in mg/L: predictions \(0.5, 1.0, 2.0, 4.0, 8.0\) and observations \(0.5, 1.1, 2.1, 4.2, 15.0\). Calculate the differences and assess the last point.
The signed differences are \(0.5-0.5=0.0\), \(1.1-1.0=0.1\), \(2.1-2.0=0.1\), \(4.2-4.0=0.2\), and \(15.0-8.0=7.0\) mg/L. The mean signed difference is \((0+0.1+0.1+0.2+7.0)/5=1.48\) mg/L. Excluding the last point for this descriptive comparison only, the mean difference is \((0+0.1+0.1+0.2)/4=0.100\) mg/L.
The final point is far above the identity line and shifts the overall mean difference substantially. It should prompt checks of the record, units, timing, data processing, and model fit. It is not enough to label it influential: that would require assessing how the fitted results change under an appropriate refit or sensitivity analysis. Removing it solely because it looks extreme would risk concealing real model failure or unusual but valid data.
A Practical Reading Sequence
A consistent reading sequence reduces avoidable errors. It also makes reports easier to review across models and runs. First confirm that axes and units are as expected and that the plotted predictions are defined consistently. Then read the location and shape of the point cloud relative to the identity line, comparing the population and individual panels without treating them as interchangeable.
Confirm observed values and prediction type, axis assignment, units, record matching, and whether points are repeated measurements from the same subject.
Check that the identity line is present and that the axes use compatible scales. Equal units alone do not guarantee that different axis limits make distances visually comparable.
Note its position, spread, possible change in spread, and any curvature or clustering. Use “above” or “below” the identity line to state the direction of discrepancy.
Ask whether subject-specific predictions reduce visible disagreement, then investigate unusual records and patterns using appropriate complementary diagnostics and model checks.
Common Mistakes and Limitations
- Reversing the axes. A point above the identity line means observed exceeds predicted only when observed is on the vertical axis and prediction is on the horizontal axis. Read the labels first.
- Calling a close individual-prediction panel proof of population fit. Individual predictions use subject-specific estimates informed by the data. They answer a conditional fit question, not the same question as population predictions.
- Calling every distant point influential. Distance describes discrepancy; influence requires checking the effect of that record or subject on the fitted results.
- Ignoring repeated observations. Many points may come from a small number of subjects. The plot shows records, not necessarily independent subjects, and a dense subject profile can dominate the picture.
- Over-interpreting the identity line. A visually close cloud can hide systematic patterns, and a visible discrepancy does not identify its cause. Examine data integrity and use complementary diagnostics before drawing a model-level conclusion.
- Comparing panels with misleading scales. Different axis limits or aspect ratios can alter the apparent distance from the line. Use clearly labelled, compatible scales and report settings consistently.
These panels are useful summaries, not complete validation. They can reveal broad calibration problems, dispersion, and records deserving closer review, but visual inspection is subjective and can be obscured by overplotting, sparse data, or a narrow range of predictions. Interpret patterns in context and keep the distinction between a population-level fit and a subject-conditional fit explicit.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- On a panel with observed values on the vertical axis and predictions on the horizontal axis, what does a point below the identity line indicate?
- A hypothetical record has a population prediction of 3.0 mg/L, an individual prediction of 3.8 mg/L, and an observed value of 4.1 mg/L. Calculate both signed differences and state which prediction is closer for this record.
- Why does a close observed-versus-individual-prediction panel not by itself establish good population prediction for new subjects?
- One point lies far from the identity line at the highest prediction level. What can be concluded from the panel, and what further check is needed before calling it influential?
Solutions. (1) The observed value is less than the prediction, so the point lies below the line. (2) The population difference is \(4.1-3.0=1.1\) mg/L; the individual difference is \(4.1-3.8=0.3\) mg/L. The individual prediction is closer for this record. (3) Individual predictions use subject-specific estimates informed by that subject's observations, so their apparent agreement is conditional and is not a direct assessment of prediction for new subjects. (4) The point has a large observed-predicted discrepancy and merits data and model checks. Influence requires assessing how the fitted results change when the record or subject is appropriately omitted or perturbed, not simply judging its distance from the line.