Observed Versus Predicted Plots At Population And Individual Levels

What an Observed-versus-Predicted Plot Shows

An observed-versus-predicted plot puts each measured response beside the model prediction for that same record. It is a direct visual check of agreement: if predictions match observations well, points should lie near the line where observed and predicted values are equal. The plot is useful at two distinct levels in a nonlinear mixed-effects model (NLME): using population predictions, which do not use an individual’s estimated random effects, and using individual predictions, which do.

Let \(Y_{ij}\) denote the observed response for individual \(i\) at record \(j\), and let \(t_{ij}\) denote the corresponding time. A general NLME model writes the expected response as a function \(f\) of the record’s conditions, population parameters \(\theta\), and individual random effects \(b_i\), with observation error added according to the residual model. For a given fitted model, a population prediction is typically calculated at the estimated population parameters with the random effects set to their typical value, often zero. An individual prediction uses an estimate \(\hat b_i\) of that individual’s random effects. Exact software definitions can vary, so check how the predictions were generated before interpreting a display.

Here PRED denotes a population prediction and IPRED an individual prediction. Both are paired with the same observation \(Y_{ij}\), and both must be on the same response scale and in the same units as that observation. For example, a concentration recorded in mg/L must be plotted against a prediction in mg/L, not against a log-concentration prediction. A plot with prediction on the horizontal axis and observation on the vertical axis is conventional. The line \(y=x\), called the identity line, marks exact agreement: a point on it has observed value equal to predicted value.

The sign has a practical interpretation. If observations tend to sit above the identity line, predictions tend to be too low (underprediction); if they tend to sit below it, predictions tend to be too high (overprediction). A smooth trend through the points helps reveal systematic disagreement across the range of predictions. If \(m(p)\) is the mean observed response at prediction value \(p\), the ideal relationship is \(m(p)=p\). A smooth curve that departs consistently from the identity line suggests that the discrepancy changes with prediction value. A smoother is descriptive: it does not, by itself, provide a formal test or account for dependence among repeated records from the same individual.

Population and Individual Displays Answer Different Questions

A population-prediction plot asks whether the model’s typical individual prediction agrees with observations across the records being displayed. It is useful for assessing how the population model performs without using estimated individual random effects to tailor predictions to each person. However, because the model is nonlinear, the prediction at \(b_i=0\) is generally not the same as the average prediction over the random-effects distribution. A population-prediction plot should therefore not automatically be read as a plot of marginal population means.

An individual-prediction plot asks how well the fitted model represents observations after using each individual’s estimated random effects. It will often look closer to the identity line because those estimates are obtained using that individual’s data. Consequently, good alignment in an IPRED plot is not independent validation of the model’s ability to predict a new individual. The contrast between the two plots is informative: a clear improvement from PRED to IPRED indicates that individual-specific estimates account for some discrepancy, but does not establish that the model’s population structure or error assumptions are adequate. The earlier material on population and individual fit targets distinguishes these two aims; here, the focus is on reading their paired-value displays.

The plot’s visual evidence depends on how records are pooled. Individuals with more observations contribute more points, and repeated observations from one individual are not independent. A dense region can dominate the appearance of a pooled smooth, while sparse regions provide little information about the trend. Consider the distribution of predictions and the number of individuals represented, not just the total number of points. Plotting the two prediction types side by side is generally more informative than using one to stand in for the other.

Why Correlation Is Not Agreement

Correlation measures the strength of a linear association, not whether observed and predicted values are equal. In particular, adding a constant to every observation leaves the correlation with predictions unchanged, even though it shifts every point away from the identity line. Multiplying all predictions by a positive constant can also preserve a perfect correlation while changing the scale of disagreement. For agreement, inspect position relative to \(y=x\), not merely whether the points form a narrow upward-sloping band.

A useful companion to the visual check is the mean signed difference, calculated here with the convention \(Y-P\). It summarizes the average direction of discrepancy in the displayed records, though it can be near zero when positive and negative differences cancel. A smooth trend can reveal that kind of changing pattern more clearly, while the plot itself shows the spread of individual points. These summaries support the visual assessment; none replaces checking the model and data context.

Reading a Smooth Trend and Checking Its Limits

A smooth trend line can reveal departures from agreement that are not captured by one overall offset. For instance, a trend that starts near the identity line and then increasingly rises above it indicates that observations exceed predictions by more as prediction values increase. A curve that bends across the range suggests that a single constant correction would not describe the discrepancy well. Always compare the smooth to the identity line, not just to a horizontal or vertical reference.

Before interpreting either display, check that each observation is paired with the prediction from the same record, that the units and response scale match, and that the prediction type is correctly identified. A mismatch in record alignment can create an apparently poor fit even when predictions are otherwise reasonable. If the response was modeled on a transformed scale, make sure the plotted prediction has been converted consistently before comparing it with observations on the original scale. Also inspect the range: a tight cluster can hide behavior at the edges, while a few extreme prediction values can strongly affect the apparent trend.

Observed-versus-predicted plots complement, rather than replace, residual displays. A residual-versus-prediction plot uses the discrepancy \(Y-P\) as its vertical coordinate and makes direction and size of errors explicit; the earlier material on residuals against predictions develops that view. The observed-versus-predicted plot emphasizes agreement with the identity line and makes broad departures in scale or curvature visually direct. Neither display alone establishes that the model is correctly specified. These plots are descriptive, and pooled points do not supply an inferential procedure that accounts for within-individual dependence.

Common Pitfalls

• Treating high correlation as good agreement. A straight band can be shifted or rescaled away from the identity line. Check the line of equality and the direction and size of departures.
• Calling an IPRED plot independent validation. Individual predictions use estimates informed by that individual’s observations. Their closer fit is expected to some degree and is not equivalent to predicting new individuals.
• Misreading the axes or sign. With predictions on the horizontal axis and observations on the vertical axis, points above the identity line indicate underprediction, not overprediction.
• Overinterpreting a smooth in a sparse region. A curve where few records occur is weak visual evidence. Check point density and avoid treating a descriptive smoother as a formal test.
• Ignoring repeated records and unequal sampling. A pooled display gives more visual weight to individuals with more records. A pattern across points need not represent a pattern across equally weighted individuals.