Why Read an Observed-Predicted Plot?
A PBPK model can produce predictions that look plausible in isolation yet systematically miss observations in an independent dataset. An observed-predicted plot puts matched values side by side: it can reveal whether predictions tend to be too high or too low, whether disagreement appears to change with dose, and whether one unusual observation dominates the visual impression. These are diagnostic questions about the pattern of validation results, not a substitute for prespecified acceptance criteria or a causal explanation of model failure.
For this purpose, each plotted pair must come from an observation and its corresponding prediction, generated before the validation results were used to alter the model. The prediction should come from the frozen model and parameter set, using the intended inputs for that validation case. Calibration residuals answer a different question: they describe discrepancies in data used during model fitting. Mixing them with independent validation pairs can make performance appear more convincing than it is.
The pairing and scale matter. For example, a predicted concentration at one sampling time should not be paired with an observed concentration at another time, nor should a dose-normalized prediction be compared with an unnormalized observation. Before interpreting a pattern, check that the intended predictions were frozen, the records are correctly matched, and units and transformations agree. An independent validation dataset may contain repeated measurements or several participants from a study; points should not be treated as unrelated merely because they appear separately on the graph.
The Identity Line and the Choice of Scale
On ordinary linear axes, the identity line is a 45-degree line through the origin. Points above it have \(O_i>P_i\): the model predicted less than was observed. Points below it have \(O_i<P_i\): the model predicted more than was observed. Distance from the line is measured in the endpoint’s units. When the values span several orders of magnitude and are positive, logarithmic axes can make relative discrepancies easier to see; on such axes, the identity line remains the set of points with equal observed and predicted values.
The visual pattern depends on scale. If observations are consistently a fixed proportion above predictions, the vertical gaps on linear axes grow as predictions grow. That does not by itself mean the proportional disagreement is getting worse at higher values. Conversely, a pattern that looks small on a wide linear scale may be large in relative terms for low concentrations. State which axes are used, and do not compare distances on unlike scales as if they were equivalent.
The last result follows directly from taking logarithms of \(O_i=cP_i\): \(\log(O_i)=\log(cP_i)=\log(c)+\log(P_i)\). Thus, a parallel shifted band can indicate a broadly similar proportional mismatch across the displayed range. This is a visual description, not proof that the ratio is exactly constant or that one particular biological process caused it. Log axes require positive values; zero and below-quantification values cannot simply be logged without a justified handling rule.
Worked Example: A Consistent Proportional Mismatch
Suppose a hypothetical independent validation dataset contains four paired concentrations, all in the same units. The locked PBPK predictions are \(1, 2, 4,\) and \(8\), and the corresponding observations are \(1.2, 2.4, 4.8,\) and \(9.6\). Assume the records are correctly matched, the values are positive, and the plot uses the same concentration definition for both axes.
The plotted coordinates are \((1,1.2)\), \((2,2.4)\), \((4,4.8)\), and \((8,9.6)\). In each pair, the observation is 20% above the prediction because \(O_i/P_i=1.2\). For example, the first vertical discrepancy on linear axes is \(1.2-1=0.2\), while the last is \(9.6-8=1.6\), in concentration units.
All points lie above the identity line, and their linear vertical gaps increase with the prediction. On log-log axes, however, they satisfy \(\log(O_i)=\log(P_i)+\log(1.2)\), so they lie on a line parallel to the identity line. The pattern is consistent with a proportional underprediction across these four values, not evidence that relative error increases with concentration. Four hypothetical points are not enough to establish a general biological explanation or to determine whether prespecified validation criteria are met.
Looking for Dose-Related Patterns
A plot can help assess whether discrepancies are associated with dose, but dose is not automatically represented by either axis. If the scientific question concerns dose extrapolation, annotate points by dose or show separate panels for prespecified dose groups. Then examine whether points at one dose, or across increasing doses, sit systematically on different sides of the identity line. If points are colored by dose, include a clear legend; if doses overlap in the same plotting area, facets may be easier to interpret.
A dose-related pattern is not the same as a pattern caused by concentration alone. Dose, predicted concentration, formulation, study, sampling time, or participant characteristics may vary together. A plot can flag that mismatch is associated with a dose grouping in the validation data; it cannot isolate dose as the cause when other conditions differ systematically. Comparisons should therefore be read alongside the study design and the intended prediction question.
Worked Example: A Pattern Across Dose Groups
Consider a hypothetical independent validation with three dose groups. Each point is a correctly paired concentration in the same units and at the same prespecified sampling time. Assume the model and parameters were frozen before these observations were examined.
| Dose group | Prediction \(P_i\) | Observation \(O_i\) | Position relative to identity |
|---|---|---|---|
| Low | 2.0 | 2.1 | Above |
| Middle | 5.0 | 5.2 | Above |
| High | 12.0 | 17.0 | Above, farther away |
On linear axes the corresponding vertical discrepancies are \(2.1-2.0=0.1\), \(5.2-5.0=0.2\), and \(17.0-12.0=5.0\) concentration units. The high-dose point is visibly farther above the identity line. Its proportional comparisons are \(2.1/2.0=1.05\), \(5.2/5.0=1.04\), and \(17.0/12.0\approx1.42\). In this small hypothetical set, the high-dose observation is not merely farther away because its prediction is larger; it is also more elevated relative to its prediction.
The appropriate visual conclusion is that this example raises a concern about greater underprediction in the high-dose group. It does not establish a dose mechanism: the high-dose case might also differ in study, formulation, or another relevant condition. Showing dose labels and checking design comparability are necessary before describing the pattern as dose-associated. The points alone do not identify which PBPK assumption or parameter is responsible.
Influential Observations: Investigate, Do Not Erase
An observation can be influential on the visual reading of a plot even when no regression is fitted. A point far from the main cloud may expand the axis limits so that the remaining points appear almost indistinguishable from the identity line. A point at an extreme prediction may also determine how the range of model performance is perceived. This visual influence is different from leverage or influence in a fitted regression model, and the plot should not be interpreted as though a regression line had been estimated.
First inspect the full dataset and confirm that the unusual point is not a matching, unit, transcription, or simulation-input error. Check the original record, the prediction file, and the prespecified analysis rules. If helpful, add a clearly labeled zoomed view of the dense region or show the full-range plot beside dose- or study-stratified panels. A zoomed panel aids inspection; it does not replace the full plot or make the extreme observation disappear from the validation evidence.
Worked Example: One Point Compresses the Main Cloud
A hypothetical validation has six paired values in common concentration units: \((P_i,O_i)=(2,2.1),(4,3.9),(6,6.2),(8,7.8),(10,10.1)\), and \((30,55)\). The first five lie close to the identity line. For the last point, the linear vertical discrepancy is \(55-30=25\) concentration units, much larger than the discrepancies among the first five.
On a plot scaled to include predictions up to 30 and observations up to 55, the first five points occupy a relatively small part of the axes. Their proximity to the identity line may be difficult to judge. The sixth point is therefore visually influential both because it is far from agreement and because it determines much of the display range.
A sound presentation retains the full-range plot and may add a labeled zoom of the region containing the first five points. The full plot communicates the large discrepancy; the zoom makes the rest of the pattern legible. The unusual point should be checked against source data and prediction-generation records, but it should not be discarded solely because it changes the picture. Exclusion is justified only by a prespecified, documented data-quality or eligibility rule applied consistently, not by an unfavorable validation result.
Assumptions, Diagnostics, and Complementary Displays
Reading these plots is most useful when the following conditions hold: observations and predictions are paired to the same target; predictions were generated independently of the outcomes being assessed; units and endpoint definitions match; and the plotting scale is appropriate for the values. If the plot pools several studies or dose groups, those labels must remain available so that a pooled pattern does not conceal distinct subgroups. Visual patterns are exploratory descriptions unless an analysis plan specifies formal criteria for evaluating them.
A practical diagnostic sequence is to start with the full-range plot and identity line, then inspect direction of departure, scale dependence, and grouping. Check whether points consistently lie above or below the line, whether the apparent departure changes on a log scale, whether groups occupy different regions, and whether one observation controls the axes. Recheck the pairing and source records whenever a point is surprising. A plot cannot determine whether discrepancies arise from assay noise, protocol differences, structural model limitations, or incorrect inputs.
Confirm the endpoint, time, dose, units, and validation case for every observed-predicted pair; use the locked prediction file.
Describe whether points tend to be above, below, or near agreement, and state the plotted scale and units.
Use dose or study labels, and check whether a small number of points controls the display or the apparent pattern.
Report the visible pattern and its context, then distinguish what the plot suggests from what it cannot establish.
An observed-predicted plot is not the only useful display. A residual-versus-prediction display can make the signed discrepancy easier to inspect around zero, while the observed-predicted plot preserves the direct comparison with the identity line. A log-log version can clarify proportional patterns when all values are positive. Dose-stratified panels can expose group differences hidden by pooling. These views answer complementary questions; none turns an independent validation into calibration data or replaces prespecified evaluation criteria.
A frequent mistake is to fit a line through the observed-predicted points and declare the model adequate because the fitted slope is near one or the points correlate strongly. Correlation measures how values vary together, not whether they agree: observations can track predictions closely while remaining systematically above the identity line. Likewise, an attractive pooled cloud can conceal dose-specific disagreement. The correction is to show the identity line explicitly, retain relevant grouping information, and describe the plotted departures rather than treating correlation or a fitted trend as a test of agreement.
Check Your Understanding
Answer each question using the plot interpretation principles above. The solutions follow the questions.
- On a linear observed-predicted plot, what does a point below the identity line mean?
- Positive observations are consistently 1.5 times their predictions. How would this pattern appear on log-log axes?
- Why is it not enough to say that high-dose points are farther from the identity line when assessing a dose-related pattern?
- One unusual point expands the axes and compresses the rest of the cloud. What should the analyst do, and what should the analyst avoid?
- Why does a strong correlation between observations and predictions not establish agreement?
Solutions. (1) The prediction exceeds the observation, so the point is below the identity line. (2) The points follow \(\log(O_i)=\log(P_i)+\log(1.5)\), a line parallel to the identity line and above it. (3) Linear vertical gaps can grow with the size of the prediction even when proportional disagreement is unchanged; compare an appropriate scale and consider dose labels and study conditions. (4) Retain the full-range plot, investigate the record, and optionally add a labeled zoom or stratified display; do not remove the point merely because it changes the visual impression. (5) Correlation describes co-variation, not equality: predictions and observations can correlate strongly while remaining systematically biased relative to the identity line.