Why Compare Both Prediction Types?
A concentration model can be compared with observed concentrations in two related ways. Population predictions show what the model predicts for a subject using the population parameter estimates and that subject’s recorded dose and covariate history. Individual predictions also use subject-specific estimates of the random effects. Comparing both helps separate a mismatch in the population-level description from patterns that remain after subject-specific information is incorporated.
The comparison is most informative when the two prediction types are paired with exactly the same observed records. A prediction attached to the wrong subject, time, dose occasion, or concentration unit can create an apparent model problem—or hide a real one. The task is therefore not simply to decide which scatterplot looks closer to the identity line. It is to verify the pairing, examine the direction and size of discrepancies, and check whether numerical summaries agree with the plotted patterns.
For a given record, define the signed prediction errors as observed minus predicted concentration:
A positive error means the observed concentration exceeds the prediction; a negative error means it is below the prediction. Both errors have the concentration unit, such as mg/L. A zero error means exact agreement for that record, not that the model is generally adequate.
Pair Records Before Interpreting Patterns
A valid comparison requires that each observed value and both predictions refer to the same subject and observation occasion. Check subject identifiers, event records, observation times, dose histories, covariates, units, and any data exclusions. These checks are especially important when outputs are sorted differently or when records have been joined across files. A row-by-row subtraction is meaningful only after the rows have been matched by their identifiers and occasion.
Observed concentrations below a quantification limit need separate attention. A reported limit is not an exact concentration measurement, so treating it as an ordinary numeric \(DV\) and calculating an ordinary signed error can be misleading. Confirm whether the record is censored and how it was represented in the model output; assess it using the model’s censoring treatment rather than silently substituting a limit value as if it were observed exactly.
Once records are paired, inspect at least two complementary views: observed concentration against prediction, with the identity line \(y=x\), and signed errors against prediction or time. A point above the identity line has \(DV>PRED\) (or \(DV>IPRED\)); a point below it has the reverse. Error plots make the direction of discrepancy easier to see and can reveal changes with concentration or time. Where useful, display subject identity or inspect separate subject-level traces. A pooled plot alone can conceal opposite patterns in different subjects.
Worked Example: Calculate and Compare Paired Errors
Suppose two hypothetical subjects have the following concentrations, all in mg/L. For each record, calculate both signed errors and compare the pooled summaries.
| Subject | Time (h) | \(DV\) (mg/L) | \(PRED\) (mg/L) | \(IPRED\) (mg/L) |
|---|---|---|---|---|
| A | 1 | 8.4 | 7.0 | 8.0 |
| A | 4 | 5.1 | 4.8 | 5.0 |
| B | 1 | 3.2 | 4.0 | 3.5 |
| B | 4 | 2.0 | 2.5 | 2.1 |
For subject A at 1 hour, \(e^P=8.4-7.0=1.4\) mg/L and \(e^I=8.4-8.0=0.4\) mg/L. Applying the same subtraction to all records gives population errors \(1.4,\ 0.3,\ -0.8,\ -0.5\) mg/L and individual errors \(0.4,\ 0.1,\ -0.3,\ -0.1\) mg/L.
The pooled mean signed errors are
The mean absolute errors are \((1.4+0.3+0.8+0.5)/4=0.750\) mg/L for \(PRED\), and \((0.4+0.1+0.3+0.1)/4=0.225\) mg/L for \(IPRED\). In this small example, individual predictions have smaller pooled absolute error. But the pooled signed errors hide subject-level direction: subject A’s mean population error is \((1.4+0.3)/2=0.850\) mg/L, while subject B’s is \((-0.8-0.5)/2=-0.650\) mg/L. The errors point in opposite directions. The pooled mean near zero is therefore not evidence that population predictions are unbiased for each subject.
Use the Relationship Between the Two Error Sets
The two sets of errors are linked record by record. Subtracting the individual error from the population error gives:
This identity is a useful cross-check. If a table, plot, or summary reports the two prediction errors and the difference between predictions, the arithmetic must satisfy it for every paired record. A failure suggests a problem such as a mismatched record, inconsistent units, or an incorrect variable definition—not a subtle statistical effect.
Individual predictions often fit the observations more closely because subject-specific random-effect estimates are informed by those subjects’ data. This is not a guarantee of smaller error at every record: the model, estimation method, residual-error structure, and individual estimation procedure matter. Even when individual predictions are closer overall, their apparent advantage is not independent validation. They incorporate subject-specific information from the same data being assessed. Moreover, when individual random effects are weakly informed, shrinkage can pull their estimates toward the population, making \(IPRED\) resemble \(PRED\).
Worked Example: Detect Opposing Subject-Level Bias
Consider six hypothetical measurements from two subjects. All concentrations are in mg/L. Calculate each subject’s mean signed population error and mean absolute population error.
| Subject | Population errors across observations (mg/L) | Individual errors across observations (mg/L) |
|---|---|---|
| C | 1.0, 0.8, 1.2 | 0.2, 0.1, 0.3 |
| D | -0.7, -0.9, -0.8 | -0.1, -0.2, -0.1 |
For C, the mean signed population error is \((1.0+0.8+1.2)/3=1.00\) mg/L, and the mean absolute error is also \(1.00\) mg/L. For D, the mean signed error is \((-0.7-0.9-0.8)/3=-0.800\) mg/L; the mean absolute error is \((0.7+0.9+0.8)/3=0.800\) mg/L. Thus, population predictions are consistently low for C and consistently high for D in these records.
The corresponding mean individual errors are \(0.200\) mg/L for C and \((-0.1-0.2-0.1)/3=-0.133\) mg/L for D. These are smaller in magnitude, but the remaining directions are still visible. The pattern could be compatible with subject-specific differences being captured partly by random effects; it does not by itself establish an implementation error. Review the subject-level plot alongside the model specification, data, and random-effects behavior before assigning a cause.
Cross-Check Displays and Summaries
No single display answers every review question. An observed-versus-\(PRED\) plot assesses population-level agreement across the range of concentrations. An observed-versus-\(IPRED\) plot asks how well the model’s subject-specific fitted predictions track those observations. Signed-error plots emphasize direction, while subject-level traces show whether a pooled pattern is shared or driven by a few subjects. Numerical summaries can support these views, but their definitions and grouping must be explicit.
A short derivation explains why pooled signed error can be deceptive. If subject \(i\) has \(n_i\) observations and mean signed error \(\bar e_i\), then the pooled mean is
This is a sample-size-weighted average of subject means. Positive and negative subject-level errors can cancel, and subjects with more observations contribute more weight. Therefore, report or inspect subject-level patterns when the review question concerns consistent subject-specific mismatch. A mean absolute error, calculated as \(\sum_{i,j}|e_{ij}|/N\) for \(N\) records, avoids cancellation of signs, but it too is pooled and can be dominated by subjects with many observations.
Worked Example: Reconcile a Plot With a Summary
Suppose a hypothetical output contains three records, in mg/L: \(DV=(10,6,3)\), \(PRED=(8,5,2.5)\), and \(IPRED=(9.5,5.5,2.8)\). Calculate the errors and verify the relationship between them.
The population errors are \(10-8=2.0\), \(6-5=1.0\), and \(3-2.5=0.5\) mg/L. The individual errors are \(10-9.5=0.5\), \(6-5.5=0.5\), and \(3-2.8=0.2\) mg/L. The prediction differences \(IPRED-PRED\) are \(1.5,\ 0.5,\ 0.3\) mg/L. The error differences are \(2.0-0.5=1.5\), \(1.0-0.5=0.5\), and \(0.5-0.2=0.3\) mg/L, so the identity holds at every record.
The mean absolute errors are \(3.5/3=1.17\) mg/L for \(PRED\) and \(1.2/3=0.400\) mg/L for \(IPRED\). A display claiming these same records have a larger mean absolute individual error would be inconsistent with the listed values. The reviewer should check the plotted data, filtering, pairing, and summary calculation before interpreting the disagreement as a model feature.
Diagnostics, Alternatives, and Limitations
Patterns worth investigating include a consistent positive or negative error across prediction range, increasing error magnitude with concentration, time-dependent runs of errors, and subject-specific offsets or trends. Such patterns are signals for review, not diagnoses of a particular cause. For example, a time pattern could reflect misspecified structural behavior, an incorrectly aligned time variable, or a dosing-history problem. Check against the model specification and input records rather than inferring a cause from the plot alone.
Compare population and individual displays using the same observations, axis limits, units, and inclusion rules. Different filters or scales can make one plot appear better simply because it shows a different subset or compresses discrepancies. If one display looks inconsistent with its paired error summary, recompute a small selection of rows directly from the source values. Large discrepancies concentrated at particular records may point to data handling or prediction alignment; a broad pattern across many records may warrant a model-level review. Neither pattern proves its cause without further checks.
Residual diagnostics, reviewed in the earlier material on residual diagnostics, offer complementary ways to assess error patterns and should not be replaced by observed-versus-prediction plots. Predictive checks answer a different question by comparing observed summaries with distributions of simulated data. No prediction comparison alone establishes model adequacy, and a good-looking individual-prediction plot is especially limited as evidence because the individual estimates use subject data. These plots are diagnostic tools, not formal tests with a universal acceptance threshold.
A Practical Review Sequence
Verify the software meaning of \(PRED\) and \(IPRED\), then match each prediction to its observed subject, time, dose history, and units.
For selected records, calculate \(DV-PRED\) and \(DV-IPRED\); check that their difference equals \(IPRED-PRED\).
Compare identity-line plots, signed-error patterns, and subject-level displays using consistent records and scales.
Check whether pooled means hide cancellation or unequal subject contributions, and remember that individual predictions are fitted using subject-specific information.
Check Your Understanding
Answer each question, then compare with the solutions below.
- An observation is \(DV=6.2\) mg/L, with \(PRED=5.7\) mg/L and \(IPRED=6.0\) mg/L. Calculate both signed errors and verify their difference identity.
- Two subjects have mean signed population errors of \(+1.1\) mg/L and \(-1.1\) mg/L, with equal numbers of observations. What is the pooled mean signed error, and what important information does it omit?
- Why does a smaller mean absolute error for individual predictions not independently validate the model?
- A prediction plot shows a point far from the identity line, but a table summary reports very small errors for the same record. Name checks to make before interpreting the discrepancy.
Solutions. (1) \(e^P=6.2-5.7=0.5\) mg/L and \(e^I=6.2-6.0=0.2\) mg/L. Their difference is \(0.3\) mg/L, equal to \(IPRED-PRED=6.0-5.7=0.3\) mg/L. (2) The pooled mean signed error is \((1.1-1.1)/2=0\) mg/L. It omits the opposing subject-level patterns, which cancel in the pooled result. (3) Individual predictions use subject-specific estimates informed by the same observations, so closer fit is not an independent prediction check. (4) Verify record matching, identifiers and times, variable definitions, units, filtering, axis scales, and the summary calculation; recompute the row-level errors from the source values.