Tutorials › Nonlinear Mixed-Effects Models (NLME) › Individual Prediction-Time Overlays

Goodness-of-Fit Diagnostics and Their Pitfalls · Tutorial 547 of 797

Individual Prediction-Time Overlays

Learn how to construct and interpret subject-level prediction-time overlays, recognize what they reveal and what sparse sampling can conceal, and avoid mistaking an adequate population fit for adequate individual fit.

Intermediate 10 min read

What You'll Learn

  • Define observed concentrations, individual predictions, and population predictions for a prediction-time overlay.
  • Compare the observed and predicted trajectories across pharmacokinetic phases.
  • Work through examples of peak-time ambiguity and population fit masking subject-level discrepancies.
  • Recognize how sparse sampling, shrinkage, and conditional prediction uncertainty affect interpretation.
  • Use overlays alongside other diagnostics without treating visual agreement as proof of model adequacy.

Why Overlay Predictions Against Time?

An observed-versus-predicted plot compresses a subject’s observations into a comparison of two values. A prediction-time overlay preserves their order: it shows where the model tracks a concentration profile, and where disagreement occurs along the dosing interval. That order matters in pharmacokinetics. A mismatch during absorption can have a different implication from one during the terminal decline, even if both contribute the same numerical prediction error.

An overlay is a diagnostic, not a test that certifies a model. It can reveal a trajectory that is missed even when a population-level summary looks reasonable. Conversely, a smooth curve passing near a few observations may look persuasive while being weakly informed between sparse sampling times.

Definition: For subject \(i\), let \(t_{ij}\) be the time of observation \(j\), and let \(y_{ij}\) be the observed concentration at that time. An individual prediction-time overlay plots the observations \(y_{ij}\) against time and displays the model’s predicted concentration trajectory for that subject on the same axes. A conditional individual prediction uses the estimated individual random effects, denoted \(\hat{\eta}_i\), together with the fitted population parameters, denoted \(\hat{\theta}\): \(\hat{y}^{I}_i(t)=f(t;\hat{\theta},\hat{\eta}_i)\), with the subject’s dose history and covariates supplied to \(f\). A typical-subject prediction sets the random effects to zero, \(\hat{y}^{P}_i(t)=f(t;\hat{\theta},0)\). These are different targets; neither is automatically the marginal population mean.

The plot should identify the subject, dose and time scale, concentration units, and prediction type. Plot observed concentrations at their actual sampling times and the predicted trajectory over the relevant interval. Predictions evaluated only at observation times can be shown as connected segments, but those segments should not be mistaken for a model-defined smooth trajectory. When predictions are conditional on estimated individual effects, they describe the fitted subject-specific trajectory, not a new subject’s population distribution.

Reading a Trajectory Across Pharmacokinetic Phases

For an extravascular dose, the concentration may rise during absorption, reach a peak, and then decline. A multicompartment model can also represent an early distribution decline followed by a slower terminal elimination phase. These phase labels describe pharmacokinetic behavior, not necessarily separate components that can be identified from every individual profile. In particular, a one-compartment model does not contain a separate distribution phase.

Read an overlay from left to right. Ask whether the curve and observations agree during the rise, around the peak, during any early decline, and in the terminal portion. Note the timing and direction of visible departures: for example, observations consistently above the curve over a portion of the profile. The overlay helps locate the discrepancy, but visual inspection alone does not establish its cause. A wrong structural model, an incorrect dose or sample time, unusual subject parameters, sparse data, or measurement variability can all contribute.

Worked Example: A Sampled Peak That Is Tied

Consider hypothetical concentrations for one subject after a dose. The conditional individual prediction and observed values are shown below. All concentrations are in mg/L.

Time (h)Observed \(y_{ij}\) (mg/L)Individual prediction \(\hat{y}^{I}_i(t_{ij})\) (mg/L)Observed minus predicted (mg/L)
0.50.50.7-0.2
133.5-0.5
278-1
4761
821.50.5

At each sampling time, the signed difference is \(y_{ij}-\hat{y}^{I}_i(t_{ij})\). For example, at 4 h it is \(7-6=1\) mg/L. The observed sampled maximum is 7 mg/L at both 2 and 4 h, while the predicted sampled maximum is 8 mg/L at 2 h. Therefore the observations do not show a uniquely later sampled maximum. The tied observed maximum may still motivate closer investigation of peak timing, but it cannot establish that the observed peak occurred later.

The differences also show where the curve is low or high at the sampled times: it predicts 1 mg/L above the observation at 2 h and 1 mg/L below it at 4 h. This change in direction suggests examining the region around the peak, while recognizing that only five observations are available. It does not by itself prove a systematic timing error or identify its cause.

Why a Population Fit Can Hide Individual Misfit

A population prediction summarizes a different target from a conditional individual prediction. Population summaries can appear acceptable even when subject-specific discrepancies are important, because deviations in opposite directions can cancel when averaged. The following arithmetic shows this directly; it is not an argument that every population prediction equals the average observed concentration. With nonlinear models, a typical-subject prediction and a marginal mean over random effects generally need not coincide.

Worked Example: Opposing Subject Deviations Cancel

Suppose two hypothetical subjects have observations at 1, 4, and 12 h. At those times, the typical-subject predictions are 2.5, 5, and 2.5 mg/L. The observed and conditional individual predictions are:

Time (h)Subject 1 observedSubject 1 individual predictionSubject 2 observedSubject 2 individual predictionTypical-subject prediction
14311.52.5
486235
124311.52.5

At 4 h, the mean of the two observed concentrations is \((8+2)/2=5\) mg/L, exactly the typical-subject prediction. The same is true at 1 h, where \((4+1)/2=2.5\) mg/L, and at 12 h, where \((4+1)/2=2.5\) mg/L. Thus the population prediction matches the observed mean at each displayed time.

Yet at 4 h the individual predictions are 2 mg/L below Subject 1’s observation, \(6-8=-2\), and 1 mg/L above Subject 2’s observation, \(3-2=1\). At 1 h their prediction differences are \(-1\) and \(0.5\) mg/L, respectively; at 12 h they are again \(-1\) and \(0.5\) mg/L. The opposite-signed deviations cancel in the mean. The individual curves therefore deserve scrutiny even though the displayed population fit is exact for the average. In practice, assess the full trajectories and subject-level context rather than treating a good aggregate as evidence that every subject is well described.

The cancellation can be written generally. If \(d_i=y_i-\hat{y}_i\) is a subject’s signed deviation at a given time, the mean deviation is \(\bar{d}=\frac{1}{m}\sum_{i=1}^{m}d_i\) for \(m\) subjects. For the 4 h values above, \(\bar{d}=((-2)+1)/2=-0.5\) mg/L for the individual predictions, despite the larger subject-specific deviations of \(-2\) and \(1\) mg/L. For the typical prediction, the deviations are \(8-5=3\) and \(2-5=-3\) mg/L, so their mean is zero. A small or zero average deviation does not imply small deviations for each subject.

$$ \bar{d}=\frac{1}{m}\sum_{i=1}^{m}d_i \qquad\text{can be near zero even when some } |d_i| \text{ are large.} $$

This is why individual overlays complement population-level plots. They make subject-to-subject differences visible instead of allowing opposite discrepancies to disappear in a summary. They also make clear that a conditional prediction is not a guarantee of a close fit: estimated random effects may improve a subject’s prediction, but they need not capture all of that subject’s observed trajectory.

Sampling Density and What the Curve Cannot Show

Worked Example: Close at Sample Times, Uncertain Between Them

A hypothetical subject has observed concentrations of 0.8 mg/L at 0.5 h, 6.8 mg/L at 4 h, and 1.2 mg/L at 12 h. The individual prediction at those same times is 0.6, 7.2, and 1.0 mg/L, respectively. The signed differences are:

$$ 0.8-0.6=0.2,\qquad 6.8-7.2=-0.4,\qquad 1.2-1.0=0.2\ \text{mg/L}. $$

At the three observed times, the curve is within 0.4 mg/L of the data. Suppose the fitted smooth trajectory has a predicted peak at 2.5 h. There is no observation near 2.5 h in these data, so the overlay cannot verify the predicted peak concentration or its timing. The apparent agreement at the sampled times is compatible with uncertainty about the trajectory between 0.5 and 4 h. The appropriate conclusion is limited: predictions are close at the three sampled times, but the peak region is weakly checked by these observations.

A smooth prediction line is generated by the model, not by continuous measurement. In sparse data, a visually smooth curve can create an impression of certainty that the observations do not support. When timing around a peak is important, note whether samples actually bracket that region; when terminal behavior matters, check that observations extend far enough into the decline. The overlay can expose gaps in the data’s coverage, but cannot fill them.

Assumptions, Diagnostics, and Alternatives

Interpretation assumes that the observations and predictions refer to the same subject, dose history, time origin, concentration units, and covariates. The fitted prediction must be calculated using the stated target: for an individual prediction, the subject’s estimated random effects; for a typical-subject prediction, zero random effects. A mismatch in dose timing or units can make a correct model appear wrong. Before interpreting shape, verify the data alignment and the type of prediction.

Then assess whether discrepancies concentrate in a pharmacokinetic phase or recur across several subjects. A curve that rises too slowly relative to well-timed observations, for example, may prompt review of absorption representation or recorded dose times. A curve that follows the early profile but departs in the terminal decline may prompt review of elimination behavior or sampling coverage. These are diagnostic leads, not conclusions: overlays alone cannot isolate which model component or data issue is responsible.

Conditional individual predictions are estimated from the same subject’s data being displayed. Consequently, their apparent agreement is a fit description, not an independent validation of how the model predicts a new subject. With sparse observations, estimated individual effects may be pulled toward the population distribution (shrinkage), so individual curves can look similar despite genuine differences; this limits what a visual overlay can establish about individual parameters. Prediction uncertainty is also not shown by a single curve. A narrow-looking line is not an uncertainty interval unless an appropriate interval has been calculated and displayed.

The main alternatives answer related but distinct questions. A population prediction-time overlay emphasizes typical or population-level behavior; it can hide opposing individual deviations. An individual observed-versus-predicted plot compares values without preserving their time order, so it does not locate a discrepancy in the concentration-time profile. Residual plots preserve differences from predictions and can help examine patterns across time or prediction level, while an overlay retains the original concentration scale and makes phase-specific departures easier to recognize. These displays are complementary rather than interchangeable.

Common pitfall: “The average observations follow the population curve, so the model fits every subject.” This reasoning ignores cancellation: one subject can be underpredicted while another is overpredicted by a similar amount. The correction is to inspect individual overlays as well as population summaries, and to describe the fit at the level actually shown. Also, a sampled maximum at two times is a tie; do not report it as a uniquely later observed peak.

An overlay is limited by sampling density, measurement noise, uncertainty in estimated parameters, and the fact that a fitted individual curve is conditional on the model and the observed data. It is best used to identify where a closer model or data review may be warranted, not to declare a model adequate based on visual proximity alone.

A Practical Reading Sequence

1
Confirm what is plotted.
Check the subject, dose and time alignment, units, observed points, and whether the curve is a conditional individual or typical-subject prediction.
2
Follow the profile in time.
Compare observations and prediction during the rise, around the sampled peak, and through any distribution and terminal elimination portions that the design captures.
3
Describe, then investigate.
State where the curve is above or below observations and whether that is confined to a phase. Treat possible model or data explanations as hypotheses for further checks.
4
Keep the evidence in scope.
Note whether observations are dense enough to check the feature of interest, and compare subject-level displays with population and residual diagnostics.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

  1. At one sampling time, two subjects have observed concentrations of 9 and 3 mg/L. Their mean is 6 mg/L, equal to the population prediction. Does this establish an adequate individual fit?
  2. Observed concentrations are 5 mg/L at both 2 and 4 h. What can be said about the observed sampled maximum?
  3. A conditional individual prediction is close to every observed point, but there are no samples around the predicted peak. What important limitation should be reported?
  4. Name two data-alignment details to verify before interpreting a visible mismatch between observations and a prediction curve.

Solutions. 1. No. The mean matches, but the subject-specific deviations could be substantial and opposing; inspect each subject’s trajectory. 2. The sampled maximum is tied at 2 and 4 h. These observations do not identify a unique sampled peak time. 3. Agreement at the observed times does not verify the curve between samples, including the peak’s height or timing; describe the peak region as weakly checked. 4. Examples include confirming the subject identity, dose history, time origin, dose and sample timing, covariates, concentration units, and whether the curve is conditional individual or typical-subject prediction.