Why Compare Individual and Population Predictions?
A PD model can appear to fit each subject well while making poor predictions for the population, or it can predict the population pattern reasonably while fitting some subjects poorly. These are different questions. Comparing individual fitted trajectories with population-level predictions helps show whether the apparent fit depends on subject-specific parameter estimates and whether those estimates conceal a systematic population pattern.
For subject \(i\) at observation time \(t_{ij}\), write the model prediction as \(f(t_{ij};\theta,\eta_i)\). Here, \(\theta\) denotes population parameters and \(\eta_i\) denotes subject-specific random effects. The observed response is \(y_{ij}\). This notation separates two sources of model variation: shared population structure and the subject’s deviation from that structure.
These definitions matter because “population prediction” can refer to different quantities. The typical-subject prediction describes the model at the central random-effects value, \(\eta=0\). The marginal prediction averages over the population distribution of random effects. In a nonlinear model, they need not be equal. Neither prediction includes an individual’s estimated random effect; the marginal prediction averages over possible random effects rather than conditioning on a particular subject’s data.
Individual predictions are useful for examining how well the fitted model describes each observed trajectory. But the \(\hat{\eta}_i\) values are estimated using that subject’s data. Consequently, individual fits are conditional and generally in-sample: their closeness to the observations is not, by itself, evidence that the model predicts new subjects well. Population-level predictions provide a complementary check on the shared structure.
What the Difference Between Predictions Shows
The relationship between the two residuals can be expressed directly. Define the individual residual as \(r^{\mathrm{ind}}_{ij}=y_{ij}-f(t_{ij};\hat{\theta},\hat{\eta}_i)\), and the typical-subject residual as \(r^{\mathrm{typ}}_{ij}=y_{ij}-f(t_{ij};\hat{\theta},0)\). Adding and subtracting the individual prediction gives:
Thus, a small individual residual can coexist with a large typical-subject residual when a subject’s fitted random effect moves the prediction substantially away from the population’s typical prediction. If many subjects require similar time- or response-dependent adjustments, the individual fits may look convincing while the shared population trajectory misses a pattern.
A useful comparison therefore asks more than “Which prediction is closer?” Examine the size and direction of individual-to-population prediction differences, and whether population-level residuals show a consistent pattern across time, predicted response, exposure, baseline, or covariates. The earlier material on observed-predicted PD plots and calibration provides related tools for checking agreement; this comparison adds the distinction between conditional subject fits and population predictions.
Worked Example: A Good Individual Fit Does Not Establish Population Fit
Suppose a hypothetical subject has three observed PD responses, in response units, at three scheduled times. The fitted typical-subject predictions are 7, 9, and 11 units, while the individual fitted predictions are 7.8, 10.1, and 12.1 units. The observations are 8, 10, and 12 units.
| Observation | Observed \(y\), units | Typical prediction, units | Individual prediction, units |
|---|---|---|---|
| 1 | 8.0 | 7.0 | 7.8 |
| 2 | 10.0 | 9.0 | 10.1 |
| 3 | 12.0 | 11.0 | 12.1 |
The typical-prediction residuals are \(8-7=1\), \(10-9=1\), and \(12-11=1\) response unit. Their root mean squared error is:
The individual-prediction residuals are \(8-7.8=0.2\), \(10-10.1=-0.1\), and \(12-12.1=-0.1\) response unit. Their root mean squared error is:
The individual fit is much closer to these observations, but that result is conditional on the subject-specific estimate. The typical predictions are one response unit below every observation, a consistent discrepancy that the individual fit largely absorbs. This one subject does not establish a population-wide problem; the same comparison across subjects would show whether the direction is systematic or isolated.
How to Make the Comparison
Before interpreting the plots or summaries, ensure that the predictions refer to the same observations, times, covariates, units, and response scale. If a transformation was used for fitting, the prediction and observed response must be compared on a consistent scale. Also identify whether the population prediction is typical-subject or marginal: those targets answer different questions.
For every observed response, calculate an individual fitted prediction and the specified population-level prediction at the same time and covariate values.
Plot observed responses with both predicted trajectories, and examine observed-minus-predicted residuals at each prediction level. Look for direction, timing, and consistency across subjects.
Ask whether the population-level pattern recurs across subjects and whether individual predictions improve fit mainly through large subject-specific adjustments.
Interpret this comparison alongside residual patterns and other predictive checks. Do not treat a visually close individual fit as proof that the model is adequate.
A residual summary can help organize the comparison, but it should not replace the trajectory display. For example, calculate mean residuals at matched time regions or summarize the distribution of subject-level RMSE values. A near-zero overall mean residual can conceal positive residuals at one part of the response course and negative residuals at another. Similarly, a pooled RMSE can be dominated by subjects with many observations unless the analysis explicitly gives subjects equal weight.
Worked Example: Individual Fits Conceal a Shared Time Pattern
Consider four hypothetical subjects, each observed once in an early time region and once in a late time region. For every subject, the observed response is 2.0 response units above the typical prediction early and 2.0 units below it late. The individual fitted predictions leave residuals of \(+0.2\) early and \(-0.2\) late for each subject.
| Subject | Typical residual, early (units) | Typical residual, late (units) | Individual residuals, early and late (units) | Individual RMSE (units) |
|---|---|---|---|---|
| 1 | +2.0 | -2.0 | +0.2, -0.2 | 0.200 |
| 2 | +2.0 | -2.0 | +0.2, -0.2 | 0.200 |
| 3 | +2.0 | -2.0 | +0.2, -0.2 | 0.200 |
| 4 | +2.0 | -2.0 | +0.2, -0.2 | 0.200 |
For each time region, the mean typical residual across subjects is the average of four identical values:
For each subject, the individual RMSE is \(\sqrt{(0.2^2+(-0.2)^2)/2}=\sqrt{0.0400}=0.200\) response unit. The individual fits are close, but the typical predictions have a repeated early-positive, late-negative residual pattern in all four subjects. This is evidence that the population-level trajectory does not reproduce the observed pattern well under this comparison. It does not, on its own, identify the cause or prove a particular alternative model is correct.
Typical and Marginal Predictions Are Not Interchangeable
For a linear model in the random effects, the typical prediction and the marginal mean can coincide when the random effects have mean zero. For a nonlinear model, averaging predictions over random effects generally differs from evaluating the model at the average random effect. The distinction matters when the goal is to describe the average response of a population rather than the response of a subject with \(\eta=0\).
Worked Example: A Nonlinear Population Mean
Suppose a hypothetical model predicts a response \(f(\eta)=10\exp(\eta)\) response units, and the random effect takes two values with equal probability: \(\eta=-\log(2)\) or \(\eta=\log(2)\). The typical-subject prediction at \(\eta=0\) is \(10\exp(0)=10.0\) response units.
At the two random-effect values, the predictions are \(10\exp[-\log(2)]=10(1/2)=5.0\) and \(10\exp[\log(2)]=10(2)=20.0\) response units. The marginal mean is their probability-weighted average:
Here the typical prediction is 10.0 units, whereas the marginal mean is 12.5 units. The difference arises because the exponential function is nonlinear. For a fitted model with a continuous random-effects distribution, the marginal expectation may require numerical integration or simulation; it should not be assumed equal to the typical prediction.
Assumptions, Diagnostics, and Limitations
This is a model-checking comparison, not a hypothesis test with a universal cutoff. Its interpretation depends on several practical conditions:
- Matched quantities: Each observed response must be paired with predictions for the same observation conditions. Misaligned times, covariates, units, or response scales can create apparent discrepancies.
- Meaningful random-effects structure: Individual predictions depend on how subject-specific effects are defined and estimated. Strong shrinkage, sparse observations, or weak identifiability can make \(\hat{\eta}_i\) an uncertain or muted representation of individual variation.
- Appropriate target: A typical-subject prediction and a marginal population mean answer different questions. State which one is being assessed, especially for nonlinear models.
- Careful aggregation: Repeated observations from one subject are not independent subjects. When summarizing population patterns, display subject-level variation or otherwise account for unequal numbers of observations.
Warning signs include a large, repeated gap between individual and typical predictions; population residuals with a consistent sign in particular time or response regions; and individual fits that look good only when subject-specific parameters move substantially away from the typical prediction. Such patterns are prompts for further investigation, not automatic evidence of a specific structural error. Also inspect whether the pattern is driven by a few subjects, whether it persists on a relevant scale, and whether estimated individual parameters are supported by the available data.
A common mistake is to say, “The individual fitted curves pass close to the observations, so the population model predicts well.” That reasoning overlooks that each curve uses estimates informed by that subject’s observations. The correction is to distinguish conditional fit from population prediction, then assess both and report which target each supports. The reverse mistake is also possible: a poor individual fit for one subject does not by itself show that the population model fails for everyone.
This comparison is complementary to residual-versus-time and observed-predicted diagnostics, not a replacement for them. It summarizes neither all aspects of predictive uncertainty nor the distribution of outcomes expected in future samples. The visual predictive checks covered separately address a different simulation-based question; they should not be replaced by inspecting fitted curves. Ultimately, conclusions should reflect the pattern across subjects, the model’s estimation context, and the intended prediction target.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- In one sentence, distinguish an individual fitted prediction from a typical-subject prediction.
- A subject’s typical prediction is 14 response units and individual prediction is 17 units. The observed response is 18 units. Calculate the typical and individual residuals, using observed minus predicted.
- Four subjects have early typical residuals \(+1.5\) units and late typical residuals \(-1.5\) units. What pattern should be investigated, even if their individual fits are close?
- For a nonlinear model, why might the marginal population mean differ from the typical-subject prediction?
Solutions. (1) An individual fitted prediction uses the subject-specific estimate \(\hat{\eta}_i\), whereas a typical prediction sets the random effects to zero. (2) The typical residual is \(18-14=4\) response units; the individual residual is \(18-17=1\) response unit. (3) The repeated positive early and negative late typical residuals suggest a shared time-dependent discrepancy that individual parameter estimates may be absorbing; further diagnostics are needed to investigate it. (4) In general, \(E_{\eta}[f(\eta)]\ne f(E_{\eta}[\eta])\) for nonlinear \(f\), so averaging predictions over random effects need not equal evaluating the prediction at \(\eta=0\).