Two Sources of Systematic Prediction Failure
A population pharmacokinetic model can miss observed concentrations for different reasons. Its structural model may not describe the systematic concentration-time relationship adequately, or its random-effects structure may not represent how individuals vary around that relationship. These explanations can produce similar residual patterns, so a single diagnostic plot or a better numerical fit is rarely enough to separate them.
The useful distinction is between a pattern shared across individuals and variation that differs from one individual to another. Population predictions show what the structural model predicts for a typical individual. Conditional, or individual, predictions also use information from that individual’s observations to estimate their random effects. Comparing both kinds of predictions helps ask whether individual-specific adjustment removes the discrepancy, and whether the remaining failure is shared.
This comparison is diagnostic, not a formal proof of the cause of misfit. Conditional predictions use the same individual observations being evaluated, so they generally look better than population predictions even when the model is not correct. An improvement in conditional fit alone does not show that the random-effects structure is appropriate. The two predictions answer different questions: does the typical model capture the shared pattern, and can the model’s individual adjustments account for the differences among people?
A Practical Interpretation of Prediction Patterns
Suppose population predictions show observations tending to fall above the predictions in one part of the profile and below them in another, with the same direction of discrepancy across many individuals. If conditional predictions show a similar pattern, adjusting individual random effects has not resolved the shared discrepancy. This is evidence to investigate the structural model. It does not identify which structural feature should change: the pattern might also reflect an unaccounted-for design, measurement, or data issue.
A different pattern occurs when population predictions miss in different directions for different individuals, while conditional predictions substantially reduce those person-specific discrepancies. That pattern suggests the model’s individual variation may be important. It can motivate examining whether the random-effects distribution, variance, or covariance structure represents the data adequately. But it can also arise from residual noise, influential observations, or fitting many individual effects to sparse data. Stronger evidence comes from repeatable individual-level patterns and stable estimates, not from a few large conditional adjustments.
Parameter estimates add context. A revised random-effects structure may change estimates of random-effect variances or correlations, and may also move population parameter estimates if the original model forced one part of the model to compensate for another. Conversely, a systematic population-level pattern that persists across individuals can motivate a structural change, even if a more flexible random-effects model reduces an overall objective function. Neither parameter movement nor improved fit is conclusive on its own: model flexibility, uncertainty, boundary estimates, and the data design all matter.
Worked Example: A Shared Discrepancy Across Individuals
Consider hypothetical observations from four individuals. In a particular late-time window, the observed-minus-predicted differences from the population model are \(1.9\), \(2.2\), \(1.8\), and \(2.1\) concentration units. After estimating individual random effects, the corresponding differences are \(1.6\), \(1.9\), \(1.5\), and \(1.8\) concentration units. The estimates have adjusted the predictions, but the discrepancies remain positive for all four individuals.
For the population differences, the mean is \((1.9+2.2+1.8+2.1)/4=2.00\) concentration units. For the conditional differences, it is \((1.6+1.9+1.5+1.8)/4=1.70\) concentration units. Conditional predictions have reduced the average discrepancy by \(2.00-1.70=0.30\) concentration units, but they have not removed the common direction of error.
The shared late-time pattern remaining after individual adjustment makes a structural-model problem worth investigating. This does not establish a particular structural explanation, and four individuals alone are weak evidence. The next checks would include whether the pattern recurs across adequately sampled individuals and whether it is robust to influential data and model-fitting uncertainty.
What Conditional Predictions Can and Cannot Show
The conditional prediction depends on the estimated individual effect \(\hat{\eta}_i\), and that estimate is informed by the person’s data. With rich observations, an individual effect can be estimated more precisely. With sparse observations, the data often provide little information about that effect, so the estimate is pulled toward zero. This is shrinkage. Consequently, weak apparent differences among conditional predictions in sparse data do not demonstrate that the population has little individual variability.
A short derivation makes the mechanism clear. Consider a simplified random-intercept model for log concentrations, \(y_{ij}=\mu+\eta_i+\epsilon_{ij}\), with independent \(\eta_i\sim N(0,\omega^2)\) and \(\epsilon_{ij}\sim N(0,\sigma^2)\). Here, \(\mu\) is the population mean on the log scale, \(\omega^2\) is the random-intercept variance, and \(\sigma^2\) is the residual variance. For \(m\) observations from one individual, let \(\bar{r}_i\) be the mean of the residuals \(y_{ij}-\mu\). The conditional distribution of \(\eta_i\) is normal, and its mean is the conditional estimate:
The multiplier is between zero and one when both variances are positive. As \(m\) increases, the data provide more information and the multiplier approaches one. When residual variance is large relative to the information from the observations, the estimate is pulled more strongly toward zero. This result assumes the stated normal, independent random-intercept model with known variance components; it illustrates shrinkage rather than providing a general formula for every nonlinear mixed-effects model.
Worked Example: Quantifying Shrinkage in a Sparse Individual
Suppose, hypothetically, that the simplified model above has one observation for an individual, \(\omega^2=0.25\) log-concentration units squared, and \(\sigma^2=0.36\) log-concentration units squared. The observation’s residual from the population mean is \(\bar{r}_i=0.60\) log-concentration units. The individual has only one observation, so \(m=1\).
The shrinkage multiplier is \(1(0.25)/[1(0.25)+0.36]=0.25/0.61=0.4098\). Thus, \(\hat{\eta}_i=0.4098(0.60)=0.2459\) log-concentration units. The conditional residual is \(0.60-0.2459=0.3541\) log-concentration units.
The individual estimate accounts for about 41% of the observed deviation from the population mean; the remainder is not assigned to the random intercept by this conditional estimate. It would be incorrect to infer that the individual differs from the population by only \(0.2459\) units with certainty. The estimate is shrunk because the single observation cannot cleanly distinguish individual variation from residual error.
Using Estimates as Supporting Evidence
Compare parameter estimates across carefully chosen candidate models, but interpret changes in light of uncertainty and model structure. For example, a substantial change in a population parameter after changing the random-effects structure may indicate that the original model was allocating variation poorly. It may also reflect weak identification: the data may not distinguish the population parameter from individual variation. Check whether estimates are stable, whether uncertainty is large, and whether the fitted random-effect variances are plausible and estimable.
The reverse issue also matters. A more flexible structural model can absorb variation that might otherwise be attributed to random effects. If a structural revision greatly reduces an estimated random-effect variance, that does not alone prove the variance was spurious. The revised structure may have changed what the random effect represents, and both changes must be assessed against the data and the scientific model.
A useful comparison holds as much as possible constant while changing one aspect at a time. Check population and conditional predictions, relevant parameter estimates, and diagnostics across the same individuals and observation times. If changing the random-effects structure improves individual-level predictions but a shared pattern remains, there may be evidence for both sources of inadequacy. The choice need not be a forced either-or decision.
| Evidence to examine | May point toward structural misfit | May point toward random-effects misfit |
|---|---|---|
| Population predictions | A recurring discrepancy in the same direction across individuals | Different individuals show persistent deviations in different directions |
| Conditional predictions | A shared discrepancy remains after individual adjustment | Individual adjustments reduce repeatable person-specific discrepancies |
| Parameter estimates | Systematic estimates or predictions remain unstable under the structural form | Variance or covariance estimates, or population estimates, change materially when individual variation is represented differently |
| Data support | Several individuals have informative observations across the relevant profile | Repeated observations support separating individual variation from residual error |
Workflow, Diagnostics, and Limitations
Use the following sequence to keep the diagnosis focused. It is a framework for evaluating evidence, not an automatic model-selection rule.
Review population and conditional predictions over the same data. Note which discrepancies are shared and which differ by individual.
Consider how many observations each individual contributes and whether those observations can inform the relevant random effects. Interpret conditional predictions cautiously when shrinkage is strong.
Compare population parameters and random-effect variances or covariances across targeted candidate models. Look for instability, implausible estimates, and evidence that the data cannot distinguish competing explanations.
Change the structural model or random-effects structure in a scientifically defensible way, then check whether the relevant pattern improves without creating new systematic failures.
If both revisions improve different aspects, or the data are sparse, report that the diagnosis is not uniquely resolved rather than selecting a cause from fit alone.
Warning signs include conditional predictions that appear much better only for individuals with many observations, random-effect estimates that are highly sensitive to a small number of individuals, and variance estimates at or near a boundary. A failure to estimate a variance reliably may reflect limited information, not proof that the corresponding variability is absent. Likewise, a shared residual pattern may be distorted by a few influential observations or data-quality issues; check the evidence before attributing it to the model.
The main alternative diagnostic emphasis is to evaluate prediction performance at the population level or conditional on individuals. Population-level assessment is more informative about predictions for a new individual without using that person’s outcomes to estimate random effects. Conditional assessment can reveal how well the model accounts for observed individuals, but it uses their data and can be optimistic for that purpose. Neither perspective replaces the other. The limits are especially important when the sample is small, sampling is sparse, or the candidate models differ in several ways at once.
Common Mistake
A common mistake is to see that conditional predictions fit better than population predictions and conclude that the random-effects structure has fixed the model. The reasoning fails because conditional predictions are allowed to use each individual’s observations to estimate \(\hat{\eta}_i\); some improvement is therefore expected. It is evidence for a useful individual adjustment only when it is supported by informative data and an appropriate random-effects model. If a shared systematic pattern remains, the structural model still warrants scrutiny. Conversely, a lack of visible conditional differences in sparse data may be a consequence of shrinkage, not evidence that individuals are genuinely alike.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Population prediction differences are positive for most individuals in the same observation window, and conditional prediction differences remain positive. Which model component deserves focused investigation, and why is the conclusion not definitive?
- In the simplified random-intercept model, let \(m=2\), \(\omega^2=0.20\), \(\sigma^2=0.30\), and \(\bar{r}_i=0.50\). Calculate the shrinkage multiplier and \(\hat{\eta}_i\).
- Why is improved conditional prediction performance alone insufficient evidence that the random-effects structure is correct?
- Several individuals have sparse data and conditional predictions show little individual separation. What is one reason this observation may be misleading?
Solutions. 1. Investigate the structural model because a discrepancy shared across individuals persists after individual adjustment. The pattern could still reflect data or design issues, and the evidence depends on its recurrence and support. 2. The multiplier is \(2(0.20)/[2(0.20)+0.30]=0.40/0.70=0.5714\). Therefore, \(\hat{\eta}_i=0.5714(0.50)=0.2857\) log-concentration units. 3. Conditional predictions use the individual’s observations to estimate random effects, so they can improve even when the proposed random-effects structure is inadequate. 4. Sparse data can cause strong shrinkage toward zero, hiding real individual differences; weak separation is not proof of low variability.