Three Different Questions About a Population PK Model
A population PK model can be evaluated from several angles, and the word “diagnostic” does not refer to just one kind of evidence. A model may estimate its population parameters imprecisely, show systematic disagreement with observed concentrations, or produce individual parameter estimates that are strongly pulled toward population values. These are related concerns, but they are not interchangeable. Each requires a different diagnostic target.
Consider a model with population parameters \(\theta\), such as typical clearance and volume, and subject-specific random effects \(\eta_i\). A simple representation of an observation is
Here \(y_{ij}\) is the concentration observed for subject \(i\) at occasion \(j\), \(f\) is the model-predicted concentration, and \(\epsilon_{ij}\) represents residual variation not explained by the prediction. A common assumption is that the random effects have mean zero and covariance matrix \(\Omega\), and that the residual error follows the specified residual-error model. The model also relies on its chosen structural form, dose and sampling information, covariate relationships, and assumptions about the random-effects distribution. A diagnostic can only be interpreted relative to these assumptions.
The central task is to ask what is being evaluated: the precision of the estimated population model, the agreement of predictions with data, or the information available for estimating subject-specific effects. A small standard error for a parameter does not establish good fit; small residuals do not establish precise parameter estimates; and strongly shrunken individual estimates do not by themselves demonstrate model failure.
Match the Diagnostic to Its Target
Population parameter uncertainty is assessed using uncertainty summaries for the estimated population parameters. Examples include standard errors, confidence intervals, likelihood profiles, and bootstrap distributions. These summaries describe variation or uncertainty in the estimate under the model and estimation procedure. They are not summaries of the residual pattern.
Residual misfit is assessed by comparing observed data with predictions and examining the resulting residuals. In common diagnostic usage, a population prediction uses the population parameter values and the typical subject’s random effects, often set to zero. An individual prediction uses an estimate of the subject’s random effects as well. A residual is a discrepancy between an observation and a specified prediction; its exact form may be raw, scaled, or weighted. Thus, the prediction and residual used must be named, since different choices answer different questions.
Population predictions are useful for asking whether the population model represents the data without tailoring the prediction to each subject’s observed concentrations. Individual predictions ask how well the model can describe observations after using subject-specific information. Individual predictions can look close to observations even when population predictions reveal systematic disagreement: the individual effects have been estimated from those same subjects’ data and can compensate for population-level misspecification.
Shrinkage is about the individual estimates, not about the population parameters’ standard errors or the raw size of a residual. When a subject’s observations weakly identify that subject’s random effect, the data provide limited reason to move the estimate away from the population mean. The empirical Bayes estimate is consequently pulled toward zero on the random-effects scale. This is often expected with sparse or uninformative data, but it limits what can be inferred from those individual estimates.
Worked Example: Parameter Uncertainty Is Not a Fit Statistic
Suppose a hypothetical model estimates typical clearance as \(5.00\) L/h, with an estimated standard error of \(0.45\) L/h. For illustration, assume the estimate is approximately normal and use a 95% Wald interval. Calculate the interval and identify what it tells us.
The interval is the estimate plus or minus \(1.96\) standard errors:
The calculation indicates uncertainty in the estimated typical clearance under the stated approximation. In repeated applications of the procedure and its assumptions, intervals constructed this way would cover the fixed true parameter about 95% of the time. It does not mean there is a 95% probability that this particular interval contains the parameter.
This interval does not show whether the model predicts observed concentrations adequately. A narrow interval can accompany systematic residual patterns if the model is misspecified; a wide interval can occur even when the observed concentrations are not obviously inconsistent with the model. To evaluate fit, examine appropriate predictions and residuals rather than treating the standard error as a goodness-of-fit measure.
Why the Targets Differ
The distinction can be seen by writing a raw residual as \(r_{ij}=y_{ij}-f(t_{ij};\hat{\theta},\hat{\eta}_i)\). It depends on an observation and a prediction. By contrast, the uncertainty of \(\hat{\theta}\) concerns how \(\hat{\theta}\) would vary across repeated data sets under the model and estimation procedure. These quantities have different definitions and answer different questions.
A short linearization makes the separation clearer. Suppose \(f\) is differentiable in \(\theta\), and consider the effect of parameter-estimation error on a prediction at fixed time and random effect. Near the true value \(\theta\),
If \(\hat{\theta}\) has covariance matrix \(\operatorname{Cov}(\hat{\theta})\), the approximate variance of the predicted value attributable to parameter uncertainty is
This is uncertainty in the prediction induced by uncertainty in \(\hat{\theta}\); it is not the residual itself. A residual also reflects observation-level variation and any systematic inadequacy of the model. The approximation assumes the prediction is smooth near \(\theta\) and that estimation error is small enough for a first-order expansion to be useful. It does not separate all sources of predictive variation automatically.
Likewise, shrinkage concerns how the data inform \(\eta_i\). In a standard random-effects model, \(\eta_i\) is centered at zero in the population. An empirical Bayes estimate combines subject-specific evidence with the population distribution: when subject-level evidence is weak, the population distribution has more influence. This is not evidence that the population parameter estimate itself has shrunk, nor does it directly measure residual misfit.
Worked Example: Population and Individual Predictions Answer Different Fit Questions
Imagine two hypothetical subjects with observed concentrations of 8 and 12 mg/L at the same sampling time. The model’s population predictions are 10 mg/L for both. After estimating individual effects, the individual predictions are 8.5 and 11.5 mg/L. Use observed-minus-predicted residuals to compare the summaries.
For the population predictions, the residuals are
For the individual predictions, they are
The individual predictions are closer to these observations, but that result alone does not establish that the population model adequately describes the data. Subject-specific effects were estimated using subject data and can improve within-subject agreement. The population residuals show that, at this time, the two observations lie on opposite sides of the same typical prediction. Whether a broader population-level pattern exists requires examining the relevant data and diagnostics, not drawing a conclusion from two values.
The comparison is meaningful only if the predictions use the stated model, doses, times, and covariates, and if the individual estimates were obtained under the specified random-effects and residual-error assumptions. These raw residuals are in mg/L; their magnitudes should not be compared across concentration ranges as though variability were necessarily constant.
Interpreting Individual-Parameter Shrinkage
One descriptive summary of eta shrinkage compares the variability of empirical Bayes estimates across subjects with the population standard deviation of the corresponding random effect. For one random effect with population standard deviation \(\omega\), a common estimate is
This summary is most interpretable when the estimates are centered appropriately and compared on the same scale. It is a population-level description of how dispersed the estimated individual effects are relative to their assumed population distribution. It is not a direct estimate of the error for every subject, and its value can be affected by the study design, model, estimation method, and sample size.
Worked Example: Calculating a Shrinkage Summary
Suppose a hypothetical one-random-effect model has estimated population standard deviation \(\omega=0.40\). Across subjects, the standard deviation of the empirical Bayes estimates is \(0.12\). Calculate the common eta-shrinkage summary.
Substitute the values into the formula:
The estimated eta shrinkage is 0.70, or 70%. In this summary, the empirical Bayes estimates have only 30% of the population random-effect standard deviation. This suggests that the estimated individual effects are substantially less dispersed than the assumed population distribution. It does not mean that every subject’s parameter estimate is 70% too small, or that 70% of subjects are poorly fit.
Such a result warrants attention to how much subject-specific information the design supplies, particularly the number and timing of observations and whether those observations are informative about the parameter. It also warrants checking model assumptions and the stability of the population variability estimate. The shrinkage summary alone cannot identify which explanation is responsible.
Diagnostics, Alternatives, and Limitations
A practical evaluation begins by matching each concern to evidence that can address it. For parameter uncertainty, inspect the uncertainty and stability of population estimates. For residual misfit, inspect observation-prediction discrepancies at the population and individual levels, using a residual definition appropriate to the error model. For shrinkage, examine the distribution of individual random-effect estimates relative to the estimated population distribution, alongside the design’s capacity to identify individual effects.
| Diagnostic question | Relevant target | What it can indicate | What it cannot establish alone |
|---|---|---|---|
| How precisely are population parameters estimated? | Standard errors, intervals, profiles, or resampling distributions for \(\hat{\theta}\) | Uncertainty or instability in population estimates | Adequate prediction of observed concentrations |
| Does the typical population model describe the observations? | Population predictions and associated residuals | Systematic disagreement not tailored to each subject | Why a pattern occurs or whether a parameter is precisely estimated |
| How closely can subject-specific predictions represent the observations? | Individual predictions and associated residuals | Within-subject agreement after using individual information | Good population-level fit or reliable individual parameter estimates |
| Are individual random-effect estimates strongly pulled toward the population? | Dispersion of empirical Bayes estimates relative to the random-effects distribution | Limited subject-level information or strong influence of the population distribution | Residual misfit or the cause of the shrinkage |
A frequent failure is to treat an individual prediction plot as the sole test of model adequacy. A close correspondence can be partly expected because individual predictions use information from the same subject’s data. Conversely, large residuals do not automatically prove a structural-model problem: dose records, sampling times, assay error, covariates, and the residual-error model should also be checked.
Another limitation is that no single diagnostic identifies a unique cause. Parameter uncertainty can reflect limited data, parameter correlations, weak identifiability, or estimation difficulties. Residual patterns can reflect structural misspecification, an inadequate covariate relationship, an inappropriate residual-error model, or data problems. Shrinkage can be substantial because of sparse sampling or weak identifiability, even when the population model is useful. The most informative assessment combines evidence while keeping each diagnostic’s target explicit.
A Compact Evaluation Workflow
The following sequence prevents one type of evidence from being used as a substitute for another:
State whether the concern is population-parameter precision, residual misfit, or the information in individual parameter estimates.
Use uncertainty summaries for population estimates, prediction-residual comparisons for fit, and random-effect estimate dispersion for shrinkage.
Confirm that the prediction and residual definitions, model assumptions, data records, and study design support the intended interpretation.
Report what each diagnostic suggests and what it cannot establish; investigate alternative explanations for concerning patterns.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A clearance estimate has a large standard error. Which diagnostic target does this address, and what does it not establish?
- Why can individual predictions appear close to observations even when population-level fit remains a concern?
- A random-effect standard deviation is 0.30, while the standard deviation of its empirical Bayes estimates is 0.09. Calculate the common eta-shrinkage summary.
- Would a large raw residual by itself prove structural-model misspecification? Give one other possible explanation.
- Which evidence is most directly relevant to whether individual random-effect estimates are pulled toward the population distribution?
Solutions. (1) It addresses uncertainty in the population parameter estimate, not whether the model adequately predicts observations. (2) Individual predictions incorporate subject-specific estimates informed by those observations, so their apparent agreement can improve without demonstrating population-level adequacy. (3) \(1-0.09/0.30=1-0.30=0.70\), or 70% shrinkage by this summary. (4) No. For example, an incorrect sampling time, dose record, or residual-error assumption could contribute to a large discrepancy. (5) Compare the dispersion of empirical Bayes random-effect estimates with the corresponding population random-effects distribution, while considering the design and model assumptions.