A Diagnostic Workflow For Population And Individual Fit

Why an Ordered Diagnostic Workflow Matters

A fitted two-compartment model can describe the overall concentration profile reasonably well while missing important features for particular subjects, or it can appear to fit individuals closely while providing weak information about their individual parameters. A diagnostic workflow helps separate these questions. It starts with predictions at the population level, then examines individual conditional fit, residual patterns and summaries, and finally shrinkage. The order matters: a small individual residual is not, by itself, evidence that the model has learned that subject’s parameters from the data.

For repeated-dose data, the prediction at each observation depends on dose history as well as time since a dose. A residual should therefore be interpreted against the prediction made for that observation’s dose and sampling history. The workflow below is descriptive: plots and summaries can reveal patterns that warrant investigation, but they do not prove that a model is correct.

Definition: Write the observation for subject \(i\) at observation \(j\) as \(Y_{ij}\), and represent the model prediction as \(f_{ij}(\theta,\eta_i)\). Here \(\theta\) denotes population parameters, including typical values and other fixed effects, and \(\eta_i\) denotes subject \(i\)'s random effects. A population prediction, often called PRED, is \(f_{ij}(\theta,0)\). An individual conditional prediction, often called IPRED, is \(f_{ij}(\theta,\hat{\eta}_i)\), using an empirical Bayes estimate \(\hat{\eta}_i\). With the sign convention used here, a raw residual is observed minus predicted: \(Y_{ij}-\text{prediction}\).

For example, if a random effect on clearance is included, it changes the individual’s predicted concentrations across the dosing interval. Population predictions set that effect to zero; individual conditional predictions use its estimated value. Neither prediction is the observed concentration, and the distinction between them is essential when interpreting a residual.

Assumptions and the Reference for Residuals

Diagnostics are only as interpretable as the model and the data preparation underlying them. Confirm that the recorded doses, dose times, sample times, units and subject identifiers are correct; that predictions use the full repeated-dose history; and that the fitted model specifies an appropriate residual-error structure. The model should also represent the intended analysis population and handling of below-quantification observations. A plot cannot repair errors in these inputs.

Suppose, for illustration, that on the modeled observation scale a measurement can be represented as a prediction plus a subject-level random contribution and a residual error. If those two contributions are independent, a residual around a population prediction contains both. Thus dividing that residual only by the residual-error standard deviation does not generally produce a quantity with variance one.

Y_{ij} = f_{ij}(\theta,0) + g_{ij}^{T}\eta_i + \epsilon_{ij}, \qquad \operatorname{Var}(Y_{ij}-f_{ij}(\theta,0)) \approx g_{ij}^{T}\Omega g_{ij} + \sigma_{ij}^{2}

Here \(g_{ij}\) describes how the prediction changes locally with the random effects, \(\Omega\) is their covariance matrix, and \(\sigma_{ij}^{2}\) is the residual-error variance. The approximation follows by linearizing the prediction around \(\eta_i=0\). For nonlinear predictions or substantial random-effect variation, the exact predictive variance may require more than this local approximation. Across repeated observations from one subject, random effects are shared, so population-prediction residuals can also be correlated.

By contrast, residuals based on individual conditional predictions aim to assess discrepancies after conditioning on the subject’s estimated random effects. Appropriately standardized conditional observation errors may be expected to be centered near zero with a scale near one under a suitable model. This is a reference, not a guarantee: empirical Bayes estimation, finite data, model nonlinearity, and the precise residual calculation can affect their distribution. In particular, a population-prediction residual divided only by the residual-error standard deviation has no general unit-scale reference.

Worked Example: Standardizing a Population-Prediction Residual

Suppose one hypothetical concentration is \(8.0\ \text{mg/L}\), and its population prediction is \(6.5\ \text{mg/L}\). The residual-error standard deviation is \(1.0\ \text{mg/L}\), and the random-effects contribution to the predictive variance at this observation is estimated as \(1.8^2=3.24\ (\text{mg/L})^2\).

The raw population residual is \(8.0-6.5=1.5\ \text{mg/L}\). Dividing by the residual-error standard deviation alone gives \(1.5/1.0=1.50\). That calculation does not account for between-subject variation and should not be read as a residual on a unit-scale reference.

Using the stated approximate total predictive variance, the predictive standard deviation is \(\sqrt{1.0^2+3.24}=\sqrt{4.24}=2.059\ \text{mg/L}\). The residual relative to this scale is \(1.5/2.059=0.729\). This is a more relevant population-prediction comparison under the stated approximation, but it is not interchangeable with a conditional weighted residual.

The Ordered Workflow

Use the same sequence across subjects and across the relevant sampling occasions. Keep the plots and summaries tied to the scale on which the model is defined: a residual on a log-transformed scale is not numerically the same quantity as a concentration residual on the original scale.

Check inputs and prediction construction.
Verify dose records, times, units, observation handling, and that the predictions reflect the repeated-dose history. Separate population predictions from individual conditional predictions in labels and output.

Inspect population prediction plots.
Plot observations against population predictions and inspect the relationship across the concentration range. Also inspect observations and predictions against time after dose or another meaningful time coordinate. Look for systematic departures, not just overall closeness.

Inspect residual plots.
Plot residuals against population prediction, time, and other relevant variables, and examine subject-level panels. Use a predictive variance that includes random-effects variation for standardized population residuals; use the defined conditional residual for individual conditional diagnostics.

Summarize residuals with their construction in view.
Review centers, spread, extremes, and patterns by subject or design feature. Do not treat all repeated observations as independent when drawing conclusions from pooled summaries.

Calculate and use shrinkage to qualify interpretation.
Assess eta shrinkage for the random effects relevant to the plots and epsilon shrinkage for residual diagnostics under a stated convention. Treat high shrinkage as a warning that apparent individual-level patterns may be muted.

Worked Examples of Plots and Summaries

Worked Example: Reading Population and Individual Predictions Together

For one hypothetical subject receiving repeated doses, suppose a plot shows an observed concentration of \(8.0\ \text{mg/L}\), a population prediction of \(6.5\ \text{mg/L}\), and an individual conditional prediction of \(7.7\ \text{mg/L}\) at the same sample time. The corresponding raw residuals are \(8.0-6.5=1.5\ \text{mg/L}\) for the population prediction and \(8.0-7.7=0.3\ \text{mg/L}\) for the individual prediction.

The individual prediction is closer to this observation, but that alone does not establish that the subject’s parameter estimates are precise or that the model is adequate. The population plot asks whether the typical model describes the population data; the conditional plot asks how well the fitted model describes observations after using subject-specific estimates. Check the full subject panel and the applicable shrinkage before interpreting the smaller conditional residual as evidence of strong individual information.

In a two-compartment repeated-dose model, inspect predictions over the complete observed range of sampling times and dose intervals. A close match at one concentration or one phase of the interval does not establish that distribution and elimination behavior are represented adequately elsewhere.

Worked Example: Summarizing Conditional Residuals

Consider four hypothetical conditional residuals standardized by their conditional standard deviations: \(0.8,-1.2,1.8,-0.6\). Their mean is \((0.8-1.2+1.8-0.6)/4=0.20\). Their sample standard deviation is calculated from the deviations from that mean:

s = \sqrt{\frac{(0.8-0.2)^2+(-1.2-0.2)^2+(1.8-0.2)^2+(-0.6-0.2)^2}{4-1}} = \sqrt{\frac{5.52}{3}} = 1.356

The small hypothetical set has a positive mean of \(0.20\) and a sample standard deviation of \(1.356\). It does not, by itself, establish a systematic bias or a model failure. Examine residuals against time and prediction, inspect the subject-level context, and consider how the residuals were constructed. If the four residuals come from one subject, they are not four independent subjects; if they come from several subjects, the grouping should still be preserved when interpreting pooled results.

Shrinkage and Individual Fit

Eta shrinkage describes how much the empirical Bayes estimates of a random effect are compressed toward zero relative to the estimated population distribution. For a random effect with estimated population standard deviation \(\omega\), a commonly used variance-based estimate is one minus the ratio of the empirical variance of the subject estimates to the estimated population variance. In the standard-deviation form below, the estimate is based on the sample standard deviation of those estimates. State the convention and the estimate used when reporting it.

\text{Eta shrinkage} = 1 - \frac{\operatorname{SD}(\hat{\eta}_i)}{\omega}

This expression is a descriptive estimate, not a direct measure of the precision of every subject’s estimate. It can be affected by the number and design of observations, the fitted model, and the sample of subjects. Assess shrinkage separately for each random effect: one parameter can have substantial shrinkage while another does not.

Worked Example: Calculating and Interpreting Shrinkage

Suppose, in a hypothetical fit, the estimated population standard deviation for the random effect on log clearance is \(0.40\), and the sample standard deviation of its empirical Bayes estimates is \(0.18\). The estimated eta shrinkage is \(1-(0.18/0.40)=1-0.45=0.55\), or \(55\%\).

Under this convention, the empirical Bayes estimates vary less than the fitted population random-effect distribution, so individual clearance estimates and plots against those estimates should be interpreted cautiously. The result does not mean that every subject’s estimate is wrong by \(55\%\), nor does it identify the cause of the compression.

Suppose also that the sample standard deviation of the model’s conditional weighted residuals is \(0.72\). Under the common convention that epsilon shrinkage is \(1-\operatorname{SD}(\text{CWRES})\), the estimate is \(1-0.72=0.28\), or \(28\%\). This is a qualification on residual diagnostics using those conditional residuals. It is not a substitute for checking their plots, their construction, and the model’s assumptions.

Comparisons, Failure Modes, and Limitations

Diagnostic view	Main question	Key qualification
Population prediction versus observations	Does the typical model describe the overall data pattern?	Population residuals include random-effects variation as well as residual error.
Individual conditional prediction versus observations	How closely does the model describe data after conditioning on subject estimates?	Close fit can be encouraged by conditioning and may be uninformative when shrinkage is high.
Residual plots and summaries	Are discrepancies centered and pattern-free on an appropriate scale?	The reference depends on residual definition, variance standardization, and estimation; repeated observations may be dependent.
Shrinkage calculations	How strongly are subject estimates compressed relative to the population distribution?	Shrinkage qualifies interpretation; it does not by itself diagnose the cause or determine model adequacy.

A common mistake is to divide every observed-minus-predicted residual by the residual-error standard deviation and expect a standard deviation near one. That reasoning ignores random-effects variation when the prediction is a population prediction. For such residuals, use a suitable predictive variance that includes the random-effects contribution, or interpret the unstandardized residual without claiming a unit-scale reference. A different mistake is to treat a tight IPRED plot as proof of good individual estimation. High eta shrinkage can make individual estimates similar to the population value, and epsilon shrinkage can make conditional residuals appear less variable. Check both before making individual-level claims.

Plots are complementary rather than interchangeable. A population prediction plot can reveal broad disagreement but may conceal subject-specific structure. A conditional residual plot can expose discrepancies after using individual estimates, but those same estimates are data-derived. Numerical summaries compress information and can hide patterns that are visible across time, concentration, or subjects. Finally, no single diagnostic establishes that the two-compartment structure, residual model, or random-effects distribution is correct. Use the workflow to identify what merits further model assessment, not as a mechanical pass-or-fail rule.

Key takeaway: Read population and individual fit separately. Standardize each residual using a variance appropriate to how its prediction was constructed, inspect plots before relying on pooled summaries, and use parameter-specific shrinkage to qualify conclusions about individual fit.

Check Your Understanding

Answer each question, then compare with the brief solutions.

A population-prediction residual is divided by the residual-error standard deviation. Why might its scale exceed one even when the model is correctly specified?
For an observation of \(5.4\ \text{mg/L}\), the population prediction is \(4.9\ \text{mg/L}\), and the conditional prediction is \(5.2\ \text{mg/L}\). Find both raw residuals using the stated sign convention.
If the empirical standard deviation of \(\hat{\eta}_i\) is \(0.12\) and the estimated population standard deviation \(\omega\) is \(0.30\), calculate eta shrinkage.
Why should four repeated residuals from one subject not automatically be treated as four independent subjects in a pooled summary?

Solutions. (1) The population residual includes random-effects variation as well as residual error, so the denominator may omit part of its predictive variance. (2) The population residual is \(5.4-4.9=0.5\ \text{mg/L}\); the conditional residual is \(5.4-5.2=0.2\ \text{mg/L}\). (3) \(1-(0.12/0.30)=0.60\), or \(60\%\). (4) Observations from one subject share subject-level random effects and can be dependent; the number of observations is not the number of independent subjects.