Reporting Goodness-Of-Fit And Shrinkage Results

Why Reporting Requires More Than a Plot

A goodness-of-fit figure is useful only when a reader can tell what was predicted, how residuals were formed, and what features deserve attention. In population pharmacokinetics, the same observation can be compared with a population prediction or with a prediction that uses that individual's empirical Bayes estimates (EBEs). These comparisons answer different questions. In addition, apparent individual agreement can be misleading when the EBEs or residuals are strongly shrunken.

A concise report should therefore identify its prediction and residual definitions, summarize the main diagnostic patterns, give shrinkage by parameter, and state what the diagnostics do and do not support. It should not turn a collection of plots into a claim that a model has been proved correct.

The prediction definitions should be tied to the fitted model and the data used to make each prediction. For example, a population prediction may still use an individual's dose, covariates, and sampling times; “population” means that individual random effects are not inserted. The individual prediction includes the fitted individual's EBEs. State whether residuals are on the concentration scale or another modeled scale, and include units for raw residuals.

Raw residuals are not directly comparable across observations when the model implies different residual variances. A weighted residual standardizes discrepancies using a model-based residual variance or covariance. Implementations differ: WRES and CWRES are names for particular weighted-residual calculations, not a license to leave the method unspecified. IWRES is an individual weighted residual. A report should give the residual type and software or calculation convention, especially when conditional approximations are involved. Under an appropriate model and approximation, standardized residuals are expected to be roughly centered near zero with scale near one; that is a diagnostic reference, not a guarantee for a finite sample.

What a Goodness-of-Fit Summary Should Contain

The usual diagnostic set compares observations and predictions and examines weighted residuals against quantities that could reveal systematic errors. For population fit, a plot of \(DV\) against \(PRED\) can show broad disagreement, but points below and above the identity line should be considered across the prediction range. A plot of WRES or CWRES against time can reveal time-structured discrepancies, while residuals against predictions can reveal concentration-dependent spread or bias. Other useful displays include residuals by dose, study, or a clinically relevant covariate when those dimensions matter to the model.

Individual-fit displays, such as \(DV\) against \(IPRED\) or individual residuals against time, can help locate observations or individuals that merit review. However, good-looking individual predictions do not by themselves establish that individual parameters are well estimated: EBEs are pulled toward the population distribution, and residual shrinkage can make residuals look less variable than the underlying discrepancies. These plots should be interpreted alongside parameter-specific shrinkage, not in place of it.

A report need not narrate every point in every figure. It should state the main pattern, its scope, and any important exception. If a pattern is seen only in one time region or subgroup, say so. If the plot appears broadly unstructured, describe that as an absence of an obvious pattern in the displayed diagnostics, not as proof of adequate fit.

Reporting Shrinkage by Parameter

Eta shrinkage describes how much less variable the EBEs are than the estimated population distribution for a random effect. For parameter \(k\), the common sample estimate compares the sample standard deviation of \(\hat{\eta}_{ik}\) with \(\omega_k\). Epsilon shrinkage similarly compares the IWRES standard deviation with its expected unit scale. These summaries are useful because shrinkage affects how much information individual plots and EBE-based analyses contain.

The formula can be understood directly from the scale comparison. If the EBE standard deviation equals the population standard deviation, their ratio is one and the estimated eta shrinkage is zero. If the EBE standard deviation is half the population standard deviation, then the ratio is \(0.5\), so the estimated shrinkage is \(1-0.5=0.5\), or 50%. The same logic applies to epsilon shrinkage when the residual scale is one. This is a descriptive estimate, not a direct measure of the accuracy of every individual's estimate.

Give eta shrinkage separately for each random effect rather than reporting a single pooled percentage. A pooled value can conceal that some parameters are informed much more strongly by the data than others. Report the number of individuals contributing EBEs, the estimated \(\omega_k\), the EBE standard deviation, and the shrinkage estimate when space allows. Likewise, identify which residuals were used for epsilon shrinkage and their expected scale.

These estimates can be negative if the observed EBE or IWRES standard deviation exceeds its reference scale. Do not silently truncate a negative estimate to zero: report the value and investigate whether it reflects sampling variation, estimation uncertainty, or a mismatch in the stated scale or calculation. High shrinkage indicates that individual estimates are strongly pulled toward the population distribution; it does not, by itself, diagnose model misspecification or prove that all individual estimates are useless.

From Diagnostic Findings to a Concise Report

A practical report can be assembled in a consistent order. First define the prediction and residual quantities, including any transformations or weighting. Then summarize the principal population-level plots, with the specific pattern and domain named. Next report shrinkage for each random effect and the residuals. Finally, qualify individual-level interpretations according to the shrinkage and the information available for the parameters.

Assumptions, Alternatives, and Common Pitfalls

The interpretation of these diagnostics depends on correct implementation. Predictions must use the intended fitted model and the appropriate dosing, covariate, and observation records. Residual weighting must correspond to the residual-error model, and the stated standardization should match the method used to calculate WRES, CWRES, or IWRES. Shrinkage calculations require a consistent parameter scale: the EBE spread and \(\omega_k\) must refer to the same random effect and scale. When EBEs are transformed or parameterized differently, the report should make that clear.

Goodness-of-fit plots are descriptive and can be insensitive to some forms of misspecification, especially with sparse sampling or few observations in a region. The observed data also have finite-sample variation, and visual patterns can be affected by the number and timing of records. Shrinkage itself is an estimate influenced by the design, sample size, and fitted model. Neither a low shrinkage estimate nor an apparently unstructured plot is a general guarantee of reliable individual parameters or correct population predictions.

Population-level goodness-of-fit plots are the direct alternative when the main question is whether typical predictions align with the observed data. Individual-level plots add a conditional view, but need shrinkage context. Numerical summaries of residuals can make a report more reproducible, but a single mean or standard deviation can obscure time or subgroup structure. Predictive checks provide a complementary model-evaluation approach, but they answer questions using a different comparison framework; they should not be substituted for precise definitions of the residuals and plots reported here.

Common pitfall: “The individual predictions closely follow the observations, so the individual clearance estimates are reliable.” This reasoning confuses conditional fit with parameter information. EBEs are estimated using the population distribution and can be pulled toward typical values; residual shrinkage can also reduce the apparent spread of residuals. The correction is to report the relevant shrinkage by parameter, describe the individual plots as conditional diagnostics, and avoid claiming individual precision from visual fit alone.