Model development as a decision trail
A population PK model is usually built through a sequence of decisions: which structural form to use, how to represent between-subject variability, whether a covariate improves the model, and whether a proposed refinement is estimable and useful. A report should let a reviewer understand the important decisions and their consequences without reproducing every exploratory run.
The goal is not to make model development look inevitable. It is to show what meaningful alternatives were considered, how they were compared, what evidence informed each decision, and why the retained model is suitable for the stated analysis objective. A compact decision trail is more informative than a long chronological log because it emphasizes comparisons that could change the interpretation or use of the model.
The reproducible analysis methods establish how the data were handled, how models were estimated, and which diagnostics were used. The development summary should refer to those methods rather than repeat them in full. It should still state enough information for a reader to interpret each comparison: which models differ, whether they were fit to the same observations, and which measures support the decision.
What belongs in a useful summary
Group the development story by decision, not by software run. For example, a report might summarize the choice of structural model, the handling of between-subject variability, and the evaluation of a prespecified covariate. Each group should include the alternatives that were genuinely informative, not every intermediate parameterization or failed experiment.
- Identify candidates. Describe the feature that distinguishes each candidate, such as one- versus two-compartment structure or inclusion versus omission of a specified covariate.
- Establish comparability. State whether candidates used the same analysis data, likelihood, and estimation approach. If those differ, explain why a numerical fit statistic is not a direct comparison.
- Report decision-relevant evidence. This may include objective-function value (OFV), parameter estimates and plausibility, convergence status, diagnostic patterns, and predictive evidence when available.
- State the decision and its basis. Explain why the evidence supports retaining one candidate for the analysis objective, and note meaningful costs such as additional complexity or instability.
- Keep an auditable link. Use stable model labels and point to the relevant analysis output or record, without placing the entire run history in the main text.
A lower OFV indicates better fit under the same likelihood and comparable estimation conditions, but it does not by itself establish that a model is adequate, clinically useful, or preferable. Likewise, adding parameters can improve fit simply because the model has more flexibility. A reported comparison should therefore pair numerical fit evidence with an account of complexity, estimation behavior, and diagnostic or predictive considerations appropriate to the decision.
For nested models estimated with the same likelihood, a likelihood-ratio comparison may be relevant under its regularity assumptions. The test statistic is the reduction in OFV, and its reference distribution depends on the number of additional parameters under the null. The approximation may not apply when a parameter is on a boundary, when the models are not nested, or when other regularity conditions fail. For non-nested candidates, do not describe an OFV difference as a likelihood-ratio test. Information criteria or predictive and diagnostic evidence may be useful alternatives, but each has its own assumptions and interpretation.
Worked examples of concise reporting
Worked Example: Summarizing a Structural-Model Choice
Suppose a hypothetical analysis compares two structural candidates using the same subjects, observations, likelihood, and estimation procedure. The one-compartment model has OFV 842.6 and 8 estimated parameters. The two-compartment model has OFV 831.0 and 10 estimated parameters. Both converge, and the two-compartment model has no implausible parameter estimates. The analysis plan uses AIC as a supporting comparison, with \(AIC=OFV+2k\), where \(k\) is the number of estimated parameters.
For the one-compartment candidate, \(AIC=842.6+2(8)=842.6+16=858.6\). For the two-compartment candidate, \(AIC=831.0+2(10)=831.0+20=851.0\). Thus the AIC difference, calculated as the one-compartment value minus the two-compartment value, is \(858.6-851.0=7.6\), favoring the two-compartment candidate by this criterion.
A concise report could say: “The two-compartment candidate was retained over the one-compartment candidate. Both models converged on the same analysis data; the two-compartment model reduced OFV by 11.6 units and had an AIC 7.6 units lower, despite estimating two additional parameters. Its parameter estimates were plausible. The decision was therefore supported by improved fit after this complexity penalty and acceptable estimation behavior.”
The statement does not claim that AIC proves the model is true or that the model is adequate for every use. If diagnostic evidence were poor, that would need to be reported and could outweigh this comparison.
Worked Example: Reporting a Covariate Comparison
Consider a hypothetical comparison of a base model and a model that adds a prespecified body-size effect on clearance. They use the same data and likelihood, both converge, and the added covariate contributes one estimated parameter. Their OFVs are 516.4 and 511.2, respectively. Assume the added coefficient is an interior parameter and the usual regularity conditions for a one-degree-of-freedom likelihood-ratio approximation are reasonable.
The OFV reduction is \(\Delta OFV=516.4-511.2=5.2\). Under the null hypothesis that the added coefficient is zero, the approximate likelihood-ratio statistic follows a chi-squared distribution with \(df=1\). The upper-tail probability is approximately \(P(\chi^2_1\geq5.2)=0.0226\). This is evidence against the null under the stated model and approximation; it is not the probability that the null is true.
A report should not stop at the p-value. Suppose the estimated effect has the expected direction, is reasonably precise, and improves a relevant diagnostic pattern. A suitable decision record would state those findings and the OFV reduction, then say the covariate was retained because the combined statistical and diagnostic evidence supported its inclusion. If it had little practical relevance or produced unstable estimates, the same p-value alone would not settle the modeling decision.
Worked Example: Recording a Rejected Refinement
In a hypothetical analysis, adding a second random-effects component reduces OFV from 734.0 to 731.8, but the expanded model does not converge reliably across reasonable initial values. Its additional variance estimate is close to zero, and the associated uncertainty is very large. The proposed model has one extra parameter. The OFV reduction is \(734.0-731.8=2.2\).
The comparison is not simply “the lower OFV wins.” The small fit improvement is accompanied by unstable estimation and weak information about the new variance component. The report could state: “The additional random-effects component was not retained. Although OFV decreased by 2.2 units, the expanded model showed sensitivity to initial values and an imprecise variance estimate near zero. The simpler model was retained to avoid an unsupported and unstable component.”
This account makes the decision reviewable: it gives the direction and size of the fit change, the parameter-level concern, and the reason parsimony was preferred. It does not imply that every near-zero variance must be removed; the conclusion is tied to the observed estimation behavior and analysis objective.
Justifying the comparison and the conclusion
A useful check is to ask whether the reported numerical difference answers the stated comparison. For example, an OFV difference is not interpretable as a model improvement if candidates were fit to different observation sets or different likelihoods. Even when the setup is comparable, a fit statistic summarizes only one aspect of model performance.
The logic of a nested likelihood-ratio comparison can be stated compactly. Let the larger model have \(q\) additional parameters and let both models be estimated by maximum likelihood using the same observations. Under the null, the smaller model is correct, and regularity conditions hold, the likelihood-ratio statistic is \(2(\ell_{\mathrm{large}}-\ell_{\mathrm{small}})\), where \(\ell\) is the maximized log likelihood. Since \(OFV=-2\ell\), this becomes \(OFV_{\mathrm{small}}-OFV_{\mathrm{large}}\). Under the null and the stated conditions, this statistic is approximately chi-squared with \(df=q\). This justification explains why the subtraction is ordered small-model OFV minus large-model OFV and why the same-data, same-likelihood condition matters. If the difference is negative, the larger model did not improve the maximized likelihood as expected, which may signal numerical or comparison problems.
This result supports a particular inferential comparison; it does not replace scientific judgment or diagnostics. Boundary parameters, such as a variance tested at zero, can violate the usual chi-squared reference distribution. In such cases, the report should not present the ordinary likelihood-ratio p-value as automatically valid. The comparison may require a method justified for that setting, or it may be reported descriptively with appropriate caution.
For a non-nested comparison, a report should name the basis for preferring one model, such as a prespecified information criterion, predictive performance, interpretability, or a combination. State what the criterion measures and do not imply a formal significance test where none was performed. The chosen model should also be evaluated against the analysis objective: a model intended to describe population parameters may face different practical demands from one intended for individual forecasting.
Diagnostics, alternatives, and failure modes
Before writing a decision, check for signs that the apparent comparison is misleading. Confirm that the candidates use the intended records and consistent likelihood conventions. Check successful minimization and, where relevant, covariance-step behavior, parameter precision, boundary estimates, and sensitivity to initial estimates. Compare relevant diagnostic displays or predictive summaries rather than treating a single scalar statistic as a complete assessment.
A decision summary should distinguish three kinds of evidence. Numerical fit evidence includes OFV or a declared information criterion. Model-behavior evidence includes convergence, plausible estimates, and diagnostic patterns. Scientific and practical judgment includes whether the model represents a defensible mechanism and is sufficiently useful for its intended purpose. If these disagree, describe the disagreement rather than hiding it behind one preferred number.
The principal alternative to a compact decision record is a complete chronological run log. A run log can be valuable for auditability, but is usually too detailed to communicate the development argument in the main report. Another alternative is to report only the final model; that is concise but leaves the rationale opaque. A practical report places the key comparisons in the main narrative or a focused table and preserves full run-level detail in supporting records when needed.
Common reporting failures include cherry-picking only favorable candidates, reporting an OFV change without identifying the models, calling a non-nested difference a likelihood-ratio test, and treating a convergence warning as a minor footnote. Another mistake is to write “the final model was selected because it had the lowest OFV.” The correction is to state the comparison basis, acknowledge the added complexity, and report whether estimation and other relevant evidence support the choice.
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions below.
- Two nested models are fit to the same observations with the same likelihood. Their OFVs are 620.5 for the smaller model and 614.1 for the larger model. What is the OFV reduction, and what additional information is required before reporting a likelihood-ratio p-value?
- Why is it inadequate to report only that a candidate had the lowest OFV?
- A non-nested model has a lower OFV than the retained model. Can this difference automatically be described as a likelihood-ratio test? What should the report do instead?
- A proposed refinement reduces OFV but is unstable across initial values and has a highly imprecise parameter estimate. What is a balanced way to report the decision?
Solutions. (1) The reduction is \(620.5-614.1=6.4\). The number of additional parameters is needed to set \(df\), and the regularity assumptions for the likelihood-ratio approximation must be considered. (2) OFV alone does not address added complexity, diagnostics, estimation reliability, or the model’s purpose. (3) No. A likelihood-ratio test requires nested models and its assumptions; report a suitable comparison basis, such as a justified information criterion or predictive evidence, without claiming a formal test. (4) State the numerical improvement and the instability and imprecision, then explain why the simpler model was retained or why further evaluation is needed.