Make the Decision Path Reviewable
A final population pharmacokinetic model is the result of a sequence of choices, not a single calculation. A regulatory reviewer needs to understand which choices were considered, what evidence informed them, and why the selected option was carried forward. A report that presents only the final model conceals whether alternatives were examined fairly, whether the analysis followed its plan, and where judgment was required.
The purpose of reporting model development is not to reproduce every exploratory computation. It is to give a concise, reproducible account of the material decisions that shaped the final model. A material decision is one that could reasonably change the model structure, its estimated variability, its covariate relationships, or the predictions and conclusions based on it. The report should identify the decision, the alternatives, the evidence used, the selection rationale, and the consequence for the final model.
This account complements the analysis plan and the material on data lineage and computational settings. It should make clear what was planned, what was actually done, and what entered the final model. It is not a substitute for the later presentation of parameter estimates, diagnostic evidence, or sensitivity analyses; it should point to those results where they provide support for a decision.
Report Decisions in Sequence
Present the decisions in the order they were evaluated, rather than arranging them retrospectively to make the final model seem inevitable. The relevant sequence depends on the analysis, but may include structural choices, the random-effects structure, residual-error alternatives, covariate relationships, and decisions about retaining or removing model components. Identify the actual choices made in the analysis; do not imply that every analysis must evaluate every category.
For each material choice, report enough information for a reviewer to reconstruct the reasoning. The alternatives should be specific, not simply “other models.” State which data and analysis set were used for the comparison, whether the alternatives were prespecified, and whether they were fitted under comparable conditions. Summarize the relevant evidence, such as relative fit, diagnostic patterns, numerical stability, data support, biological plausibility, or parsimony. State which option was selected and what changed in the final model as a result.
Identify the modeling question and why it mattered to the analysis.
Describe the candidate choices precisely and distinguish planned comparisons from additional exploratory ones.
Report the comparison basis and relevant findings, including limitations such as instability or weak data support.
Explain why the selected choice was preferred and state how it appears in the final model or affects its interpretation.
A useful report distinguishes evidence from judgment. For example, “the more complex option had a lower AIC, but the improvement was small and its added component was unstable; the simpler option was retained” reports both the comparison and the decision. “The model fit better” does not tell the reader how fit was assessed, whether the improvement was material, or what other considerations affected the selection.
Using Fit Comparisons Carefully
One common summary for comparing candidate models is the Akaike information criterion (AIC). For a fitted model with \(k\) estimated parameters and objective function value \(OFV=-2\ell\), where \(\ell\) is the maximized log-likelihood, \(AIC=OFV+2k\). When models are fitted to the same observations using comparable likelihoods, a lower AIC indicates a more favorable balance of fit and parameter count among the models compared. It does not establish that a model is biologically true, prove that its added terms are meaningful, or guarantee good predictions.
This expression shows why fit improvement alone is not the whole comparison. If a complex model adds \(\Delta k\) parameters, its objective function must decrease enough to offset the penalty \(2\Delta k\) for its AIC to be lower. AIC differences are comparative summaries, not hypothesis tests or universal thresholds for acceptance. Report the values and the direction of comparison rather than treating a small difference as decisive.
Likelihood-ratio comparisons may also be appropriate for certain nested models, but their usual reference distribution requires regularity conditions, including suitable nesting and parameters that are not on problematic boundaries. Those conditions can fail for some variance-component comparisons. If a test is used, state the comparison, assumptions, and result; do not select a model solely because a test crosses a threshold. Regardless of the numerical criterion, model adequacy, numerical behavior, data support, and the intended use remain relevant.
Worked Example: Comparing Residual-Error Alternatives
Hypothetical data. Two residual-error alternatives are fitted to the same concentration observations using the same likelihood and analysis set. The proportional-error model has \(OFV=1240.6\) and \(k=12\). A combined additive-and-proportional alternative has \(OFV=1234.8\) and \(k=14\). Compare them using AIC and describe a defensible report conclusion.
For the proportional-error model,
For the combined alternative,
Thus, \(AIC_{\text{combined}}-AIC_{\text{proportional}}=1262.8-1264.6=-1.8\). The combined model has the lower AIC by 1.8 points, so AIC slightly favors it among these two candidates. This difference does not by itself require selecting the combined model. If the proportional model adequately described the observations and the added component was poorly supported or unstable, retaining the simpler model could be justified. The report would need to state that evidence and the resulting final choice; it should not say that AIC favored the proportional model.
Worked Decisions Across a Development Sequence
Worked Example: Evaluating a Prespecified Covariate Relationship
Hypothetical data. An analysis plan prespecifies evaluating a weight relationship on clearance. The baseline model has \(OFV=810.0\) with \(k=10\); the model including the relationship has \(OFV=802.9\) with \(k=11\). Both use the same observations and likelihood. Calculate AIC and write the decision record if the added relationship is retained after review of its data support and model behavior.
The baseline AIC is \(810.0+2(10)=830.0\). The covariate model AIC is \(802.9+2(11)=824.9\). The difference is \(824.9-830.0=-5.1\), so the covariate model has a lower AIC by 5.1 points.
A suitable record would say that the prespecified weight relationship was evaluated against the baseline model on the same analysis set; the covariate model had a lower AIC by 5.1 points; and the relationship was retained after review of data coverage and model behavior. The final model therefore includes the prespecified relationship. This supports reporting that the relationship improved the model under the stated comparison; it does not establish a causal effect of weight or justify claiming an effect outside the observed data range.
Worked Example: Deciding Whether to Retain a Random-Effects Correlation
Hypothetical data. A model with separate random effects and no estimated correlation has \(OFV=990.0\) and \(k=13\). A candidate that estimates their correlation has \(OFV=986.5\) and \(k=14\). Both were fitted to the same data and likelihood. The correlation estimate is numerically unstable across fitting attempts. Calculate AIC and state how to report the decision.
For the model without an estimated correlation, \(AIC=990.0+2(13)=1016.0\). For the candidate with a correlation, \(AIC=986.5+2(14)=1014.5\). The correlation model has an AIC lower by \(1014.5-1016.0=-1.5\), a modest numerical preference for the more complex candidate.
AIC alone does not resolve the decision. Given the stated instability, a defensible choice is to retain the model without an estimated correlation, provided it is otherwise adequate for the analysis. The report should state that the correlated alternative was evaluated, had an AIC 1.5 points lower, but was not retained because the estimated correlation was numerically unstable. The final model consequently does not estimate that correlation; this choice affects how joint random-effects variation is represented. The report should not conceal the lower AIC or describe the simpler model as numerically superior.
Planned Choices, Deviations, and Alternatives
Prespecification helps reviewers distinguish planned model development from choices made after seeing the results. When a prespecified choice is followed, identify it as such. When the analysis departs from the plan, report what changed, why it changed, when the decision was made, and whether the change affected the final model. A deviation is not automatically inappropriate, but an undocumented deviation prevents a reviewer from assessing its rationale.
Worked Example: Reporting a Deviation From the Plan
Hypothetical analysis. The plan specifies a proportional residual-error model. During development, the team fits a combined additive-and-proportional alternative because residual patterns are poorly represented at low concentrations. The alternative improves the prespecified comparison criterion, and the final model uses it.
The report should identify the proportional model as the planned choice and the combined model as an additional alternative evaluated during development. It should state the reason for the additional analysis, summarize the comparison evidence, and say that the combined model was selected and therefore defines the residual-error component of the final model. The report should also cross-reference the relevant supporting diagnostic evidence rather than simply asserting that the alternative was “better.” If the analysis plan was formally amended, describe that amendment in the appropriate account of plan changes; do not make the development narrative imply the choice was prespecified.
A concise decision table can make a sequence easier to review. It should not replace the explanation of a consequential or difficult choice.
| Decision | Alternatives and evidence | Rationale and final-model consequence |
|---|---|---|
| Residual-error structure | Proportional versus combined error; compare using the same data and likelihood, with relevant diagnostic evidence. | State which was selected and identify the error structure used in the final model. |
| Covariate relationship | Baseline versus prespecified relationship; summarize fit, data coverage, and model behavior. | State whether the relationship was retained and the population to which the evidence applies. |
| Random-effects structure | Independent versus correlated effects; summarize comparative fit and numerical stability. | State which structure was retained and which variability features the final model estimates. |
Common Pitfalls, Diagnostics, and Limitations
A central pitfall is to report only the selected model and describe it as the inevitable result of “best fit.” That wording hides which alternatives were considered and what “best” means. Correct it by naming the alternatives, giving the basis of comparison, summarizing relevant evidence, and acknowledging trade-offs. Another common error is to treat a lower OFV as sufficient reason to select a more complex model. A lower OFV describes fit at the fitted likelihood; it does not account for complexity by itself or establish that the added feature is supported.
Check that reported alternatives were fitted to comparable data and that the stated criterion is appropriate for the comparison. AIC values are not directly comparable when models use different observations or incompatible likelihood definitions. Check that the reported parameter counts and arithmetic agree with the fitted models. If a candidate failed to converge, was unstable, or could not be supported by the available data, report that limitation rather than presenting its comparison as equivalent to a stable fit.
Different approaches answer different questions. Prespecified sequential evaluation emphasizes transparent, planned decisions. Information criteria compare relative fit and complexity, while likelihood-ratio tests can address suitable nested comparisons under their assumptions. A purely stepwise procedure can be sensitive to thresholds and search order; it should not be presented as an objective guarantee of the final model. Diagnostic and predictive evidence address aspects of adequacy that a single fit criterion cannot capture. In practice, a defensible decision may draw on multiple forms of evidence and explain any disagreement among them.
The decision record has limits: it cannot make weak data coverage informative, remove uncertainty from model choice, or prove that a selected model is uniquely correct. A concise account should therefore avoid overstating what a comparison establishes. It should preserve enough detail to understand the final model’s development without repeating every numerical output in the main text. Detailed candidate results can be placed in supporting materials, with a clear reference from the decision summary.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A more complex candidate has a lower OFV but a higher AIC than a simpler candidate. What does this indicate, and what should the report avoid claiming?
- Two models have AIC values of 702.4 and 699.9, respectively. What is their AIC difference, and what can you conclude?
- A model-development choice differed from the analysis plan. Name four elements that should be documented.
- Why is an AIC comparison generally not interpretable if the candidate models use different observations?
- A candidate has the lowest AIC but an unstable added component. Should the report omit the candidate? Explain.
Solutions. (1) The more complex model’s fit improvement was not enough to offset its parameter penalty, so its AIC is higher. Report both the fit and complexity-aware comparison; do not call the lower OFV alone decisive. (2) The difference is \(699.9-702.4=-2.5\), so the model with AIC 699.9 is favored by this criterion among the two candidates; the difference does not prove it is true or adequate. (3) State the planned choice, what was changed, why and when the change was made, and how it affected the final model; also identify relevant comparison evidence. (4) The models then do not summarize fit to the same observations, so the AIC difference does not provide a like-for-like comparison of the candidates as fitted. (5) No. Report the candidate and its lower AIC, then explain the instability and why the final choice differs from the numerical ranking. Omitting it would hide material evidence relevant to the decision.