Why Record Model Decisions?
An NLME analysis can involve several plausible descriptions of the same pharmacokinetic process. A one-compartment model may be adequate, while a two-compartment model may better describe distribution. A model with first-order elimination may be a reasonable starting point, while capacity-limited elimination may be biologically plausible at higher concentrations. Choosing one is not just a matter of obtaining a converged fit: the decision should be connected to the question, the evidence, and the alternatives considered.
A model decision log is a concise, contemporaneous record of those choices. It helps a reader distinguish a decision supported by evidence from one made for convenience, and makes it possible to understand why a candidate was not retained. It is not a substitute for a model specification, fit output, or analysis plan. Its purpose is to link each choice to its rationale and evidence, then state the decision and the status of alternatives.
The decision question should be narrow enough to answer. “Which structural model best describes the concentration-time profiles for the analysis population?” is more useful than “Which model is best?” A structural decision concerns the mathematical form of the process, such as the number of disposition compartments or the elimination function. It is different from a decision about random-effects structure, residual error, or covariates. Keeping those questions distinct makes the rationale interpretable.
What Makes Evidence Comparable?
Before using a fit statistic to choose between candidates, establish that the comparison is meaningful. The models should be fitted to the same analysis data, use the same observation scale and likelihood basis, and target the same estimand. The estimation procedure should have produced a credible optimum for each candidate. If the data, likelihood, or estimation target changes, a difference in fit statistics cannot be attributed solely to structural form.
For example, Akaike's information criterion (AIC) is defined as \(\mathrm{AIC}=-2\ell(\hat{\theta})+2k\), where \(\ell(\hat{\theta})\) is the maximized log likelihood and \(k\) is the number of estimated parameters under a stated counting convention. Lower AIC is preferred as a relative fit-versus-complexity criterion among comparable candidates. In NLME work, the likelihood and parameter-count conventions must be stated; conditional and marginal likelihoods, or inconsistent counting conventions, should not be mixed as though they were directly comparable.
This subtraction gives each candidate’s distance from the lowest AIC in the comparison. It does not give a probability that the candidate is true, and AIC is not a hypothesis test. Its motivation is an approximate correction for the optimism in a model’s maximized log likelihood: under regularity conditions, the expected penalty for estimating \(k\) parameters is approximately \(2k\) on the \(-2\ell\) scale. The approximation is not a guarantee of good prediction, especially with weak identification, boundary parameters, or small samples.
A decision log should therefore record more than a ranking. Relevant evidence may include parameter precision, convergence behavior, residual patterns, individual profile fits, and whether the candidate’s added structure is supported over the observed design. Diagnostics do not prove a model correct; they help identify ways it is inadequate. The log should distinguish “not supported by these data” from “biologically impossible.”
A Reproducible Decision Entry
A practical entry can be organized in a small table or as a short narrative. State the question and candidate models first, then summarize the reason for considering each candidate. Record the data and fit evidence used, including the criterion and its calculation when relevant. Finally, state the decision, the rejected alternative, and any limitation or condition that could motivate revisiting it.
Specify the structural feature under consideration and the population or profiles to which the decision applies.
Describe their actual structural difference and why each is scientifically or statistically plausible.
Identify the data, likelihood basis, fit summaries, diagnostics, and any estimation concerns relevant to the choice.
Explain what was retained and why; label other candidates as rejected or deferred, with a reason that does not exceed the evidence.
Worked Example: One- Versus Two-Compartment Disposition
All data and fit results below are hypothetical. Consider an analysis of concentration-time profiles from 24 participants after an intravenous dose. The decision question is whether a one-compartment disposition structure adequately describes the observed profiles, or whether a two-compartment structure is warranted. Both candidates use the same analysis records, residual-error model, random-effects structure, and marginal-likelihood convention. Assume both fits converged to credible optima and use the same parameter-count convention.
The hypothetical data include early and late samples. A few representative observations from one participant are 8.1 mg/L at 0.25 h, 5.0 mg/L at 1 h, 2.1 mg/L at 4 h, and 0.55 mg/L at 12 h. These observations alone do not establish the population structure; the comparison uses all participants and their observations.
The hypothetical fit summaries are:
| Candidate | Maximized log likelihood | Estimated parameters, k | AIC |
|---|---|---|---|
| One compartment | -412.6 | 6 | 837.2 |
| Two compartments | -398.1 | 9 | 814.2 |
For the one-compartment model, \(\mathrm{AIC}=-2(-412.6)+2(6)=825.2+12=837.2\). For the two-compartment model, \(\mathrm{AIC}=-2(-398.1)+2(9)=796.2+18=814.2\). Thus \(\Delta\mathrm{AIC}\) for the one-compartment candidate is \(837.2-814.2=23.0\), and for the two-compartment candidate it is \(0\). On this criterion, the two-compartment candidate has substantially better relative support in this comparison.
The hypothetical diagnostic review also shows that the one-compartment fit systematically underpredicts some early concentrations and overpredicts some intermediate concentrations, whereas the two-compartment fit reduces that pattern. Suppose the two-compartment fit has no evident estimation failure and its additional parameters are estimated with usable precision. A defensible entry would retain the two-compartment structure for this analysis and record the one-compartment structure as rejected because it had higher AIC and a remaining time-dependent residual pattern. The conclusion is not that two compartments are certainly the true biological description; it is that the combined evidence favors that candidate among those assessed.
Worked Decisions Beyond a Clear Ranking
Not every comparison produces a strong or unambiguous result. A decision log is especially valuable when numerical evidence is close, data are sparse in the region that distinguishes candidates, or an extra parameter is poorly estimated. In those cases, describe the uncertainty and avoid converting a modest numerical difference into a categorical scientific claim.
Worked Example: First-Order Versus Capacity-Limited Elimination
All data and fit results are hypothetical. Suppose concentration-time data are available across a limited dose range. The candidates are a model with first-order elimination and a model with Michaelis-Menten elimination. The scientific rationale for considering capacity limitation is that elimination rate may not remain proportional to concentration at higher concentrations. Both models are fitted to the same records under the same likelihood basis, and both fits are usable.
Assume the first-order model has \(\ell=-255.4\) and \(k=7\), while the Michaelis-Menten model has \(\ell=-251.8\) and \(k=9\). The calculations are \(\mathrm{AIC}_{\mathrm{first}}=-2(-255.4)+2(7)=510.8+14=524.8\) and \(\mathrm{AIC}_{\mathrm{MM}}=-2(-251.8)+2(9)=503.6+18=521.6\). The difference is \(524.8-521.6=3.2\), favoring the Michaelis-Menten candidate by AIC, but the evidence is not decisive by itself.
Suppose the hypothetical data contain few observations at high concentrations, where the models differ most, and the estimated capacity parameter is imprecise. A measured decision might retain first-order elimination as the primary structure for the current analysis, while recording the Michaelis-Menten model as a plausible but weakly identified alternative. The rationale is not that AIC favors the simpler model here—it does not—but that the modest AIC difference is accompanied by limited information about the distinguishing region and imprecision in the added structure. The entry should state that the conclusion is conditional on the available dose range and is open to reassessment with more informative data.
Worked Example: Lag-Time Alternative With Little Evidence
All data and fit results are hypothetical. Consider oral concentration-time profiles where a delayed onset of absorption is plausible. Compare a first-order absorption model without lag time to the same structure with an estimated lag time. Assume the models use identical data and likelihood conventions and that both fits are computationally usable.
Suppose the no-lag model has \(\ell=-180.0\), \(k=8\), and the lag-time model has \(\ell=-179.2\), \(k=9\). Then the no-lag AIC is \(-2(-180.0)+2(8)=360.0+16=376.0\). The lag-time AIC is \(-2(-179.2)+2(9)=358.4+18=376.4\). The lag-time model's log likelihood is higher, but its AIC is \(0.4\) higher because the improvement in fit does not offset its added parameter penalty.
Suppose the hypothetical diagnostic plots show no consistent early-time residual pattern, and the estimated lag time is close to zero with substantial uncertainty. The decision entry can retain the no-lag model and reject the lag-time alternative for the primary specification, noting both the negligible fit gain after penalization and the lack of clear diagnostic evidence for a delay. It should not say that absorption begins instantaneously in every participant; the analysis only failed to establish a useful lag-time component from these data.
Diagnostics, Alternatives, and Common Pitfalls
A decision is only as reliable as the evidence on which it rests. Before recording a candidate as preferred, check whether the comparison is being distorted by data or estimation problems. Examples include different observations being used for different candidates, a fit that failed to reach a credible optimum, parameters at implausible boundaries, very large standard errors, or residual patterns that remain despite a lower AIC. A numerical criterion cannot repair an invalid comparison or establish that the model is scientifically adequate.
The main alternatives to an AIC-based ranking are complementary, not interchangeable. Scientific knowledge can rule out implausible structures or motivate candidates. Residual and predictive diagnostics assess aspects of adequacy that a single summary may conceal. Information criteria summarize relative fit and complexity under a specified likelihood basis. A formal likelihood-ratio test can be appropriate for certain nested models when its regularity conditions hold, but extra compartment or variance parameters can lie on boundaries, making a routine reference chi-square distribution questionable. The decision log should name the evidence actually used rather than imply that one method settles every question.
A frequent mistake is: “The candidate with the lowest AIC is the true model, so every alternative can be called wrong.” This reasoning confuses relative ranking with truth. The correction is to state that the candidate had the lowest AIC among the specified, comparably fitted candidates, then report whether diagnostics and scientific considerations support its use and what limitations remain. Another mistake is to record only the selected model. Without the rationale and rejected alternatives, a later reader cannot tell whether other plausible structures were evaluated or simply overlooked.
Check Your Understanding
Answer each question, then compare with the brief solutions below.
- Two candidates are fitted to different analysis datasets. Can their AIC values be used as a direct structural comparison? Why?
- A simpler model has AIC 410.0 and a more complex model has AIC 408.5. Calculate the AIC difference from the minimum and state what it does, and does not, establish.
- Name four elements that make a model decision entry traceable.
- A model has the lowest AIC but an evident systematic residual pattern. What should the decision record say?
Solutions. (1) No. Since the data differ, the likelihood values are not a like-for-like comparison; refit candidates to the same analysis records and likelihood basis. (2) The simpler model has \(\Delta\mathrm{AIC}=410.0-408.5=1.5\); the complex model ranks lower by AIC in this comparison, but that does not prove it is true or adequate. (3) For example: the decision question, candidate structure and rationale, evidence and its source, and the outcome with the rejected or deferred alternative. (4) Record both the relative AIC result and the diagnostic failure. Do not call the model adequate solely because it ranks first; investigate the pattern and qualify or revise the decision.