From a Run History to a Decision Rationale
An NLME development report should let a reader understand what decision was made, why it was considered, what evidence informed it, and what was ultimately retained. A list of model runs does not do this by itself. “Model 12 converged; Model 13 had a lower objective function” leaves unanswered whether the models were comparable, whether the change was planned, and whether the numerical improvement mattered scientifically.
A more useful account is organized by decision: structural model, variability structure, covariate relationships, or another feature relevant to the analysis. For each decision, identify the candidates, describe the controlled difference between them, summarize the relevant numerical and diagnostic evidence, and state the rationale for retaining or rejecting the feature. The earlier material on comparing candidate models and recording rejected alternatives provides the development context; the report turns that record into an interpretable explanation.
The report is not a transcript of every attempted fit. It should include enough information to make consequential decisions traceable, including the candidate specifications and the evidence that could reasonably change the decision. A decision may reflect several kinds of evidence: likelihood-based comparisons where justified, parameter estimates and uncertainty, diagnostic plots or summaries, stability of estimation, and scientific plausibility. These sources should be described separately rather than blended into a vague statement that a model “fit better.”
State the Comparison Before Interpreting It
A likelihood-ratio test can compare two nested models fitted to the same observations with the same likelihood definition. The reduced model must be obtained by imposing restrictions on the fuller model. If the objective function is \(OFV=-2\ell\), where \(\ell\) is the maximized log likelihood, define the improvement as the reduced-model objective function minus the full-model objective function. Under the null hypothesis and regularity conditions, this difference is approximately chi-squared, with degrees of freedom equal to the number of independent restrictions.
This approximation requires, among other conditions, comparable likelihoods based on the same data, a genuine nesting relationship, identifiable parameters, and a null value that is not a problematic boundary of the parameter space. For example, a test of a fixed-effect covariate coefficient equal to zero is often regular when the coefficient is estimable. A variance component tested at zero is on the boundary, so the ordinary chi-squared reference distribution may not apply. A report should not quote a routine likelihood-ratio p-value without checking the comparison and its conditions.
Worked Example: Reporting a Nested Covariate Comparison
Hypothetical data. A development analysis compares a reduced clearance model with a fuller model in which clearance depends on body weight through a power relationship. Both models use the same observations and likelihood. The reduced model has \(OFV=1250.0\); the fuller model has \(OFV=1241.4\). The fuller model adds one estimable fixed-effect parameter.
The models are nested because the reduced model is recovered when the added weight coefficient is set to zero. The comparison is not a test of a variance component, and assume the coefficient is identifiable and the estimation and likelihood conditions are satisfactory. The likelihood-ratio statistic is
With one restriction, \(df=1\). Under \(H_0\), the reference statistic is \(\chi^2_1\), giving \(p=P(\chi^2_1\geq 8.6)\approx0.0034\). At a prespecified \(\alpha=0.05\), this is evidence against the no-weight-effect model for this comparison; it does not establish that the covariate is clinically important or that the model is correct in every respect.
Suppose the estimated power coefficient is \(0.240\), with standard error \(0.0898\). The approximate Wald 95% interval for the coefficient is \(0.240\pm1.96(0.0898)=[0.064,0.416]\). For a model proportional to \((WT/70)^\beta\), the clearance ratio at 90 kg versus 50 kg is \((90/50)^\beta\). At the point estimate, this is \(\exp\{0.240\ln(1.8)\}=\exp(0.1411)\approx1.152\), or about 1.15.
Transforming both coefficient limits in the same way gives \(\exp\{0.064\ln(1.8)\}\approx1.038\) and \(\exp\{0.416\ln(1.8)\}\approx1.277\). Thus the approximate ratio interval is 1.04–1.28. In context, the estimated typical clearance at 90 kg is about 1.15 times that at 50 kg, with an approximate 95% Wald interval from 1.04 to 1.28 for this ratio. A report can state that the weight effect was retained, give the nested comparison and ratio estimate with interval, and add the relevant diagnostic evidence rather than relying on the p-value alone.
When Candidates Are Not Nested
For nonnested candidates, a likelihood-ratio test is generally not justified simply because one candidate has a lower OFV. When the likelihoods are comparable and are fitted to the same data, the Akaike information criterion can help compare relative fit and complexity:
where \(k\) is the number of estimated parameters counted under the stated likelihood convention. Lower AIC indicates a better trade-off between likelihood fit and parameter count among the candidates compared. It is not a test of a null hypothesis, and an AIC difference is not a probability that one candidate is true. Its interpretation also depends on candidates using the same observations and a comparable likelihood formulation.
Worked Example: Comparing Nonnested Structural Candidates
Hypothetical data. A team compares one- and two-compartment disposition models. Both are fitted to the same subjects, records, and likelihood. The one-compartment model has \(OFV=1240.0\) and \(k=8\); the two-compartment model has \(OFV=1228.5\) and \(k=10\). Assume both fits meet the estimation checks and their likelihoods are comparable.
The AIC values are \(1240.0+2(8)=1256.0\) and \(1228.5+2(10)=1248.5\). The difference, defined as two-compartment minus one-compartment AIC, is \(1248.5-1256.0=-7.5\). The two-compartment candidate therefore has the lower AIC by 7.5 units, despite its two additional parameters.
This is relative evidence, not proof that the two-compartment model is the data-generating mechanism. Suppose the team also finds that the one-compartment model leaves a systematic late-time pattern in conditional residual diagnostics, while that pattern is reduced under the two-compartment model and the added parameter estimates are stable and physiologically plausible. Those diagnostics provide context that supports the AIC comparison. If the two-compartment fit instead had unstable estimates or unresolved diagnostic problems, the AIC difference alone would not justify presenting it as an unqualified improvement.
Explain Planned and Exploratory Decisions Separately
A report should identify whether a comparison was specified in the analysis plan or arose during development. An exploratory result can be informative, but it should not be presented as if it had been prespecified. State what prompted the exploration, which candidates were fitted, what evidence was examined, and whether the final decision changed. If a planned analysis was altered or not completed, say so directly and explain the reason and implications for that decision.
Worked Example: Reporting a Deviation From the Planned Covariate Form
Hypothetical data. The plan specified a linear body-weight effect on clearance. During development, residual diagnostics suggested that the linear form did not describe the weight trend adequately. The team therefore fitted a power relationship as an exploratory alternative. The linear model has \(OFV=1247.2\), and the power model has \(OFV=1241.4\); each adds one parameter to the same base model and uses the same data and likelihood.
Because the linear and power forms are alternatives rather than nested restrictions of one another, the team does not report a likelihood-ratio p-value for this contrast. Each has the same \(k\), so the AIC difference is also the OFV difference: \(AIC_{\mathrm{power}}-AIC_{\mathrm{linear}}=(1241.4+2k)-(1247.2+2k)=-5.8\). The power form has lower AIC by 5.8 units. Suppose its residual pattern is less pronounced and its parameter estimates are stable. These findings support choosing the power form for the final model, while the exploratory status remains important.
A suitable report explains that the planned linear form was assessed, describes the diagnostic feature that prompted the additional fit, reports the AIC and diagnostic comparison, and labels the power-form assessment exploratory. It should not imply that the original plan contained the power comparison. The evidence may support the final choice, but the unplanned exploration should be considered when judging how much confidence to place in that decision.
Build the Rationale Around Traceable Evidence
For each consequential decision, readers should be able to identify the candidates and the precise change between them. A short table can make this information scannable, provided the accompanying text explains the reasoning.
| Report element | What to state | Why it matters |
|---|---|---|
| Decision | Feature retained, rejected, or left unresolved | Makes the conclusion explicit |
| Candidate contrast | What changed, and what was held constant | Shows whether the comparison is interpretable |
| Evidence | Relevant OFV or AIC, estimate and uncertainty, and diagnostics | Shows the basis for the decision rather than asserting “better fit” |
| Plan status | Prespecified, exploratory, modified, or not completed | Separates planned evidence from later investigation |
| Decision rationale | How statistical and scientific considerations were weighed | Prevents a single metric from being mistaken for the whole argument |
Be selective but not selective in a way that hides contrary evidence. A candidate with a smaller OFV may have gained that advantage by adding parameters, using a different data subset, or changing more than one feature at once. If several features change simultaneously, the comparison cannot isolate which change accounts for the difference. If a model was rejected because of poor stability, report the relevant instability rather than presenting only its unfavorable fit statistic.
Diagnostics are evidence about model adequacy, not decorative additions. Describe the observed issue in context—for example, a remaining time-related residual pattern or implausible parameter estimates—and whether it improved under the alternative. Avoid claiming a problem was “fixed” solely because one plot looked better. Explain when numerical evidence and diagnostics point in different directions, and state the reason for the final choice. The report should also make clear when no candidate was sufficiently supported to resolve a decision.
Common Reporting Pitfalls
- Wrong: “The model was selected because its OFV was lower.” Correction: State which candidates were compared, whether they were nested, whether the same data and likelihood were used, and what complexity or diagnostic evidence was considered.
- Wrong: Treating a smaller p-value as the size or importance of an effect. Correction: Report the effect estimate and an interval in interpretable units, then discuss its relevance separately from the test result.
- Wrong: Reporting an exploratory candidate as though it were prespecified. Correction: Name the deviation, its motivation, and the evidence; label its status accurately.
- Wrong: Listing successive run numbers without explaining the change. Correction: Organize the narrative by decision and state the controlled contrast between candidates.
- Wrong: Treating a favorable criterion as proof of model truth. Correction: Describe the result as relative evidence among the candidates considered, with relevant assumptions and remaining concerns.
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions.
- A full model has \(OFV=900\) and a nested reduced model has \(OFV=905\), with one restriction. What is \(\Delta OFV\), and what must be checked before reporting a likelihood-ratio p-value?
- Two nonnested candidates have AIC values 510 and 516. What does the difference support, and what does it not establish?
- A report says, “The covariate was retained because the p-value was significant,” but gives no estimate or interval. What important information is missing?
- A team fits an unplanned alternative after noticing a residual pattern. What should the report say about that analysis?
Solutions. (1) \(\Delta OFV=905-900=5\), with \(df=1\). Check that the models are nested, use the same data and comparable likelihood, and satisfy the relevant regularity conditions, including that the null value is not a problematic boundary. (2) The candidate with AIC 510 has lower AIC by 6 units and is favored on relative fit-complexity grounds among these candidates; this does not prove it is true or give the probability it is correct. (3) Give the effect estimate, its uncertainty interval, and units or an interpretable contrast, along with relevant diagnostics and the comparison used. (4) Explain what prompted the exploration, identify it as unplanned, describe the candidates and evidence, and state how it affected the decision.