Why Compare Structure and Variability Together?
An NLME model represents both the typical time course of the response and how individuals differ around that typical course. A compartment structure specifies the disposition model, while an interindividual-variability specification determines which individual parameters vary and how those random effects are distributed. A more elaborate model can fit the observed data better, but it also estimates more parameters. AIC provides a single criterion that balances fit against this added complexity.
The comparison is useful only when its inputs are genuinely comparable. Candidate models should be fitted to the same response observations, with the same likelihood target and compatible treatment of residual error, dosing, and missing data. In particular, compare models using maximum likelihood when their fixed-effects structures differ. AIC from restricted maximum likelihood is generally not comparable across models with different fixed-effects designs because the restricted likelihood depends on that design.
Lower AIC indicates a preferable trade-off between fit and complexity among the candidate models being compared. The penalty is based on the number of estimated parameters, not on the number of compartments, the number of random effects as a label, or the number of subjects alone. The earlier material on AIC and parameter counting establishes the general calculation; here the emphasis is on applying it consistently to structural and variability choices.
What Must Be Held Comparable?
AIC differences answer a relative question: which candidate in the specified set has the smallest estimated information loss for the stated modeling target? They do not make unlike likelihoods comparable. Before calculating a ranking, check the following conditions.
- Same observations: every model uses the same response records and the same handling of below-quantification observations, missing values, and exclusions.
- Same likelihood target: use marginal likelihoods for the observed data, integrating over the random effects, and use a consistent estimation method. Approximate NLME likelihoods may be necessary, but their approximation and numerical settings should be comparable across candidates.
- Consistent data model: dosing history, covariates, residual-error assumptions, and any fixed effects not under comparison should be kept consistent unless they are explicitly part of the candidate set.
- Reliable fits: the likelihood should be maximized successfully, with plausible parameter estimates and adequate numerical integration or approximation.
The parameter count includes every freely estimated scalar in the likelihood. For example, a two-compartment model may estimate typical clearance, central volume, intercompartmental clearance, and peripheral volume. Adding interindividual variability on two of these parameters adds two variance parameters; allowing their random effects to correlate adds one covariance parameter. Residual-error parameters also count. Fixed values and parameters shared or constrained by the model are not counted as independent free parameters.
Some NLME likelihoods are computed approximately. The resulting AIC is then based on that approximate log likelihood, so a small AIC gap may be sensitive to approximation accuracy. AIC also relies on regularity conditions for its usual information-theoretic motivation. Variance estimates at a boundary, weak identifiability, or an inadequate random-effects distribution can undermine that motivation; such issues call for careful diagnosis rather than blind reliance on the numerical ranking.
Worked Example: Comparing One- and Two-Compartment Structures
Suppose a hypothetical dataset contains concentration measurements from 32 individuals after the same dosing regimen. Two models are fitted by maximum likelihood to all the same records, with the same residual-error model and the same integration settings. Both include interindividual variability on clearance and the volume assigned to the central compartment, with the two random effects assumed independent.
The one-compartment model estimates typical clearance and volume, two interindividual variances, and one residual-error parameter. Thus \(K=2+2+1=5\). Its maximized log likelihood is \(\ell=-438.2\), giving
The two-compartment model estimates four typical structural parameters (clearance, central volume, intercompartmental clearance, and peripheral volume), the same two interindividual variances, and the same residual-error parameter. Thus \(K=4+2+1=7\). Its maximized log likelihood is \(\ell=-428.6\), so
The AIC difference is \(886.4-871.2=15.2\) in favor of the two-compartment candidate. The more complex model improves the maximized log likelihood enough to offset its two additional parameters. This supports the two-compartment model relative to this one-compartment alternative under the stated analysis; it does not establish that the true biological disposition is necessarily two-compartmental.
Comparing Interindividual-Variability Specifications
Variability candidates can differ in which individual parameters have random effects and in whether those random effects are correlated. Each such choice changes the number of parameters and may change the fitted likelihood. A diagonal covariance matrix for two random effects has two variance parameters; an unrestricted \(2\times2\) covariance matrix has those two variances plus one covariance parameter. The covariance matrix must remain positive semidefinite, so the estimated variances and covariance must define a valid distribution.
Worked Example: Choosing Which Parameters Vary Between Individuals
For a second hypothetical dataset, consider four one-compartment models fitted to the same concentration records by maximum likelihood. Every candidate has two typical structural parameters and one residual-error parameter. The models differ only in their interindividual-variability specification. The fitted log likelihoods below are hypothetical.
| Variability specification | Parameter count \(K\) | Maximized log likelihood \(\ell\) | AIC |
|---|---|---|---|
| No interindividual random effects | 3 | -455.0 | 916.0 |
| Clearance random effect only | 4 | -439.0 | 886.0 |
| Clearance and volume random effects, diagonal covariance | 5 | -436.8 | 883.6 |
| Clearance and volume random effects, estimated covariance | 6 | -436.4 | 884.8 |
For the diagonal two-random-effect model, \(K=2+1+2=5\), so \(AIC=-2(-436.8)+2(5)=873.6+10=883.6\). For the correlated model, the covariance adds one parameter: \(K=6\), and \(AIC=-2(-436.4)+2(6)=872.8+12=884.8\). Although the correlated model has the largest log likelihood, its AIC is 1.2 points higher because its improvement in fit does not offset the added covariance parameter.
Among these four candidates, the diagonal model has the smallest AIC. The no-random-effects model is much less supported within this set, while the clearance-only and diagonal models are relatively close. The result does not mean that the correlated model is impossible; it means that, on this criterion and with these data, the estimated gain from its additional covariance parameter is insufficient to give it the lowest AIC.
Why the Penalty Has This Form
AIC is motivated by estimating how well a fitted model is expected to predict new data from the same target process. A fitted likelihood is optimistic because the parameters were chosen to fit the observed sample. Under regularity conditions and a correctly specified likelihood framework, estimating \(K\) parameters produces an approximate optimism of \(K\) units on the log-likelihood scale. Since the likelihood deviance is \(-2\ell\), the corresponding correction is approximately \(2K\), yielding \(AIC=-2\ell+2K\). This is the justification for the penalty; it is not a derivation of a null distribution for differences in AIC.
The penalty is linear in the number of free parameters. It does not increase because one parameter happens to be a random-effect variance rather than a fixed effect. The practical challenge is therefore to count all free parameters correctly and to ensure that the likelihood values represent comparable fits. A small numerical difference should be treated cautiously if the fits are sensitive to initial estimates, integration settings, or weakly identified parameters.
Worked Example: Comparing a Small Candidate Grid
A third hypothetical analysis uses the same 32 individuals and six concentration records per individual for every candidate. All models use maximum likelihood, the same residual-error form, and the same likelihood approximation settings. The four candidates combine one- or two-compartment structures with diagonal or correlated clearance and central-volume random effects.
| Candidate | \(K\) | \(\ell\) | AIC | \(\Delta AIC\) |
|---|---|---|---|---|
| One compartment, diagonal variability | 5 | -436.8 | 883.6 | 12.4 |
| One compartment, correlated variability | 6 | -436.4 | 884.8 | 13.6 |
| Two compartments, diagonal variability | 7 | -428.6 | 871.2 | 0.0 |
| Two compartments, correlated variability | 8 | -427.9 | 871.8 | 0.6 |
For example, the two-compartment correlated candidate has \(K=8\), so \(AIC=-2(-427.9)+2(8)=855.8+16=871.8\). The minimum AIC is 871.2. For each candidate, \(\Delta AIC=AIC-AIC_{\min}\); hence the correlated two-compartment candidate has \(\Delta AIC=871.8-871.2=0.6\).
For a descriptive comparison, the relative likelihood of a candidate is \(\exp(-\Delta AIC/2)\). The correlated two-compartment candidate has relative likelihood \(\exp(-0.6/2)=\exp(-0.3)\approx0.741\) compared with 1 for the minimum-AIC candidate. The one-compartment diagonal candidate has relative likelihood \(\exp(-12.4/2)=\exp(-6.2)\approx0.00203\). Thus the two-compartment candidates are close by AIC, while both one-compartment candidates rank substantially worse within this particular four-model set. This calculation is a relative ranking, not a probability that any candidate is true.
Diagnostics, Alternatives, and Limitations
An AIC ranking is only as credible as the fitted likelihoods behind it. Before interpreting a close comparison, verify convergence from suitable starting values, inspect whether estimates are finite and plausible, and check that covariance estimates are valid. For approximate likelihoods, assess whether the approximation is adequate for the design and fitted model; differences in approximation quality can masquerade as differences in model fit. Also examine standard NLME diagnostics, including residual patterns over time and predictions, and whether individual-level deviations show systematic structure left unexplained by the model.
A frequent mistake is to choose the candidate with the largest log likelihood because it fits the observed data best. That reasoning ignores the complexity penalty. In the small candidate grid above, the correlated two-compartment model has a higher log likelihood than the diagonal two-compartment model, but its AIC is 0.6 higher. A second mistake is to compare AIC values from fits using different observations or different likelihood targets. The correction is to align the data and estimation target first, then calculate AIC and its differences.
A likelihood-ratio test is an alternative when models are nested and its distributional assumptions are appropriate; it answers a hypothesis-testing question rather than ranking predictive adequacy across a candidate set. For non-nested candidates, AIC is often convenient because it does not require the models to be nested. BIC imposes a different complexity penalty, and AICc adds a finite-sample correction when its assumptions and sample-size definition are suitable. These criteria can rank models differently because they target different trade-offs.
AIC cannot correct an incorrect structural model, guarantee stable parameter estimates, or replace scientific judgment. The candidate set matters: a minimum identifies the best-ranked model only among the models considered. In NLME analyses, a useful decision therefore combines the AIC comparison with estimation diagnostics, residual checks, identifiability, and whether the structural and variability assumptions are scientifically plausible.
Check Your Understanding
Answer each question, then compare with the solutions below.
- A model has \(\ell=-210.5\) and \(K=6\). Calculate its AIC.
- Two candidates fitted to identical records have AIC values 502.0 and 505.4. Calculate the AIC difference from the minimum and identify the lower-AIC candidate.
- A model estimates three typical fixed effects, two independent random-effect variances, one residual-error parameter, and one random-effect covariance. What is \(K\)?
- Why is comparing REML-based AIC values generally inappropriate when the fixed-effects structures differ?
Solutions. (1) \(AIC=-2(-210.5)+2(6)=421.0+12=433.0\). (2) The minimum is 502.0, so the differences are 0.0 and \(505.4-502.0=3.4\); the first candidate has the lower AIC. (3) \(K=3+2+1+1=7\), counting the covariance as a separate free parameter. (4) The restricted likelihood depends on the fixed-effects design, so changing that design changes the likelihood target; the resulting AIC values are not directly comparable. Use maximum-likelihood fits for that comparison.