AICc for Small NLME Datasets

Why Add a Small-Sample Correction?

Akaike’s information criterion (AIC) balances model fit against the number of estimated parameters. In small datasets, however, the usual AIC penalty can be too modest: a model with many parameters may appear attractive partly because it has had more opportunity to fit the data. The small-sample corrected criterion, AICc, adds a penalty that grows as the sample size approaches the number of estimated parameters.

In an NLME analysis, that extra penalty requires care. Repeated concentration records from the same subject are not generally independent sampling units, and the phrase “sample size” can refer to different things. AICc is useful only when its parameter count and sample-size convention are stated clearly and used consistently across the models being compared.

AICc is a relative model-selection criterion: among models fitted to the same response data with comparable likelihoods, lower AICc indicates the model with the more favorable estimated fit–complexity trade-off under this criterion. It is not a probability that a model is true, nor a test of whether a particular parameter is zero. Differences in AICc are meaningful only when the models use a comparable likelihood and the same data.

What Counts as a Parameter?

For the usual marginal-likelihood AICc calculation, \(K\) counts the free parameters estimated in the model likelihood. In an NLME model this typically includes fixed-effect parameters, estimated random-effect variance and covariance parameters, and parameters of the residual-error model. If a model estimates residual variance separately for multiple components, each free variance parameter contributes to \(K\). Parameters fixed in advance rather than estimated do not contribute.

The likelihood here is the marginal likelihood: it accounts for the distribution of the subject-specific random effects rather than treating each subject’s empirical Bayes estimate as an additional freely estimated parameter. Counting those individual estimates as parameters in this calculation would mix a conditional fitting approach with the conventional marginal-likelihood AICc definition. Use the same likelihood framework and parameter-counting rules for every candidate model.

When models differ in fixed effects, comparisons are ordinarily based on maximum likelihood (ML), because restricted maximum likelihood (REML) likelihoods are not generally comparable across models with different fixed-effect design matrices. More generally, the likelihoods must represent the same response data and use compatible conventions. A change in the analyzed records, outcome definition, or likelihood scale can make a numerical AICc comparison misleading.

Why the Correction Grows in Small Samples

The extra term is positive when \(K>0\) and \(n>K+1\). As \(n\) becomes large relative to \(K\), the denominator grows and the correction shrinks, so AICc approaches AIC. As \(n\) approaches \(K+1\) from above, the denominator becomes small and the correction increases sharply. This reflects the concern that estimating many parameters from a limited amount of information can make apparent fit overly optimistic.

The relationship to the large-sample limit can be seen by factoring \(n\) out of the denominator. When \(n>K+1\), $$ \frac{2K(K+1)}{n-K-1} = \frac{2K(K+1)}{n}\frac{1}{1-(K+1)/n} $$ If \((K+1)/n\) is small, the expansion \(1/(1-u)=1+u+u^2+\cdots\) gives $$ \frac{2K(K+1)}{n-K-1} = \frac{2K(K+1)}{n} \left(1+\frac{K+1}{n}+\left(\frac{K+1}{n}\right)^2+\cdots\right) $$ Thus the correction becomes small as \(n\) grows, while it is amplified when \(n\) is not large compared with \(K\). This algebra explains the behavior of the penalty; it does not make the formula universally exact for every clustered-data design. Its small-sample motivation comes from correcting the tendency of estimated fit to be optimistic under specified likelihood and sampling assumptions.

Choosing and Reporting the Sample-Size Convention

The observational unit is the unit treated as independently sampled for the finite-sample argument behind the chosen correction. For repeated-measures pharmacokinetic data, possible conventions include the number of subjects or the total number of usable concentration records. These choices answer different questions about the sampling structure. The subject count emphasizes independent clusters; the record count treats each measurement as contributing to the sample size. Because within-subject records are correlated through shared random effects and repeated observations, the latter should not be adopted automatically as if all records were independent.

A defensible analysis should identify the convention before inspecting which model it favors. It should state the number of subjects and records, report the \(n\) used in the calculation, and keep that definition consistent across all candidate models. If the ranking changes under reasonable definitions, report that sensitivity and avoid making a strong selection claim from AICc alone. If the prediction target is explicitly subject-level, the independent-subject structure is especially relevant; the chosen convention should still be justified rather than inferred solely from the target.

Diagnostics, Alternatives, and Limits

Before interpreting AICc, check the ingredients and assumptions that make the comparison meaningful:

• Confirm that all models use the same outcome data and comparable marginal likelihoods.
• Verify each parameter count from the fitted model specification, including estimated variance and covariance parameters.
• Check that the same sample-size definition is applied across models and that \(n>K+1\) for each one.
• Inspect whether the ranking is sensitive to a plausible alternative observational-unit definition, especially when many records come from a small number of subjects.
• Check model fit and estimation diagnostics separately. A lower AICc does not repair a poorly identified model, an implausible residual model, or a failed likelihood optimization.

AIC is the main alternative covered in the earlier material on AIC: it uses the same fit term but omits the finite-sample correction. It may be adequate when the sample size is large relative to the number of parameters, but it can favor a more complex model when data are limited. The Bayesian information criterion (BIC) is another information criterion, but it uses a different penalty and has a separate sample-size issue; it should not be substituted into the AICc formula. Neither criterion replaces scientific judgment, diagnostics, or validation when the goal is prediction.

The conventional AICc formula is not a universal remedy for dependence, model misspecification, weak identification, or an ambiguous sampling design. Its correction is tied to particular finite-sample assumptions and a specified \(n\). With clustered NLME data, selecting the observational unit is itself a modeling judgment. AICc is most informative when that judgment is explicit, candidate models are fitted comparably, and the resulting ranking is not overinterpreted.