AICc for Small Samples

Why Correct AIC for Small Samples?

AIC compares candidate models by balancing fit against a penalty for the number of estimated parameters. Its penalty is useful for comparing predictive performance, but in a small sample it may not adequately account for the optimism that can arise when a model has many parameters relative to the available observations. The corrected Akaike Information Criterion, usually written AICc, adds a finite-sample adjustment to AIC.

The adjustment is not equally important for every comparison. It is small when the sample is large compared with the number of parameters, but it can become substantial when the sample is small or the candidate model is complex. Because it varies across models, it can change their ranking.

Definition: For a candidate model with maximized log-likelihood $\ell$, sample size $n$, and $K$ estimated parameters, define $$ \mathrm{AIC}=-2\ell+2K $$ and, when $n>K+1$, $$ \mathrm{AICc}=\mathrm{AIC}+\frac{2K(K+1)}{n-K-1}. $$ The added quantity is the small-sample correction. Smaller AICc values indicate a better relative balance of fit and complexity among the models being compared.

What Counts as a Parameter?

In the usual Gaussian regression setting, $K$ counts all parameters estimated by maximum likelihood: the regression coefficients and the error variance. For example, a model with an intercept, two slopes, and an estimated error variance has $K=4$. The sample size $n$ is the number of observations used in fitting the likelihood, not the number of coefficients.

The formula should be applied to candidate models fitted to the same response observations using a comparable likelihood and the same response scale. In the standard regression setting, the observations are treated as independent and the Gaussian model specifies the likelihood used to calculate AIC. Specialized likelihoods or dependence structures can require a correction tailored to that setting; the displayed expression is the conventional AICc formula used here for regression comparisons.

AICc is a relative comparison, not a test of whether a model is true. Its numerical value has no universal cutoff: the useful information comes from comparing values for an appropriate candidate set. AICc also cannot make an unfair comparison fair; for example, candidates fitted to different observations do not provide a straightforward comparison.

Calculating the Adjustment

To calculate AICc, first identify $n$ and count $K$ consistently, then calculate the correction, and finally add it to AIC. Always inspect the denominator $n-K-1$. The standard formula is defined only when $n>K+1$.

Worked Example: Correcting AIC for One Regression Model

Suppose a hypothetical Gaussian regression uses $n=24$ patients’ observations to model a continuous outcome. It has an intercept, two slopes, and an estimated error variance, so $K=4$. Suppose the model’s AIC is 88.0. Calculate AICc.

The model has $24-4-1=19$ in the denominator, so the formula is defined. Its correction is $$ \frac{2K(K+1)}{n-K-1} =\frac{2(4)(5)}{24-4-1} =\frac{40}{19} =2.105. $$

Therefore, $$ \mathrm{AICc}=88.0+2.105=90.105. $$

For this candidate, the finite-sample adjustment raises the criterion by about 2.105 AIC units. The result is still a relative score: by itself, 90.105 does not say whether the model is adequate or whether another candidate is preferable.

When the Correction Changes a Ranking

AICc adds a different amount to models with different parameter counts. A more complex model generally receives a larger correction, so a model with a slightly better AIC may no longer have the smallest AICc. The correction can also be the same for all candidates if they have the same $K$, sample size, and likelihood setup; in that case it does not change their ranking.

Worked Example: A Small AIC Advantage Is Reversed

Consider two hypothetical regression models fitted to the same 20 observations. Model A has three regression coefficients plus an estimated error variance, giving $K=4$, and has AIC $=100.0$. Model B has two regression coefficients plus an estimated error variance, giving $K=3$, and has AIC $=101.0$. AIC alone favors Model A because $100.0<101.0$.

For Model A, $$ \mathrm{AICc}_A =100.0+\frac{2(4)(5)}{20-4-1} =100.0+\frac{40}{15} =102.667. $$

For Model B, $$ \mathrm{AICc}_B =101.0+\frac{2(3)(4)}{20-3-1} =101.0+\frac{24}{16} =102.500. $$

AICc now favors Model B: $102.500<102.667$. The difference is only 0.167, so this is a very close comparison, not strong evidence that one model is definitively best. The example shows why reporting only the AIC ranking can be misleading when the sample is small relative to model complexity.

Both candidates were fitted to the same observations, and both have $n>K+1$: the denominators are 15 and 16. Those conditions allow this comparison using the stated formula.

Why Sample Size and Parameter Count Matter

The correction depends on $n$ and $K$ together. With a fixed parameter count, increasing $n$ makes the denominator larger and the correction smaller. With a fixed $n$, increasing $K$ increases the numerator and decreases the denominator, making the correction grow. Thus, a model can receive a consequential adjustment even when its absolute number of parameters does not seem large, if the sample itself is small.

Worked Example: Comparing Sample Sizes and Model Complexity

Suppose a hypothetical regression has $K=5$ estimated parameters. For a sample of $n=30$, the correction is $$ \frac{2(5)(6)}{30-5-1} =\frac{60}{24} =2.500. $$ For a sample of $n=120$ with the same parameter count, it is $$ \frac{2(5)(6)}{120-5-1} =\frac{60}{114} =0.5263. $$

The correction is much smaller for the larger sample. Now, with $n=30$, consider a more complex model with $K=12$, such as 11 regression coefficients and an estimated error variance. Its correction is $$ \frac{2(12)(13)}{30-12-1} =\frac{312}{17} =18.35. $$

This much larger adjustment reflects the combination of a relatively small sample and many estimated parameters. These computations do not show that the more complex model is wrong; they show that AICc penalizes its complexity more heavily under these conditions. Each comparison still requires the models to be fitted on comparable observations and likelihoods.

A Short Check on the Formula

The form of the adjustment explains its main behavior. For fixed $K$, as $n$ grows without bound, the denominator grows without bound while the numerator stays fixed: $$ \lim_{n\to\infty}\frac{2K(K+1)}{n-K-1}=0. $$ So AICc approaches AIC for a large sample when the parameter count stays fixed. In the other direction, as $n$ approaches $K+1$ from above, the positive denominator approaches zero and the correction grows without bound. For fixed $n$, adding a parameter increases $K(K+1)$ and reduces $n-K-1$, so the correction increases as well, as long as the formula remains defined. These facts justify checking the sample size and parameter count rather than assuming the adjustment is negligible.

Key takeaway: AICc equals AIC plus $2K(K+1)/(n-K-1)$. Count all estimated parameters consistently, including the error variance in the usual Gaussian regression setting. The correction is most consequential when $n$ is not large relative to $K$, and the formula requires $n>K+1$.

Common Mistakes and Practical Use

Counting only slopes. In the usual Gaussian regression setting, $K$ also includes the intercept and the estimated error variance. Leaving parameters out makes the correction too small.
Using the same correction for every model. The correction depends on $K$ as well as $n$. Calculate it separately for each model if their parameter counts differ.
Assuming a large correction proves a model is bad. The adjustment changes a model’s relative criterion score; it does not establish that the model is scientifically false or that its predictions are useless.
Calculating when $n\leq K+1$. The denominator is then zero or negative, so the standard formula is undefined or inapplicable. Do not interpret the resulting arithmetic as an AICc score.
Treating the smallest AICc as certainty. The minimum identifies the lowest score among the candidates considered, not a probability that the selected model is true. The candidate set and intended use still matter.

In practice, report the candidate models, their AICc values, and the parameter-count convention used. If AIC and AICc rank models differently, show both rankings and note whether the correction is large relative to the AIC differences. This makes clear whether the small-sample adjustment affects the comparison rather than hiding that choice.

Check Your Understanding

Answer these questions using the definition and examples above.

A Gaussian regression has an intercept, four slopes, and an estimated error variance. What value of $K$ should be used in the conventional AICc formula?
For $n=18$ and $K=4$, calculate the finite-sample correction and state whether the formula is defined.
Two candidate models have the same $n$ and $K$. Will the AICc correction change their AIC ranking? Explain.
Why can a more complex model receive a much larger AICc correction in a small sample?
What happens to the correction as $n$ grows while $K$ stays fixed?

What You'll Learn