Why AICc Adds a Small-Sample Correction

Why AIC Needs a Small-Sample Adjustment

AIC balances model fit against complexity, but its familiar penalty $2K$ is based on an approximation that works best when the sample is large relative to the number of estimated parameters. In a short sample, fitting a model can make its observed fit look especially favorable: the same data were used both to estimate the model and to assess that fit. AICc adds a further penalty to account for this finite-sample optimism.

The adjustment is not a new measure of fit or a different way to count observations. It changes the penalty attached to the fitted model. The change is small when $n$ is large relative to the parameter count, and can be substantial when the sample is small or the model is complex.

Definition: For a fitted model with maximized log-likelihood $\ell(\hat{\theta})$, AIC is $-2\ell(\hat{\theta})+2K$. The small-sample corrected criterion is $$ \mathrm{AICc}=\mathrm{AIC}+\frac{2K(K+1)}{n-K-1}, $$ where $n$ is the sample size and $K$ is the total number of estimated parameters in the likelihood. Smaller AICc indicates a preferable balance of fit and complexity among the candidate models being compared.

In the Gaussian regression convention used here, $K$ includes the regression coefficients and the error variance. Thus a model with $q$ regression coefficients, including the intercept, has $K=q+1$. This distinction matters: substituting $q$ where the formula requires $K$ gives a different correction. Some references use another parameter-count convention, so check the definition accompanying any reported AICc value before comparing calculations.

The Gaussian Regression Setting

For a candidate Gaussian linear regression, write the model as $Y=X\beta+\varepsilon$, where $Y$ is the response vector, $X$ is the fixed design matrix, and the errors are independent with a common normal distribution. The usual likelihood-based formula assumes that the mean is linear in the stated coefficients, the error variance is estimated, and the fitted model has full column rank. The candidate models must be fitted to the same response observations for their information criteria to be compared directly.

Conditions for the standard formula: Count all estimated likelihood parameters consistently, including the variance; require $n>K+1$, so the denominator $n-K-1$ is positive; and compare models fitted to the same data under the same response scale and likelihood convention. For a regression with $q$ coefficients and an estimated variance, the sample-size condition is $n>q+2$. A positive denominator makes the formula defined, but does not guarantee that a very small-sample approximation is reliable.

The boundary has a practical interpretation. If the fitted model uses nearly as many parameters as the data can support, the estimated coefficients and variance can be highly sensitive to the particular sample. The formula responds by imposing a much larger correction. At or below the boundary, AICc is undefined under this convention; it is not appropriate to report a negative, infinite, or improvised correction in its place.

Why the Correction Has This Form

AIC’s $2K$ penalty is an asymptotic estimate of the optimism in evaluating a fitted model on the same data used to estimate it. In Gaussian regression with an estimated variance, finite-sample likelihood calculations give a larger optimism penalty. The normal-theory calculation depends on the residual sum of squares as well as on the fitted coefficients; estimating the variance from a limited number of residual degrees of freedom increases the adjustment.

To see where the denominator comes from, let $q$ be the number of regression coefficients, so $K=q+1$. Under a correctly specified Gaussian regression with fixed design, the residual sum of squares divided by the true variance has a chi-square distribution with $\nu=n-q$ degrees of freedom. For a chi-square random variable $U$ with $\nu>2$, the inverse moment is $E(1/U)=1/(\nu-2)$. Here,

\nu-2=n-q-2=n-(q+1)-1=n-K-1.

That inverse-moment term explains why finite-sample risk calculations for an estimated variance require the $n-K-1$ denominator, and why the requirement is $n>K+1$. Combining the Gaussian regression contributions from the fitted coefficients and variance gives the finite-sample optimism penalty $2Kn/(n-K-1)$. AIC already includes the asymptotic penalty $2K$. Subtracting that AIC penalty leaves the extra amount:

\frac{2Kn}{n-K-1}-2K = \frac{2Kn-2K(n-K-1)}{n-K-1} = \frac{2K(K+1)}{n-K-1}.

This is the AICc correction. The calculation explains its form under the stated Gaussian regression setup; it does not mean that AICc removes every possible source of prediction error or makes a misspecified model correct. As with other information criteria, its usefulness depends on the model and comparison being appropriate.

Worked Example: Computing AICc for One Model

Suppose a hypothetical Gaussian regression has $n=30$ observations, three regression coefficients including the intercept, and an estimated error variance. Its reported AIC is 100. Find AICc.

The total parameter count is $K=3+1=4$, where the extra parameter is the variance. The condition is satisfied because $30>4+1$. The correction is

\frac{2K(K+1)}{n-K-1} = \frac{2(4)(5)}{30-4-1} = \frac{40}{25} = 1.600.

Therefore, $\mathrm{AICc}=100+1.600=101.6$. The correction raises the criterion above AIC because the finite-sample penalty is larger than AIC’s asymptotic penalty. The value 101.6 is a relative model-comparison score, not a probability or a direct measure of prediction error in response units.

How the Adjustment Changes with Sample Size

The correction depends on both $n$ and $K$. For a fixed parameter count, increasing $n$ makes the denominator larger, so the adjustment shrinks. For a fixed sample size, increasing $K$ increases the numerator and reduces the denominator, so the adjustment grows. This is why a model with many parameters can receive a particularly large AICc penalty in a small sample.

As $n$ becomes large relative to $K$, the correction approaches zero:

\lim_{n\to\infty}\frac{2K(K+1)}{n-K-1}=0, \qquad \lim_{n\to\infty}\mathrm{AICc}=\mathrm{AIC} \quad\text{for fixed }K.

This limiting relationship does not make AICc and AIC interchangeable at every sample size. The correction can be consequential even when the formula is defined, especially if two candidate models have similar AIC values.

Worked Example: Checking the Boundary and the Size of the Correction

Consider a hypothetical model with $q=8$ regression coefficients, including the intercept, and an estimated variance. Then $K=9$. Determine whether AICc is defined when $n=10$, and calculate its correction when $n=11$.

When $n=10$, the condition requires $10>9+1$, which is false. Equivalently, the denominator is $10-9-1=0$. AICc is undefined for this model and sample size under the stated formula.

When $n=11$, the condition $11>10$ holds, but only just. The denominator is 1, so

\frac{2K(K+1)}{n-K-1} = \frac{2(9)(10)}{11-9-1} = 180.

The correction is 180 AIC units. This large value signals that the model has very little information beyond what is needed to estimate its parameters. It is not a reason to ignore the formula or reduce the parameter count; it is a warning that this model is poorly supported by such a small sample.

Using AICc to Compare Candidate Regressions

AICc is used to rank candidate models, not to declare that the lowest-scoring model is certainly the true model. When models are fitted to the same observations and response, compute each model’s AICc using the same parameter-count convention. The model with the lower AICc has the more favorable fit-complexity balance according to this criterion. The size of the difference helps describe how separated the scores are, but does not by itself establish that one model is scientifically correct.

Worked Example: AIC and AICc Can Rank Models Differently

Two hypothetical Gaussian regressions are fitted to the same $n=25$ observations. Model A has AIC 80 and $K=3$; Model B has AIC 78 and $K=6$. Each $K$ includes the estimated variance. Compare their AICc values.

For Model A, $25>3+1$, and the correction is

\frac{2(3)(4)}{25-3-1} = \frac{24}{21} \approx 1.143.

Thus Model A has $\mathrm{AICc}\approx80+1.143=81.14$. For Model B, $25>6+1$, and

\frac{2(6)(7)}{25-6-1} = \frac{84}{18} \approx 4.667.

Thus Model B has $\mathrm{AICc}\approx78+4.667=82.67$. AIC alone favors Model B because 78 is lower than 80. AICc favors Model A because 81.14 is lower than 82.67; the additional small-sample penalty outweighs Model B’s two-point advantage in AIC. This ranking is evidence about relative support within these candidates, not proof that Model A is the correct account.

Common Mistakes and Practical Interpretation

Counting only the regression coefficients. If the formula defines $K$ as all estimated likelihood parameters, include the estimated variance. For $q$ regression coefficients, use $K=q+1$.
Using AICc when its denominator is zero or negative. Check $n>K+1$ before calculating. If the condition fails, the formula is undefined; do not replace the denominator with an arbitrary small number.
Assuming any positive denominator ensures a dependable comparison. A model just above the boundary can have a very large correction and may be unstable. The condition is necessary for this formula, not a guarantee of adequate data.
Comparing scores from incompatible fits. AICc comparisons require a consistent likelihood basis, including the same response observations and response scale. A difference in criterion values is meaningful only within a fair candidate set.
Treating the lowest AICc as a probability statement. AICc ranks estimated information loss among the candidates; it is not the probability that a model is true.

The central practical point is that the correction is not an arbitrary penalty added to discourage complicated models. It adjusts for finite-sample optimism in likelihood-based model assessment, and its magnitude makes the sample-size-to-parameter-count relationship explicit. Always state the parameter-count convention, verify the denominator condition, and interpret the ranking as comparative evidence.

Check Your Understanding

Try these questions before moving on.

A Gaussian regression has four regression coefficients, including the intercept, and an estimated error variance. What value of $K$ is used under the convention in this tutorial?
For $n=18$ and $K=6$, is the standard AICc formula defined? Show the denominator.
Why does the AICc correction shrink as the sample size increases while $K$ stays fixed?
Two models have AIC values 50 and 51. Explain why their AICc ranking could differ from their AIC ranking.
What does a very large AICc correction suggest about a model fitted just above the sample-size boundary?