Calculating AIC From Residual Sums Of Squares

From Regression Output to AIC

For Gaussian linear regression, AIC combines model fit with a penalty for the number of estimated parameters. In a comparison, a model with a smaller residual sum of squares (RSS) fits the observed responses more closely, but adding parameters also increases the penalty. The practical task is to calculate the scores on the same scale, then choose the model with the smaller AIC.

This tutorial focuses on turning sample size, RSS, and parameter counts into comparable AIC values. The likelihood-based definition of AIC is $ -2\ell(\hat{\theta})+2K $, where $\ell(\hat{\theta})$ is the maximized log-likelihood and $K$ is the number of estimated likelihood parameters. A convenient formula for regression output omits constants shared by all candidate models. It is essential to know which parameter count that formula expects.

Definition: Suppose a Gaussian linear regression model has $n$ observations, positive residual sum of squares $RSS$, and $q$ regression coefficients estimated, including the intercept if one is fitted. The reduced Gaussian-regression AIC score is $$ AIC_{\mathrm{red}}=n\log\left(\frac{RSS}{n}\right)+2q. $$ This score omits constants common to candidate models fitted to the same response observations. If $K=q+1$ counts all likelihood parameters, including the error variance, the full AIC is $$ AIC_{\mathrm{full}}=n\log\left(\frac{RSS}{n}\right)+n\left[1+\log(2\pi)\right]+2K. $$

The reduced score is not a different model-selection rule: for models using the same response data, it gives the same ranking as the full AIC when the parameter-count convention is applied consistently. In particular, $q$ counts regression coefficients but not the error variance; $K$ counts those coefficients and the variance. Do not use $2K$ in the reduced formula, or $2q$ in the full formula.

Why the RSS Formula Works

Assume the usual Gaussian linear regression model: the response vector is a linear function of the regression coefficients plus errors that are independent and normally distributed with mean zero and common variance $\sigma^2$. For a fixed fitted model, the residual sum of squares is the sum of squared differences between observed and fitted responses. The maximum-likelihood estimate of the coefficients is the least-squares estimate, and the maximum-likelihood estimate of the error variance is $RSS/n$.

The likelihood at the maximum can therefore be written in terms of $RSS/n$. The short derivation below shows how this gives the AIC formula and why the constants can be dropped only in a suitable comparison.

Write the Gaussian log-likelihood.
For residuals $e_i$, the log-likelihood is $-\frac{n}{2}\log(2\pi\sigma^2)-\frac{1}{2\sigma^2}\sum_{i=1}^n e_i^2$. At the maximum, $\sum e_i^2=RSS$ and $\hat{\sigma}^2=RSS/n$.

Substitute the variance estimate.
Replacing $\sigma^2$ by $RSS/n$ gives $-2\ell(\hat{\theta})=n\log(2\pi)+n+n\log(RSS/n)$.

Add the AIC penalty.
There are $K=q+1$ likelihood parameters when the $q$ regression coefficients and the error variance are all counted. Thus $AIC_{\mathrm{full}}=n\log(RSS/n)+n[1+\log(2\pi)]+2K$. Since $2K=2q+2$, omitting the common $n[1+\log(2\pi)]+2$ leaves $n\log(RSS/n)+2q$.

The variance estimate here is the maximum-likelihood estimate $RSS/n$, not the unbiased regression estimate $RSS/(n-q)$. Substituting the latter into the likelihood derivation does not produce the AIC formula above. The formula also presumes a positive RSS; if $RSS=0$, the fitted Gaussian likelihood has no finite maximum under this variance model, and the displayed calculation is not applicable.

Worked Comparisons

Worked Example: Correctly Comparing Two RSS Values

A hypothetical portfolio study fits two Gaussian regressions to the same 40 annual observations of a response measured in thousands of dollars. Model A has $RSS=80$ and $q=3$ regression coefficients. Model B has $RSS=64$ and $q=4$. Calculate both reduced AIC scores and their difference. Identify the preferred model.

Method and convention. Use $AIC_{\mathrm{red}}=n\log(RSS/n)+2q$, with the natural logarithm. The stated counts are regression coefficients, so the variance parameter is not added to $q$.

Model A. First, $RSS/n=80/40=2$. Therefore, $40\log(2)=27.7258872$, and the penalty is $2(3)=6$. The score is $27.7258872+6=33.7258872$, or $33.7259$ to four decimal places.

Model B. Here $RSS/n=64/40=1.6$. Thus $40\log(1.6)=18.8001452$, and the penalty is $2(4)=8$. The score is $18.8001452+8=26.8001452$, or $26.8001$ to four decimal places.

Compare using unrounded values. The difference, Model A minus Model B, is $40\log(80/64)+2(3-4)=40\log(1.25)-2=6.9257421$, which rounds to $6.9257$ to four decimal places. Model B has the lower score and is preferred by AIC. Common mistake avoided: subtracting the displayed rounded scores gives $33.7259-26.8001=6.9258$. That subtraction is correct for the displayed numbers, but it is not the correctly rounded difference calculated from the unrounded scores.

Check. Model B reduces RSS by $80-64=16$, while adding one coefficient raises the penalty by $2$. The fit improvement more than offsets that penalty, consistent with Model B's lower AIC.

Worked Example: A Larger Sample and a Smaller RSS

A hypothetical pricing analysis uses the same 60 policy-year observations for two Gaussian regressions. Model C has $RSS=150$ and $q=2$; Model D has $RSS=120$ and $q=3$. Find the reduced AIC values and determine whether the RSS improvement compensates for the additional coefficient.

Method. Calculate each $n\log(RSS/n)+2q$, then use the unrounded score difference. Because the observations and response are the same, the omitted constants are common.

For Model C, $RSS/n=150/60=2.5$, so $60\log(2.5)=54.9774439$. Its penalty is $2(2)=4$, giving $AIC_{\mathrm{red}}=58.9774439$, or $58.9774$ to four decimal places.

For Model D, $RSS/n=120/60=2$, so $60\log(2)=41.5888308$. Its penalty is $2(3)=6$, giving $AIC_{\mathrm{red}}=47.5888308$, or $47.5888$ to four decimal places.

The difference, Model C minus Model D, is $60\log(150/120)+2(2-3)=60\log(1.25)-2=11.3886131$, or $11.3886$ to four decimal places. Model D is preferred. The RSS reduction produces a larger fit reward than the two-point increase in the coefficient penalty. Common mistake avoided: comparing RSS alone would ignore the complexity penalty; here both the RSS and AIC favor Model D, but that need not always happen.

Check. The values are positive, and the smaller score belongs to the model with the smaller RSS and one additional coefficient. The difference formula agrees with subtracting the unrounded scores.

Worked Example: Keeping Full and Reduced AIC on the Same Scale

Two hypothetical Gaussian regressions use the same 30 observations. Model E has $RSS=45$ and $q=2$ regression coefficients; Model F has $RSS=40$ and $q=3$. Calculate the reduced scores, then calculate the full AIC values using the full parameter count. Check that the ranking is unchanged.

Reduced scores. For Model E, $RSS/n=45/30=1.5$, so $30\log(1.5)=12.1639532$. Its penalty is $2(2)=4$, giving $16.1639532$. For Model F, $RSS/n=40/30=4/3$, so $30\log(4/3)=8.6304622$. Its penalty is $2(3)=6$, giving $14.6304622$. To four decimal places, the scores are $16.1640$ and $14.6305$, respectively.

Full scores. Model E has $K=q+1=3$ likelihood parameters, and Model F has $K=4$. The common term $30[1+\log(2\pi)]$ is $30(2.8378771)=85.1363120$. Add this term and the full penalties $2K$: Model E has $12.1639532+85.1363120+6=103.3002652$; Model F has $8.6304622+85.1363120+8=101.7667742$. Thus the full AIC values are $103.3003$ and $101.7668$ to four decimal places.

Compare and check. The reduced-score difference, E minus F, is $30\log(45/40)+2(2-3)=30\log(1.125)-2=1.5334911$. The full-score difference is also $103.3002652-101.7667742=1.5334910$ at the shown precision; the tiny displayed discrepancy is rounding of intermediate values, and the unrounded difference is $1.5334911$. Model F has the lower AIC on either scale. Common mistake avoided: using $K=3,4$ in the reduced formula would add the same extra $2$ to both scores here, but mixing conventions can lead to incorrect absolute values or inconsistent comparisons.

Conditions for a Meaningful Comparison

AIC ranks candidate models by their estimated relative information loss, with smaller values preferred. The numerical score is useful only if the likelihoods being compared are on a consistent basis. In the setting covered here, check the following before calculating:

Same response observations: use the same $n$ observations and response variable for each candidate. If models use different observations, their likelihood scores do not share the same comparison basis.
Same likelihood specification: the formula assumes Gaussian errors with a common variance within each model. A change in the response scale or likelihood requires care; do not assume the same constants can be discarded.
Consistent parameter counts: decide whether the supplied count is $q$, the regression coefficients, or $K$, all likelihood parameters including variance. Then use the matching formula throughout.
Natural logarithms: the likelihood formula uses $\log$, meaning the natural logarithm. A base-10 logarithm gives a different and incorrect score.

A frequent exam trap is to treat a constant as removable without checking whether it is actually common. In a direct comparison on the same $n$ observations, $n[1+\log(2\pi)]$ is the same for all candidates. The additional $2$ difference between the full and reduced formulas comes from counting the common variance parameter in $K$ rather than in $q$. If all candidates use the same variance convention, that term is common too.

Key takeaway: For a Gaussian regression with $n$ observations, RSS, and $q$ regression coefficients, calculate $n\log(RSS/n)+2q$. Use the full likelihood count $K=q+1$ only with the full formula. Compare unrounded scores from models fitted to the same response observations; the smaller AIC is preferred. This calculation is aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.

Check Your Understanding

For each comparison, use the reduced score unless the question explicitly requests the full AIC. Round final scores or differences to four decimal places. Use unrounded values when calculating a difference.

Two Gaussian regressions use the same 20 observations. Model G has $RSS=40$ and $q=2$; Model H has $RSS=30$ and $q=3$. Calculate the difference $AIC_G-AIC_H$ and identify the lower-AIC model. (Answer: $3.7536$; Model H.)
With $n=25$, Model J has $RSS=50$, $q=1$, and Model K has $RSS=40$, $q=2$. Which model has lower reduced AIC, and by how much? (Answer: Model K, by $3.5786$.)
For the same 100 responses, Model L has $RSS=250$, $q=3$, while Model M has $RSS=225$, $q=4$. Compute $AIC_L-AIC_M$ and state the preferred model. (Answer: $8.5361$; Model M.)
A single Gaussian regression has $n=12$, $RSS=24$, and full likelihood parameter count $K=3$. Calculate its full AIC, including the Gaussian constant, to four decimal places. (Answer: $48.3723$.)
Two models on the same 40 observations have $(RSS,q)=(100,2)$ and $(90,3)$. Find the reduced AIC difference, first model minus second model, and interpret it. (Answer: $2.2144$; the second model has lower AIC.)

What You'll Learn