Why Careful Calculation Matters
When competing regression models are fitted to the same response data, a model with more predictors can reduce its residual sum of squares (RSS), even if the added complexity is not worthwhile. Adjusted R-squared provides a score that accounts for both fit and the number of fitted predictor coefficients. A useful comparison therefore requires more than reading off the RSS: you must count coefficients correctly, use the appropriate degrees of freedom, and verify that the models use the same observations.
This tutorial works through those calculations for hypothetical risk-modeling settings. The objective is to compute and interpret adjusted R-squared scores, not to establish that one model is universally preferable. The score is one comparison criterion, and its meaning depends on how the models were fitted and on what is being compared.
Formula and Conditions for a Comparison
Suppose an ordinary least-squares regression with an intercept is fitted to \(n\) observations. Let \(p\) be the number of predictor coefficients, not including the intercept. Write \(y_i\) for the observed response, \(\hat{y}_i\) for its fitted value, and \(\bar{y}\) for the sample mean. The residual sum of squares and total sum of squares are \(RSS=\sum_{i=1}^n(y_i-\hat{y}_i)^2\) and \(TSS=\sum_{i=1}^n(y_i-\bar{y})^2\). The formula below requires \(TSS>0\) and positive residual degrees of freedom, \(n-p-1>0\).
The familiar unadjusted coefficient of determination is \(R^2=1-RSS/TSS\). It measures the fraction of the observed response variation about its mean accounted for by the fitted model. Adjusted R-squared instead compares the residual mean square with the sample variance of the response. Its arithmetic formula can be used without assuming normally distributed errors. For the usual regression interpretation and related inference, assumptions such as independent errors with mean zero and constant variance are relevant; those assumptions do not change the score’s calculation.
For a direct comparison, use the same response and the same analysis observations, so that the models share \(n\) and \(TSS\). If models use different rows, their residual sums of squares and total sums of squares refer to different data, so the resulting scores do not provide the same like-for-like comparison.
Why the Degrees-of-Freedom Adjustment Appears
The adjustment follows from comparing two mean squares. The residual variation is divided by its residual degrees of freedom, \(n-p-1\), because the regression estimates an intercept and \(p\) predictor coefficients. The variation of responses around their sample mean is divided by \(n-1\). Taking their ratio gives a residual-to-total variation measure on a degrees-of-freedom-adjusted scale; subtracting that ratio from one yields adjusted R-squared.
Thus the adjusted score is not obtained by subtracting a fixed amount for every predictor. Both the RSS and the residual degrees of freedom matter. The intercept is included in the model, but it is not counted in \(p\); its estimation is already reflected in the \(n-p-1\) residual degrees of freedom.
Worked Comparisons
Worked Example: Ranking Three Models on a Common Sample
A hypothetical insurer models the annual cost per policy using the same \(n=40\) policies for three candidate regressions. The response has \(TSS=1{,}200\). Model A has \(p=2\) predictor coefficients and \(RSS=720\); Model B has \(p=4\) and \(RSS=650\); Model C has \(p=6\) and \(RSS=620\). Calculate each adjusted R-squared and identify the highest-scoring model. The method is to compute the residual degrees of freedom and substitute each model’s RSS into the same formula.
Model A. The residual degrees of freedom are \(40-2-1=37\). Its unadjusted score is \(1-720/1{,}200=0.4000\). For the adjusted score, \(RSS/(n-p-1)=720/37\), and \(TSS/(n-1)=1{,}200/39\). Therefore \(R^2_{\mathrm{adj}}=1-(720/37)/(1{,}200/39)=1-28{,}080/44{,}400=0.3675676\), or \(0.3676\) to four decimal places.
Model B. The residual degrees of freedom are \(40-4-1=35\). Its unadjusted score is \(1-650/1{,}200=0.4583\). The adjusted calculation is \(1-(650/35)/(1{,}200/39)=1-25{,}350/42{,}000=0.3964286\), or \(0.3964\) to four decimal places.
Model C. The residual degrees of freedom are \(40-6-1=33\). Its unadjusted score is \(1-620/1{,}200=0.4833\). The adjusted calculation is \(1-(620/33)/(1{,}200/39)=1-24{,}180/39{,}600=0.3893939\), or \(0.3894\) to four decimal places.
Check and comparison. The RSS values decrease from 720 to 620, and the unadjusted scores increase, but the adjusted scores do not increase throughout. Model B has the highest adjusted R-squared, \(0.3964\), compared with \(0.3894\) for Model C and \(0.3676\) for Model A. The unrounded difference between Models B and C is about \(0.0070\), and the displayed four-decimal scores also differ by \(0.0070\). The common mistake this calculation avoids is ranking models by RSS alone: that would select Model C, despite its lower adjusted score.
Worked Example: Counting Coefficients for a Categorical Predictor
A hypothetical claim-cost model uses \(n=28\) policies, with \(TSS=540\). Model X contains two continuous predictors and a categorical predictor with four levels. Model Y contains three continuous predictors and a categorical predictor with two levels. The factor levels are represented by indicator coefficients, with one level serving as the reference. Model X has \(RSS=324\), and Model Y has \(RSS=330\). Calculate both adjusted scores and compare them.
Count the predictor coefficients. A categorical predictor with \(k\) levels represented using a reference level requires \(k-1\) indicator coefficients. Thus Model X has \(p=2+(4-1)=5\) predictor coefficients. Model Y has \(p=3+(2-1)=4\). Neither count includes the intercept.
Model X. Its residual degrees of freedom are \(28-5-1=22\). Its unadjusted R-squared is \(1-324/540=0.4000\). The adjusted score is \(1-(324/22)/(540/27)=1-324/440=0.2636364\), or \(0.2636\) to four decimal places. Here \(540/27=20\), which makes the denominator calculation explicit.
Model Y. Its residual degrees of freedom are \(28-4-1=23\). Its unadjusted R-squared is \(1-330/540=0.3889\). Its adjusted score is \(1-(330/23)/(540/27)=1-330/460=0.2826087\), or \(0.2826\) to four decimal places.
Check and comparison. Model X has the smaller RSS and the larger unadjusted R-squared, but Model Y has the higher adjusted R-squared. The comparison is valid because both models use the same 28 policies and the same response, so they have a common \(TSS\). The common mistake this example avoids is counting a four-level factor as one predictor coefficient. With reference-level indicators, it contributes three coefficients, which affects the residual degrees of freedom and the adjusted score.
Worked Example: Calculating and Checking Close Candidate Scores
A hypothetical risk team fits three models to the same \(n=64\) observations of a response with \(TSS=2{,}560\). Model M has \(p=3\) and \(RSS=1{,}800\); Model N has \(p=5\) and \(RSS=1{,}680\); Model Q has \(p=4\) and \(RSS=1{,}740\). Calculate their adjusted R-squared values to four decimal places and rank them.
Model M. Its residual degrees of freedom are \(64-3-1=60\). The unadjusted score is \(1-1{,}800/2{,}560=0.296875\). The adjusted score is \(1-(1{,}800/60)/(2{,}560/63)=1-113{,}400/153{,}600=0.26171875\), or \(0.2617\) to four decimal places.
Model N. Its residual degrees of freedom are \(64-5-1=58\). The unadjusted score is \(1-1{,}680/2{,}560=0.34375\). The adjusted score is \(1-(1{,}680/58)/(2{,}560/63)=1-105{,}840/148{,}480=0.2871767\), or \(0.2872\) to four decimal places.
Model Q. Its residual degrees of freedom are \(64-4-1=59\). The unadjusted score is \(1-1{,}740/2{,}560=0.3203125\). The adjusted score is \(1-(1{,}740/59)/(2{,}560/63)=1-109{,}620/151{,}040=0.2742313\), or \(0.2742\) to four decimal places.
Check and comparison. The ordering by adjusted score is N, Q, then M. Model N’s adjusted score exceeds Model Q’s by approximately \(0.0129\) before rounding; the displayed scores differ by \(0.0130\). Keeping unrounded values until the ranking is established avoids letting rounding determine a close comparison. The common mistake this example avoids is using \(n-p\) rather than \(n-p-1\) for residual degrees of freedom in a model with an intercept.
Common Mistakes and Practical Checks
Before ranking candidate models, make a small calculation table with one row per model. Record \(n\), \(p\), \(RSS\), \(TSS\), residual degrees of freedom, and adjusted R-squared. This makes it easier to spot a mismatched sample size or an incorrect coefficient count before comparing the final scores.
- Including the intercept in \(p\). For a model with an intercept and \(p\) predictor coefficients, use \(n-p-1\), not \(n-p\), for the residual degrees of freedom.
- Counting a factor as one coefficient. With a reference level, a categorical predictor with \(k\) levels contributes \(k-1\) indicator coefficients. Count the coefficients actually fitted.
- Comparing RSS values as if they were adjusted scores. RSS does not account for predictor count, and it is not directly comparable when the analysis samples differ.
- Using different observations without warning. If missing data leave models with different rows, their \(TSS\) values can differ too. A higher adjusted score is then not automatically a like-for-like model comparison.
- Rounding before ranking. Keep enough digits during calculation, compare unrounded results, and round only when reporting. The difference of displayed rounded scores may not equal the unrounded difference rounded to the same number of places.
A higher adjusted R-squared identifies the better-scoring model among the candidates under this particular measure. It does not, by itself, establish that the selected model is appropriate for every purpose or that its predictors have a causal interpretation. Interpret the result in the context of the response, the analysis sample, and the modeling objective.
Check Your Understanding
For each problem, calculate as requested before checking the answer. Unless stated otherwise, models include an intercept, and report adjusted R-squared to four decimal places.
- A model fitted to \(n=25\) observations has \(TSS=300\), \(RSS=180\), and \(p=2\) predictor coefficients. Calculate its adjusted R-squared. (Answer: \(0.3455\))
- Two models use the same \(n=36\) observations with \(TSS=700\). Model A has \(p=3\), \(RSS=420\); Model B has \(p=5\), \(RSS=390\). Calculate both adjusted scores and identify the higher-scoring model. (Answer: Model A \(0.3438\); Model B \(0.3500\); Model B scores higher.)
- A model uses \(n=50\) observations and has \(TSS=1{,}200\), \(RSS=860\), two continuous predictors, and a five-level categorical predictor represented with a reference level. Count \(p\) and calculate adjusted R-squared. (Answer: \(p=6\); adjusted R-squared \(0.1833\).)
- Two models have \(p=2\) and report the following summaries: Model U has \(n=60\), \(TSS=900\), \(RSS=540\); Model V has \(n=55\), \(TSS=820\), \(RSS=492\). Calculate each adjusted score, then state whether the scores alone support a direct ranking if the models used different observations. (Answer: Model U \(0.3789\); Model V \(0.3769\); no direct like-for-like ranking is supported because the analysis samples differ.)
- Two models use the same \(n=80\) observations with \(TSS=4{,}000\). Model C has \(p=3\), \(RSS=2{,}900\); Model D has \(p=5\), \(RSS=2{,}800\). Calculate the adjusted scores and identify the higher-scoring model. (Answer: Model C \(0.2464\); Model D \(0.2527\); Model D scores higher.)