Adjusted R-Squared

Why Adjusted R-Squared Is Used

Ordinary R-squared summarizes how much of the observed variation in an outcome is accounted for by a fitted regression model. But when a model includes more predictor coefficients, its ordinary R-squared cannot decrease, even if those additions contribute little useful information. Adjusted R-squared gives a related summary that accounts for the number of coefficients fitted.

This adjustment matters when describing the fit of a model with several predictors. It does not establish that a model is correct, that its predictors have meaningful effects, or that it will predict well for new observations. It is a particular summary of in-sample fit, and it should be interpreted as such.

Model Setup and Definition

Consider an ordinary least-squares regression model with an intercept, fitted to \(n\) observations of one quantitative outcome. Let \(k\) be the number of fitted predictor coefficients, not counting the intercept. The model has \(k+1\) fitted coefficients in total. Assume the design columns are linearly independent, so these coefficients are estimable separately, and that \(n>k+1\). Let \(RSS\) be the residual sum of squares and \(TSS\) the total sum of squares around the outcome’s sample mean. We also require \(TSS>0\), so the observed outcome is not constant.

These conditions make the usual degrees of freedom and adjusted R-squared well-defined. No normality assumption is needed simply to calculate this fit summary. The definition below is for a model with an intercept; do not automatically apply it to a regression fitted without one.

The numerator of the ratio is the residual mean square: the residual sum of squares divided by its residual degrees of freedom. The denominator is the sample variance of the observed outcome: the total sum of squares divided by its total degrees of freedom. Both quantities are measured in squared outcome units, so their ratio, and therefore adjusted R-squared, has no units.

The usual ordinary R-squared is \(R^2=1-RSS/TSS\). It compares the two sums of squares without dividing each by its degrees of freedom. Adjusted R-squared makes that division before forming its fit summary. Thus, it accounts for the number of fitted predictor coefficients in a way that ordinary R-squared does not.

Reading the Degrees of Freedom

The total degrees of freedom \(n-1\) reflect that the total sum of squares is calculated around the sample mean, which has been estimated from the data. The residual degrees of freedom \(n-k-1\) account for the intercept and the \(k\) predictor coefficients fitted in the regression. As more coefficients are included, fewer residual degrees of freedom remain.

Here \(k\) counts coefficients, not necessarily the number of named variables in a data description. For instance, a categorical predictor with four groups is commonly represented by three indicator coefficients when the model also has an intercept. Those three coefficients all contribute to \(k\). The intercept itself is not included in \(k\), but it is included in the residual-degree-of-freedom calculation.

What the Adjustment Changes

Ordinary R-squared treats the observed sums of squares as the quantities to compare. Adjusted R-squared instead compares sums of squares after scaling them by their respective degrees of freedom. That difference makes the model’s coefficient count relevant to the summary. It discourages interpreting a larger ordinary R-squared, by itself, as evidence that a model with more fitted coefficients is preferable.

This is a descriptive adjustment, not a formal test of whether a predictor belongs in the model. It does not provide a p-value, a confidence interval, or a guarantee about prediction for new cases. Nor does an adjusted value reveal which predictors matter. Those questions require other tools and reasoning.

A Short Algebraic Check

The definition can be written directly in terms of the residual mean square and the sample variance of the outcome. Let \(MSE=RSS/(n-k-1)\), and let \(s_y^2=TSS/(n-1)\) denote the sample variance of the observed outcome. Substitution gives $$ R_{\mathrm{adj}}^2 = 1-\frac{MSE}{s_y^2}. $$

This identity follows simply by replacing the two degrees-of-freedom-scaled sums of squares in the definition with their names. It clarifies the comparison: adjusted R-squared is one minus residual variation per residual degree of freedom divided by total variation per total degree of freedom. Ordinary R-squared, in contrast, uses \(RSS/TSS\) directly. The distinction is in the scaling, not in the outcome being modeled.

Negative Values and Interpretation

Unlike ordinary R-squared for an intercept model, adjusted R-squared is not constrained to be between 0 and 1. It can be negative if the residual mean square exceeds the sample variance of the outcome. A negative value is not a “negative percentage explained.” It says that, under this adjusted comparison, residual variation per residual degree of freedom is larger than total outcome variation per total degree of freedom.

Common Mistakes

• Using the wrong degrees of freedom. For the standard intercept model, use \(n-k-1\) for residual degrees of freedom and \(n-1\) for total degrees of freedom. Do not count the intercept as one of the \(k\) predictors.
• Counting variables instead of coefficients. A categorical predictor can require several indicator coefficients. Count the coefficients actually fitted.
• Calling adjusted R-squared a literal proportion explained. Ordinary R-squared has a direct sum-of-squares interpretation for the usual intercept model. Adjusted R-squared is a degrees-of-freedom-adjusted summary and can be negative.
• Interpreting it as a test or a prediction guarantee. An adjusted R-squared value alone does not establish statistical significance, practical importance, causation, or out-of-sample predictive accuracy.
• Comparing values without checking the setup. At minimum, ensure the outcome and observations are the same and that the usual intercept-model definition applies before treating the summaries as comparable.