When R-Squared and Adjusted R-Squared Disagree

Why the Two Fit Summaries Can Disagree

Adding a predictor to a regression model gives the fitted model another way to describe the observed outcomes. As a result, its residual sum of squares cannot increase, and ordinary R-squared cannot decrease. But a more flexible model can improve its fit only slightly. Adjusted R-squared accounts for the number of predictors, so it can fall even when ordinary R-squared rises.

That disagreement is useful as a warning: the additional predictor has improved the fit to these observations, but the improvement may be small relative to the extra model complexity. It is not, by itself, proof that the predictor is irrelevant, that the larger model is wrong, or that either model will perform poorly on new data.

Definitions and Conditions for Comparison

For a response \(Y\) observed on \(n\) individuals, consider an ordinary least-squares regression with an intercept and \(k\) predictors. Let \(y_i\) be the observed response, \(\hat{y}_i\) its fitted value, and \(\bar{y}\) the sample mean of the responses. The residual sum of squares is \(SSE=\sum_{i=1}^n(y_i-\hat{y}_i)^2\), and the total sum of squares around the mean is \(SST=\sum_{i=1}^n(y_i-\bar{y})^2\).

Definition: For an ordinary least-squares regression with an intercept, ordinary R-squared is \(R^2=1-SSE/SST\). With \(k\) predictors and residual degrees of freedom \(n-k-1\), adjusted R-squared is \(\bar{R}^2=1-\left[SSE/(n-k-1)\right]\big/\left[SST/(n-1)\right]\). The bar in \(\bar{R}^2\) denotes adjusted R-squared, not a sample mean.

These formulas require \(SST>0\) and positive residual degrees of freedom. For an ordinary comparison, the models must use the same response and the same observations, and both must include an intercept. The predictor columns must also allow the fitted model to be estimated; in particular, adding a predictor that is an exact linear combination of existing predictors does not provide an additional estimable predictor.

The summaries are descriptive measures of fit to the sample used to fit the models. Their definitions do not require normally distributed errors. Regression inference may require additional assumptions, but neither R-squared nor adjusted R-squared is itself an inference procedure.

What the Penalty Does

Ordinary R-squared is the fraction of the sample’s total variation around its mean accounted for by the fitted regression, when the model includes an intercept. Adjusted R-squared compares the residual variation per residual degree of freedom with the total variation per total degree of freedom. Its adjustment makes a larger value harder to achieve as predictors are added.

For two models fitted to the same outcome and observations, the \(SST/(n-1)\) part of adjusted R-squared is common to both. Adding a predictor reduces adjusted R-squared if the residual sum of squares does not fall enough to offset the reduction in residual degrees of freedom. Consequently, it is possible for the residual sum of squares to fall, ordinary R-squared to rise, and adjusted R-squared to fall—all at once.

Key result: If a model with \(k\) predictors has ordinary R-squared \(R^2_{\text{old}}\), and one estimable predictor is added using the same observations, adjusted R-squared increases exactly when the rise in ordinary R-squared is greater than \((1-R^2_{\text{old}})/(n-k-1)\). If the rise is smaller, adjusted R-squared decreases.

Why Ordinary R-Squared Cannot Decrease

The reason is about the set of possible fitted values. When a predictor is added, the expanded model can reproduce every fit available to the smaller model by assigning the new predictor a coefficient of zero. It can also choose other coefficient values. Since ordinary least squares selects the coefficients that minimize \(SSE\), the expanded model’s minimum \(SSE\) cannot be larger than the smaller model’s minimum \(SSE\).

SSE_{\text{new}}\leq SSE_{\text{old}} \quad\Longrightarrow\quad 1-\frac{SSE_{\text{new}}}{SST}\geq 1-\frac{SSE_{\text{old}}}{SST} \quad\Longrightarrow\quad R^2_{\text{new}}\geq R^2_{\text{old}}.

This argument assumes the same response observations and the same \(SST\), as well as an expanded model that includes all the smaller model’s predictors and an intercept. It establishes a property of in-sample least-squares fitting, not a guarantee of better prediction beyond the data used to fit the model.

Worked Comparisons

Worked Example: Ordinary R-Squared Rises but Adjusted R-Squared Falls

Suppose a hypothetical model predicting weekly energy use from two predictors is fitted to \(n=30\) homes. Let \(SST=290\) energy-use units squared and \(SSE=116\) units squared. A model adding a third predictor has \(SSE=114\) units squared on the same homes. Does the small reduction in residual error raise adjusted R-squared?

For the original model, \(k=2\), so its residual degrees of freedom are \(30-2-1=27\). Its ordinary R-squared is

R^2_{\text{old}}=1-\frac{116}{290}=0.6000.

Since \(SST/(n-1)=290/29=10\), its adjusted R-squared is

\bar{R}^2_{\text{old}} =1-\frac{116/27}{290/29} =1-\frac{4.2963}{10} =0.5704.

For the expanded model, \(k=3\), so the residual degrees of freedom are \(30-3-1=26\). Its ordinary R-squared is

R^2_{\text{new}}=1-\frac{114}{290}=0.6069.

The adjusted value is

\bar{R}^2_{\text{new}} =1-\frac{114/26}{290/29} =1-\frac{4.3846}{10} =0.5615.

Ordinary R-squared rises by \(0.6069-0.6000=0.0069\), while adjusted R-squared falls from \(0.5704\) to \(0.5615\). The added predictor reduces residual error, but not by enough to offset the degree-of-freedom penalty. These numbers describe fit among the 30 homes; they do not establish that the predictor has no relationship with energy use.

The single-predictor threshold gives the same conclusion. Here the increase needed to raise adjusted R-squared is

\frac{1-R^2_{\text{old}}}{n-k-1} =\frac{1-0.6000}{27} =0.0148.

The observed increase, \(0.0069\), is smaller than \(0.0148\), so adjusted R-squared falls.

A Short Derivation of the Threshold

Write \(R^2_{\text{old}}\) for the value with \(k\) predictors and \(R^2_{\text{new}}\) for the value after one predictor is added. The total degrees of freedom \(n-1\) cancel when comparing adjusted values. Thus the new adjusted value is larger exactly when

\frac{1-R^2_{\text{new}}}{n-k-2} < \frac{1-R^2_{\text{old}}}{n-k-1}.

The denominators are positive under the comparison conditions. Multiplying by them, then rearranging, gives

(n-k-1)(1-R^2_{\text{new}}) < (n-k-2)(1-R^2_{\text{old}}) \quad\Longleftrightarrow\quad R^2_{\text{new}}-R^2_{\text{old}}> \frac{1-R^2_{\text{old}}}{n-k-1}.

So an added predictor must produce a larger rise in ordinary R-squared to raise adjusted R-squared. This is an algebraic comparison, not a test of a hypothesis about the predictor’s coefficient.

Worked Example: A Larger Improvement Clears the Penalty

Use the same hypothetical sample of 30 homes, the same \(SST=290\), and the same original two-predictor model with \(SSE=116\). Suppose instead that a proposed third predictor reduces the expanded model’s \(SSE\) to 108 units squared. Calculate both summaries and compare them.

The original values remain \(R^2_{\text{old}}=0.6000\) and \(\bar{R}^2_{\text{old}}=0.5704\). For the expanded model,

R^2_{\text{new}}=1-\frac{108}{290}=0.6276, \qquad \bar{R}^2_{\text{new}} =1-\frac{108/26}{290/29} =1-\frac{4.1538}{10} =0.5846.

Ordinary R-squared increases by \(0.6276-0.6000=0.0276\), which exceeds the threshold \(0.0148\). Adjusted R-squared also rises, from \(0.5704\) to \(0.5846\). This tells us that the added predictor’s improvement in in-sample fit is large enough to offset this particular adjustment. It does not show that the predictor’s contribution is statistically significant or practically important.

Worked Example: A Block of Predictors Can Carry a Larger Penalty

A hypothetical model for annual clinic supply costs uses \(n=50\) clinics. Its two-predictor version has \(SST=980\) cost units squared and \(SSE=490\) units squared. A proposed expanded version adds four predictors and has \(SSE=470\) units squared. Compare the summaries.

For the original model, the residual degrees of freedom are \(50-2-1=47\). Its ordinary R-squared is \(1-490/980=0.5000\), and its adjusted value is

\bar{R}^2_{\text{old}} =1-\frac{490/47}{980/49} =1-\frac{10.4255}{20} =0.4787.

For the expanded model, there are six predictors and \(50-6-1=43\) residual degrees of freedom. Its ordinary R-squared is

R^2_{\text{new}}=1-\frac{470}{980}=0.5204.

Its adjusted R-squared is

\bar{R}^2_{\text{new}} =1-\frac{470/43}{980/49} =1-\frac{10.9302}{20} =0.4535.

Ordinary R-squared rises by about \(0.0204\), but adjusted R-squared falls from \(0.4787\) to \(0.4535\). The expanded model reduces \(SSE\), yet uses four additional predictors and has fewer residual degrees of freedom. The disagreement signals that, according to this adjustment, the improvement is modest relative to the added complexity. It does not identify which of the four predictors matters.

What the Disagreement Does and Does Not Establish

When ordinary R-squared rises and adjusted R-squared falls, report both values with the comparison conditions made clear. A suitable interpretation is: “On the same observations, adding these predictors increased ordinary R-squared from 0.600 to 0.607, but decreased adjusted R-squared from 0.570 to 0.562.” This states the observed changes without turning a descriptive summary into a conclusion about a population or a mechanism.

Neither summary alone establishes that a model is useful for prediction on new cases. Both are calculated from the data used to fit the model. Neither establishes causation, and adjusted R-squared is not a p-value, a confidence interval, or a formal test. To assess coefficient evidence or compare specified model structures formally, use the appropriate regression inference method rather than treating a change in adjusted R-squared as a significance decision.

A fair comparison also matters. If models use different observations, different response variables, or different intercept conventions, their R-squared values may not be directly comparable by this argument. In addition, adjusted R-squared can be negative: its formula penalizes residual error relative to the total variation, and it is not constrained to be at least zero. A negative value does not mean “negative variance explained.”

Common Mistake

Wrong reasoning: “Ordinary R-squared increased, so the added predictor must be useful.” Ordinary R-squared cannot decrease when predictors are added to a least-squares model on the same data, even if the reduction in error is small.
Correction: Check adjusted R-squared to see whether the gain in fit clears its complexity penalty, then use the relevant inferential and practical criteria to answer questions about evidence and usefulness. A decrease in adjusted R-squared is a caution about the gain relative to complexity, not proof that the predictor has no effect.

Key takeaway: Ordinary R-squared measures in-sample fit and cannot fall when predictors are added to the same intercept model. Adjusted R-squared can fall because it penalizes the loss of residual degrees of freedom. Their disagreement describes a fit-versus-complexity tradeoff; it is not a hypothesis test or a guarantee about future prediction.

Check Your Understanding

Answer each question using the definitions and comparison conditions in this tutorial.

Why can ordinary R-squared not decrease when an estimable predictor is added to an ordinary least-squares model on the same observations?
A model’s ordinary R-squared rises after adding one predictor, but its adjusted R-squared falls. What does this say about the observed fit improvement relative to the adjustment?
For \(n=40\) and an old model with \(k=3\) predictors and \(R^2_{\text{old}}=0.50\), what rise in ordinary R-squared is needed for adjusted R-squared to increase after adding one predictor?
Does a higher adjusted R-squared prove that a predictor’s coefficient is statistically significant? Explain.
Name two conditions that should hold for a direct comparison of ordinary and adjusted R-squared between models.