R-Squared for Predictive Error Assessment

Predictive R-Squared Compares Errors With a Baseline

A prediction error is easiest to interpret when it is compared with a reasonable alternative. For regression, a simple alternative is to predict the same mean outcome for every case. Predictive R-squared summarizes how much the model’s total squared error changes relative to that baseline. Its value depends on which mean is used and on which observations are evaluated, so both choices must be stated.

Let $y_i$ be the observed outcome for evaluation case $i$, $\hat{y}_i$ the model prediction, and $b_i$ the prediction from a specified baseline, for $i=1,\ldots,n$. The calculations below use unweighted observations and squared error. They do not require a probability distribution for the outcomes; the reported value describes the chosen set of observed cases.

Definition: For a fixed evaluation set and baseline predictions $b_i$, define the model sum of squared errors as $\operatorname{SSE}_{\text{model}}=\sum_{i=1}^{n}(y_i-\hat{y}_i)^2$, and the baseline sum of squared errors as $\operatorname{SSE}_{\text{base}}=\sum_{i=1}^{n}(y_i-b_i)^2$. If $\operatorname{SSE}_{\text{base}}>0$, predictive R-squared is $\displaystyle R^2_{\text{pred}}=1-\frac{\operatorname{SSE}_{\text{model}}}{\operatorname{SSE}_{\text{base}}}$.

A common choice is the evaluation-set mean, $\bar{y}_{\text{eval}}=\frac{1}{n}\sum_i y_i$, used as the constant baseline prediction for every evaluation case. In that case, the baseline sum of squares is $\operatorname{SST}=\sum_i(y_i-\bar{y}_{\text{eval}})^2$, and the formula becomes $1-\operatorname{SSE}_{\text{model}}/\operatorname{SST}$. This is a useful comparison with the best constant prediction for those observed outcomes, but that mean is not known when predictions are made in advance.

For a prospective or validation comparison, a more operational baseline may be the mean outcome in the training data, used as a constant prediction for each validation case. This avoids using validation outcomes to set the baseline. The two baselines answer different questions and can produce different R-squared values. Whichever is chosen, calculate both sums of squares on the same evaluation cases and name the baseline explicitly.

How to Read the Value

When the baseline sum of squared errors is positive, $R^2_{\text{pred}}=1-\operatorname{SSE}_{\text{model}}/\operatorname{SSE}_{\text{base}}$ compares the model’s total squared error with the baseline’s. A positive value means the model has less squared error; zero means the two totals are equal; and a negative value means the model has more squared error than the baseline. For instance, $R^2_{\text{pred}}=0.60$ means the model’s total squared error is 60% lower than the baseline’s on this evaluation set. It does not mean that 60% of individual predictions are correct.

Because a sum of squared errors cannot be negative, the statistic cannot exceed 1 when the baseline sum of squares is positive. There is no general lower limit: a model can have arbitrarily larger squared error than its baseline. If the baseline sum of squares is zero, the ratio is undefined. In particular, if the evaluation outcomes are all identical and the baseline is their mean, $\operatorname{SST}=0$, so this R-squared calculation is undefined even if the model also predicts that constant perfectly.

Worked Example: Comparing Validation Predictions With the Evaluation Mean

A hypothetical pricing team evaluates predictions of annual service costs, recorded in thousands of dollars, on four validation cases. The observed costs are $10, 12, 16,$ and $22$. The model predicts $11, 13, 15,$ and $19$. Calculate predictive R-squared using the validation-set mean as the baseline, and interpret the result.

Method. Find the mean baseline, calculate each set of squared errors on these same four cases, and apply the predictive R-squared formula.

The validation mean is $(10+12+16+22)/4=60/4=15$. Thus, the baseline predicts $15$ for each case. Its errors, observed minus predicted, are $-5,-3,1,7$, and its squared errors are $25,9,1,49$. Therefore, $\operatorname{SSE}_{\text{base}}=25+9+1+49=84$.

For the model, the errors are $10-11=-1$, $12-13=-1$, $16-15=1$, and $22-19=3$. The squared errors are $1,1,1,9$, so $\operatorname{SSE}_{\text{model}}=1+1+1+9=12$.

Substitution gives

R^2_{\text{pred}}=1-\frac{12}{84} =1-\frac{1}{7} =\frac{6}{7} \approx 0.8571.

Rounded to four decimal places, the model’s total squared error is 85.71% lower than that of the validation-mean baseline. As a check, $12/84=0.1429$ to four decimals, and $1-0.1429=0.8571$. The comparison is specific to these four cases and this baseline; it is not a claim that 85.71% of predictions are accurate.

Why the Mean Is the Squared-Error Baseline

The evaluation-set mean is not an arbitrary constant prediction: among all constants, it minimizes squared error on that set. This explains why the evaluation-mean baseline is often used to construct the familiar R-squared ratio.

Write the observed outcomes as $y_1,\ldots,y_n$, with mean $\bar{y}$, and consider predicting a single constant $c$ for every case. Since $\sum_i(y_i-\bar{y})=0$, express each deviation from $c$ as $y_i-c=(y_i-\bar{y})+(\bar{y}-c)$. Squaring and summing gives

\begin{aligned} \sum_{i=1}^{n}(y_i-c)^2 &=\sum_{i=1}^{n}\big[(y_i-\bar{y})+(\bar{y}-c)\big]^2\\ &=\sum_{i=1}^{n}(y_i-\bar{y})^2 +2(\bar{y}-c)\sum_{i=1}^{n}(y_i-\bar{y}) +n(\bar{y}-c)^2\\ &=\sum_{i=1}^{n}(y_i-\bar{y})^2+n(\bar{y}-c)^2. \end{aligned}

The middle term is zero because the deviations from the mean sum to zero. The added term $n(\bar{y}-c)^2$ is nonnegative, so the total squared error is minimized at $c=\bar{y}$. This derivation concerns constant predictions on the observed set. It does not make the evaluation mean available in advance for a future prediction; for that use, a mean estimated from training data is a different, explicitly chosen baseline.

Worked Comparisons and the In-Sample Case

Worked Example: A Model That Loses to the Training-Mean Baseline

A hypothetical analyst uses a training-set mean cost of $110$ thousand dollars as the validation baseline. On three validation cases, observed costs are $80,100,$ and $120$ thousand dollars. The model predicts $50,100,$ and $150$. Calculate predictive R-squared using the training-mean baseline, and compare it with the evaluation-mean baseline.

Method. First use the specified training mean for all three baseline predictions. Then, separately, recalculate using the validation mean to show why the baseline must be identified.

Against the training mean of $110$, the errors are $80-110=-30$, $100-110=-10$, and $120-110=10$. The squared errors are $900,100,100$, so $\operatorname{SSE}_{\text{base}}=1100$. The model errors are $80-50=30$, $100-100=0$, and $120-150=-30$. Their squares sum to $\operatorname{SSE}_{\text{model}}=900+0+900=1800$.

Therefore,

R^2_{\text{pred, train mean}} =1-\frac{1800}{1100} =1-\frac{18}{11} =-\frac{7}{11} \approx -0.6364.

The model has greater squared error than the training-mean baseline. As a check, its error exceeds the baseline error by $1800-1100=700$, and $700/1100=0.6364$ to four decimals.

The validation mean is $(80+100+120)/3=100$. With that baseline, the errors are $-20,0,20$, giving $\operatorname{SSE}_{\text{base}}=400+0+400=800$. The corresponding value is $1-1800/800=1-2.25=-1.25$. Both values indicate worse performance than their stated baselines, but they differ because the baselines differ. Do not substitute one denominator for the other without changing the interpretation.

Common mistake avoided: reporting the negative value as a calculation failure. A negative predictive R-squared is valid when model squared error exceeds baseline squared error.

There is a special in-sample result for ordinary least squares (OLS). If an OLS model with an intercept is fitted and evaluated on the same observations, then, provided $\operatorname{SST}>0$, its in-sample R-squared is between zero and one. The conditions matter: this statement does not generally apply to a validation set, a model without an intercept, or a data set with $\operatorname{SST}=0$.

To justify the result, let $\bar{y}$ be the mean of the fitted observations and let $e_i=y_i-\hat{y}_i$ be the OLS residuals. The fitted values from an OLS regression with an intercept satisfy $\sum_i e_i=0$ and $\sum_i e_i(\hat{y}_i-\bar{y})=0$. Since $y_i-\bar{y}=(\hat{y}_i-\bar{y})+e_i$, squaring and summing yields the decomposition below.

\operatorname{SST} =\sum_i(y_i-\bar{y})^2 =\sum_i(\hat{y}_i-\bar{y})^2+\sum_i e_i^2,

The cross-product vanishes by the OLS orthogonality condition. Both terms on the right are nonnegative, so $0\leq\operatorname{SSE}_{\text{model}}\leq\operatorname{SST}$. When $\operatorname{SST}>0$, it follows that $0\leq1-\operatorname{SSE}_{\text{model}}/\operatorname{SST}\leq1$. If all observed outcomes are equal, $\operatorname{SST}=0$, and this ratio is undefined rather than between zero and one.

Worked Example: Checking In-Sample OLS R-Squared

A small in-sample OLS fit with an intercept uses predictor values $x=(1,2,3,4)$ and observed outcomes $y=(2,5,5,8)$. Verify the fitted values, calculate in-sample R-squared, and check the sum-of-squares decomposition.

Method. Calculate the OLS slope and intercept, then compare the residual sum of squares with the total sum of squares around the observed mean.

The predictor mean is $\bar{x}=(1+2+3+4)/4=2.5$, and the outcome mean is $\bar{y}=(2+5+5+8)/4=5$. The cross-product sum is $(1-2.5)(2-5)+(2-2.5)(5-5)+(3-2.5)(5-5)+(4-2.5)(8-5)=4.5+0+0+4.5=9$. The predictor sum of squares is $(1-2.5)^2+(2-2.5)^2+(3-2.5)^2+(4-2.5)^2=2.25+0.25+0.25+2.25=5$. Thus the slope is $9/5=1.8$, and the intercept is $5-(1.8)(2.5)=0.5$.

The fitted values are $0.5+1.8(1)=2.3$, $0.5+1.8(2)=4.1$, $0.5+1.8(3)=5.9$, and $0.5+1.8(4)=7.7$. The residuals are $-0.3,0.9,-0.9,0.3$, so $\operatorname{SSE}_{\text{model}}=0.09+0.81+0.81+0.09=1.8$. The total sum of squares is $\operatorname{SST}=(2-5)^2+(5-5)^2+(5-5)^2+(8-5)^2=9+0+0+9=18$.

Consequently, $R^2=1-1.8/18=1-0.1=0.9$. The fitted-value sum of squares is $(2.3-5)^2+(4.1-5)^2+(5.9-5)^2+(7.7-5)^2=7.29+0.81+0.81+7.29=16.2$. The decomposition checks: $16.2+1.8=18$, and $1-1.8/18=16.2/18=0.9$.

Why R-Squared Does Not Replace RMSE

Predictive R-squared is dimensionless: it expresses a relative change in squared error, not the typical size of an error in the outcome’s units. RMSE, defined as the square root of mean squared error, supplies that scale information. A model can have a high predictive R-squared while still making errors that are materially large for the decision being made; conversely, the same RMSE can be judged differently in outcomes measured in dollars or thousands of dollars.

Changing the outcome’s units illustrates the distinction. If every observed, model-predicted, and baseline-predicted value is multiplied by the same nonzero conversion factor, both sums of squares are multiplied by the factor squared, so their ratio and predictive R-squared are unchanged. RMSE, by contrast, is multiplied by the absolute value of the conversion factor. Interpret R-squared alongside a scale-based error metric and the context of the prediction task, rather than as a stand-alone measure of practical accuracy.

Worked Example: Same Relative Score, Different Error Scale

A hypothetical model predicts annual repair amounts of $100,200,$ and $300$ dollars with predictions $150,220,$ and $280$ dollars. Use the evaluation mean as the baseline. Calculate predictive R-squared and RMSE, then express all amounts in hundreds of dollars and check what changes.

Method. Calculate model and baseline squared errors in dollars first; then repeat the sums after dividing every outcome and prediction by $100$.

The evaluation mean is $(100+200+300)/3=200$. The baseline squared errors are $(100-200)^2=10000$, $(200-200)^2=0$, and $(300-200)^2=10000$, so $\operatorname{SSE}_{\text{base}}=20000$. The model errors are $-50,-20,20$, with squares $2500,400,400$, so $\operatorname{SSE}_{\text{model}}=3300$. Thus, $R^2_{\text{pred}}=1-3300/20000=1-0.165=0.835$.

The model RMSE is $\sqrt{3300/3}=\sqrt{1100}\approx \$33.17$, rounded to the nearest cent. In hundreds of dollars, the outcomes are $1,2,3$, the baseline mean is $2$, and the predictions are $1.5,2.2,2.8$. The baseline sum of squares is $1+0+1=2$, and the model sum is $0.25+0.04+0.04=0.33$. Hence $1-0.33/2=0.835$ again. RMSE is $\sqrt{0.33/3}=\sqrt{0.11}\approx0.3317$ hundreds of dollars, which is approximately $\$33.17$. The relative score is unchanged; the RMSE number changes with the unit.

Common mistake avoided: treating R-squared as an error measured in dollars or as a substitute for RMSE. It reports relative squared-error performance, while RMSE reports error scale.

Common Calculation Traps

Forgetting the leading 1 minus. In the first worked example, $84/12=7$ is the inverse error ratio, not predictive R-squared. The correct calculation is $1-12/84=0.8571$.
Changing the baseline without saying so. A training mean and an evaluation mean are different baselines. Calculate and label the relevant baseline sum of squares.
Assuming validation R-squared cannot be negative. The in-sample OLS result has specific conditions; an out-of-sample model can perform worse than its baseline.
Using the score as an absolute error measure. A relative reduction does not reveal the error’s size in outcome units, so report a scale-based metric such as RMSE when that information matters.
Dividing by zero. If the baseline predictions match every observed value, the baseline sum of squares is zero and predictive R-squared is undefined.

Key takeaway: Predictive R-squared is $1-\operatorname{SSE}_{\text{model}}/\operatorname{SSE}_{\text{base}}$, with both sums calculated on the same evaluation cases against an explicitly named baseline. A positive value indicates lower squared error than that baseline; a negative value indicates higher error. It does not replace a metric that reports error in the outcome’s units.

Check Your Understanding

For each problem, identify the stated baseline before calculating. Round predictive R-squared to four decimal places when needed.

A validation set has outcomes $5,8,11$, and the model predicts $6,8,10$. Using the validation mean as baseline, calculate both sums of squared errors and predictive R-squared. (Answer: baseline SSE $=18$, model SSE $=2$, predictive R-squared $=0.8889$.)
A training-set mean of $40$ is used as the baseline for validation outcomes $30,50$. A model predicts $20,60$. Calculate predictive R-squared and determine whether the model improves on the baseline. (Answer: baseline SSE $=200$, model SSE $=200$, predictive R-squared $=0$; no improvement in total squared error.)
For outcomes $4,6,10,12$, a model predicts $5,7,9,11$. Using the evaluation mean as baseline, calculate the mean, both sums of squared errors, and predictive R-squared. (Answer: mean $=8$, baseline SSE $=40$, model SSE $=4$, predictive R-squared $=0.9000$.)
All three validation outcomes equal $7$, and the baseline predicts the validation mean for each case. Is predictive R-squared defined? Explain what happens to its denominator. (Answer: No; the baseline SSE is $0$, so the ratio is undefined.)
A model has validation SSE $75$, while the chosen mean baseline has SSE $60$. Calculate predictive R-squared and state the percentage change in squared error relative to the baseline. (Answer: predictive R-squared $=1-75/60=-0.25$; model squared error is 25% higher than the baseline’s.)

What You'll Learn