R-Squared as a Fraction of Variation Explained

Why Measure the Variation Accounted for by a Line?

A least-squares regression line gives a predicted response for each observed predictor value. Residuals describe the differences between those predictions and the observed responses. But how much does using the line improve on a simple baseline: predicting the same response value, the sample mean, for every observation? R-squared answers this by comparing the variation in the fitted values with the total variation in the observed responses.

R-squared is a summary of the fit of a regression line to a particular sample. Its value is a proportion, so it has no units even when the response is measured in units such as minutes, grams, or dollars. To calculate and interpret it, first distinguish the total variation in the responses from the variation left in the residuals.

The sums of squares use squared deviations, so their units are the square of the response units. For example, if \(y\) is measured in minutes, each sum of squares is measured in minutes squared. Their ratio cancels those units. The value of \(R^2\) is commonly reported either as a decimal or as a percentage.

Why the Two Formulas Agree

The earlier material on decomposing variation in simple linear regression establishes the key identity \(SST=SSR+SSE\) for least-squares regression with an intercept. The identity explains why R-squared can be calculated either from the regression sum of squares or from the residual sum of squares. Here is the essential justification.

Write the fitted line as \(\hat{y}_i=b_0+b_1x_i\), where \(b_0\) is its intercept and \(b_1\) its slope. Because the line is fitted by least squares with an intercept, its residuals satisfy \(\sum e_i=0\) and \(\sum (x_i-\bar{x})e_i=0\). The first follows from the intercept choice: the fitted values average to \(\bar{y}\), so the residuals sum to zero. The second is the least-squares slope condition. Also, \(\hat{y}_i-\bar{y}=b_1(x_i-\bar{x})\). Therefore,

For every observation, \(y_i-\bar{y}=(\hat{y}_i-\bar{y})+e_i\). Squaring and summing gives

Dividing by \(SST\), when it is positive, yields \(1=SSR/SST+SSE/SST\), and hence \(SSR/SST=1-SSE/SST\). Both sums of squares in the ratio are nonnegative. Thus, for this regression, \(0\leq R^2\leq 1\). This result relies on fitting a least-squares line with an intercept; formulas used for regressions without an intercept need not have these same properties.

Calculating R-Squared From a Fitted Line

A reliable calculation starts by finding the sample mean of the observed responses. Then calculate the deviations of the observations and fitted values from that mean. Squared observed-response deviations add to \(SST\); squared fitted-value deviations add to \(SSR\). Their ratio is \(R^2\). Alternatively, square each residual to find \(SSE\), then use \(1-SSE/SST\). The two calculations should agree, apart from small differences caused by rounding.

Calculating R-Squared From Residuals

When the residuals are available, \(1-SSE/SST\) can be the quicker calculation. It compares the squared prediction errors with the total response variation around the mean. If \(SSE\) is small relative to \(SST\), the line accounts for a larger fraction of the sample variation. If \(SSE=SST\), then \(R^2=0\); the fitted values do not reduce the sum of squared deviations from the mean. With a least-squares fit and an intercept, \(SSE\) cannot exceed \(SST\).

Interpreting the Proportion Carefully

The phrase “fraction of variation explained” is a concise way to describe \(R^2\), but it should be tied to its precise calculation: the fraction of the observed responses’ total squared deviations from their sample mean that is represented by the fitted values’ deviations from that mean. In context, it is often expressed as a percentage. For example, an \(R^2\) of 0.72 means the fitted line accounts for 72% of the sample response variation around the response mean; the remaining 28% is in the residual sum of squares relative to the total.

This percentage is not the percentage of observations predicted exactly, the percentage of predictions that are “correct,” or the typical size of a prediction error. It also does not show that the predictor caused the response to change. R-squared is one numerical summary of a fitted line, not a complete assessment of whether the line is appropriate. A residual standard error gives information about typical residual size in response units, while residual plots can reveal patterns that a single proportion does not display.

The calculation requires \(SST>0\). If every observed response has exactly the same value, then every deviation from \(\bar{y}\) is zero, so \(SST=0\). The ratio \(SSR/SST\) and the expression \(1-SSE/SST\) are then undefined. There is no observed response variation for the line to account for as a fraction.

Common Mistake

Wrong reasoning: “An \(R^2\) of 0.90 means the line predicts 90% of the individual responses correctly, or that sunlight causes 90% of plant height.” Neither statement follows from the calculation. R-squared compares sums of squared deviations; it is not a count of correct predictions and does not identify a cause.

Correction: State what was measured and what the denominator represents. For example: “In this sample, the fitted line accounts for 90% of the variation in observed plant heights around their sample mean.” Consider residuals and the context before drawing broader conclusions.