Root Mean Squared Error and Large Misses

Why Root Mean Squared Error Matters

A prediction can be close to the observed outcome for most cases and still miss badly on one case. A useful prediction metric should summarize the errors while making clear how those larger misses affect the summary. Root mean squared error, usually abbreviated RMSE, does this by squaring each prediction error, averaging the squared errors, and taking the square root.

RMSE is used to summarize predictions on a set of cases, such as a validation set or test set. Those cases should be ones on which the predictions can be evaluated fairly: each observed outcome is paired with a prediction, and the outcomes were not used to fit or tune the predictions being assessed. The training and validation material explains why that separation matters. Once the evaluation cases and predictions are fixed, RMSE is a descriptive summary of their prediction errors.

Definition: For \(n\) cases with observed outcomes \(y_i\) and corresponding predictions \(\hat{y}_i\), define the prediction error as \(e_i=y_i-\hat{y}_i\). The mean squared error is the average of the squared errors, and the root mean squared error is its nonnegative square root:

\operatorname{MSE}=\frac{1}{n}\sum_{i=1}^{n} e_i^2 \qquad\text{and}\qquad \operatorname{RMSE}=\sqrt{\operatorname{MSE}} =\sqrt{\frac{1}{n}\sum_{i=1}^{n}(y_i-\hat{y}_i)^2}.

The denominator is \(n\), the number of evaluated cases. RMSE is a prediction metric, not a sample variance estimate, so the denominator is not \(n-1\). The calculation requires paired numerical outcomes and predictions on the same scale, with finite errors. It does not require a normal distribution or a linear model. The sign convention \(e_i=y_i-\hat{y}_i\) makes a positive error an underprediction and a negative error an overprediction, but squaring removes the sign when calculating RMSE.

Calculating RMSE From Prediction Errors

The calculation has three stages: square each error, take the mean of those squares, and take the square root. Keeping the stages visible helps avoid confusing RMSE with either the average squared error or the average error itself. The square root also returns the metric to the original outcome units.

Find each error.
For every evaluated case, subtract its prediction from its observed outcome: \(e_i=y_i-\hat{y}_i\).

Square and average.
Calculate \(e_i^2\) for each case, add these values, and divide by \(n\). The result is the mean squared error.

Take the square root.
The nonnegative square root of the mean squared error is RMSE, in the same units as the outcomes.

Worked Example: Calculating RMSE for Four Predictions

Suppose, hypothetically, a model predicts the delivery time for four packages. The observed and predicted times, in hours, are shown below. Find the RMSE for these four predictions.

Package	Observed time \(y_i\) (hours)	Predicted time \(\hat{y}_i\) (hours)	Error \(e_i=y_i-\hat{y}_i\) (hours)	Squared error \(e_i^2\) (hours squared)
A	5	4	1	1
B	7	8	-1	1
C	6	3	3	9
D	4	5	-1	1

The errors are \(1,-1,3,-1\) hours. Their squared errors sum to \(1+1+9+1=12\) hours squared. There are \(n=4\) evaluated packages, so

\operatorname{MSE}=\frac{12}{4}=3\text{ hours}^2, \qquad \operatorname{RMSE}=\sqrt{3}\text{ hours}\approx1.732\text{ hours}.

Thus, the RMSE of these four predictions is approximately \(1.732\) hours. This describes the errors on these particular cases; it does not mean every package prediction is wrong by \(1.732\) hours. The square root gives the summary the same units as delivery time, rather than squared hours.

Why Squaring Makes Large Misses Count More

Squaring does more than remove the signs of the errors. It makes the contribution of an error grow quadratically with its magnitude. For example, an error of magnitude 2 contributes \(2^2=4\) to the sum of squared errors, while an error of magnitude 1 contributes \(1^2=1\). The error twice as large therefore contributes four times as much to the sum of squares.

The effect can be stated exactly. Suppose there are \(n\) errors and one error has magnitude \(a\), where \(a>0\). Replace that error by one with magnitude \(ka\), where \(k>1\), while leaving all other errors and the number of cases unchanged. The change in the sum of squared errors is

$$ (ka)^2-a^2=(k^2-1)a^2>0. $$

Because the number of cases \(n\) is unchanged and positive, the mean squared error increases by \(\frac{(k^2-1)a^2}{n}>0\). The square-root function is strictly increasing for positive inputs, so RMSE also increases. The condition \(a>0\) matters: if the original error is zero, multiplying its magnitude by \(k\) leaves it at zero, and there is no increase. This is why the claim about an increase requires a nonzero original error.

Worked Example: Replacing One Error With a Larger Miss

Consider two hypothetical sets of four prediction errors, measured in minutes. The first set is \(1,1,1,1\). In the second set, only the first error changes, from magnitude \(1\) minute to magnitude \(2\) minutes; the errors are \(2,1,1,1\). Calculate both RMSE values and the change.

For the first set, the squared errors are \(1,1,1,1\), so the sum is 4 minutes squared. Therefore,

\operatorname{MSE}_1=\frac{4}{4}=1\text{ minute}^2, \qquad \operatorname{RMSE}_1=\sqrt{1}=1\text{ minute}.

For the second set, the squared errors are \(4,1,1,1\), so their sum is 7 minutes squared. Therefore,

\operatorname{MSE}_2=\frac{7}{4}=1.75\text{ minutes}^2, \qquad \operatorname{RMSE}_2=\sqrt{1.75}\approx1.323\text{ minutes}.

The error magnitude changed from \(a=1\) to \(ka=2\), with \(k=2\), and \(a>0\). Its squared contribution rose from \(1\) to \(4\), an increase of \(3\). Dividing that increase by the four cases raises the MSE by \(3/4=0.75\) minutes squared, from \(1\) to \(1.75\). RMSE consequently rises from \(1\) to about \(1.323\) minutes. The result illustrates the influence of a larger miss without suggesting that every error changed.

Units and Interpretation

Each error has the same units as the outcome. Squaring an error squares its units, so the MSE has squared outcome units. Taking the square root restores the outcome units. This is useful when interpreting a metric: RMSE for an outcome measured in kilograms is in kilograms, and RMSE for a time outcome is in units of time.

Worked Example: RMSE After Converting Units

Suppose, hypothetically, a scale model predicts the mass of three objects. In kilograms, the observed-minus-predicted errors are \(0.2,-0.1,\) and \(0.3\) kg. Calculate RMSE in kilograms, then express the same value in grams.

Square the errors and add them: \(0.2^2+(-0.1)^2+0.3^2=0.04+0.01+0.09=0.14\) kilograms squared. Divide by the three cases and take the square root:

\operatorname{RMSE} =\sqrt{\frac{0.14}{3}}\text{ kg} \approx0.2160\text{ kg}.

Since \(1\) kg is \(1000\) g, this is approximately \(216.0\) g. Directly converting the errors gives \(200,-100,\) and \(300\) g; squaring and averaging those converted errors gives \(\sqrt{(40{,}000+10{,}000+90{,}000)/3}\approx216.0\) g. Both calculations agree. The unit conversion changes the numerical value by a factor of \(1000\), not the underlying prediction errors.

What RMSE Does—and Does Not—Summarize

RMSE is nonnegative, and it equals zero exactly when every evaluated prediction matches its observed outcome. Otherwise, it is positive. Its squaring step makes large-magnitude errors especially influential, so a few large misses can raise RMSE noticeably even when many other errors are small. This sensitivity can be useful when large errors are particularly important to report.

RMSE is a single summary, not a description of every individual error. It does not identify which cases had large errors, indicate whether a particular prediction was too high or too low, or establish how well predictions will perform on new cases by itself. A useful assessment may report RMSE alongside the evaluation design and, when relevant, a display of the individual errors. Comparisons between models are most informative when they are evaluated on the same cases and on the same outcome scale.

Common Mistakes

Stopping at the mean squared error. The average of squared errors is in squared outcome units. RMSE requires one more step: take its nonnegative square root.
Dividing by \(n-1\). RMSE averages the squared prediction errors over the evaluated cases using \(n\), not the degrees-of-freedom adjustment used for a sample variance.
Forgetting to square negative errors correctly. For example, \((-3)^2=9\), not \(-9\). A negative error still contributes a nonnegative squared amount.
Claiming that every increase in an error magnitude must increase RMSE. Replacing a zero error by a multiple of zero does not change it. For the strict increase argument, the original magnitude must be positive and the multiplying factor must exceed 1.
Interpreting RMSE as the error on every case. RMSE summarizes a set of errors; it is not a guarantee that each prediction misses by that amount.

Key takeaway: RMSE is the square root of the average squared prediction error. It is expressed in the outcome’s units, and squaring gives larger errors disproportionate influence: multiplying a positive error magnitude by \(k\) multiplies its squared contribution by \(k^2\).

Check Your Understanding

Use the definitions and reasoning above to answer these questions.

For errors \(2,-2,\) and \(1\) centimeters, calculate the mean squared error and RMSE, showing the units at each stage.
Why is the denominator \(n\), rather than \(n-1\), in the RMSE calculation?
If one error changes from magnitude \(3\) to magnitude \(6\), by what factor does its squared contribution change?
Suppose an error of magnitude \(a\) is replaced by one of magnitude \(ka\), where \(k>1\). What additional condition on \(a\) ensures that the sum of squared errors strictly increases?
In a sentence, interpret an RMSE of \(4.2\) minutes for predictions of travel time. What should the sentence avoid claiming?