From a Prediction Table to RMSE
A regression model produces a prediction for each observation, but a list of predictions is not enough to summarize performance. Root mean squared error (RMSE) reduces the differences between observed outcomes and predictions to one nonnegative number, expressed in the same units as the outcome. The calculation is short; getting the pairs, denominator, and units right is what makes the result trustworthy.
RMSE is useful when each evaluated observation is to count equally and larger errors should have a larger effect on the summary. It is a descriptive measure of the errors on the specified evaluation data. It does not, by itself, say why the model made an error or whether performance will be the same on a different set of observations. The calculation below is aligned with the SOA Exam PA (Predictive Analytics) syllabus.
The calculation assumes that each actual outcome is correctly paired with its prediction, all values use the same measurement scale, and each of the \(n\) evaluated observations receives equal weight. There is no requirement that the errors follow a particular probability distribution: RMSE can be calculated directly from observed prediction errors. If a row does not have both an actual value and a prediction, it is not a complete pair and should not be counted as an evaluated observation.
Why the Formula Uses These Steps
The formula has three steps because the signed errors alone can cancel. Squaring makes every contribution nonnegative, averaging gives each evaluated observation an equal share of the total squared error, and taking the square root returns the summary to the outcome’s original units.
To see why the result has those properties, start with the errors \(e_1,\ldots,e_n\). Their squared values are \(e_1^2,\ldots,e_n^2\), so their average is \(M = \frac{1}{n}\sum_{i=1}^{n} e_i^2\). Since every square is nonnegative, \(M\) is nonnegative and \(\sqrt{M}\) is defined. If all errors are zero, then \(M=0\) and RMSE is zero. Conversely, if RMSE is zero, then the sum of nonnegative squares is zero, which is possible only if every error is zero. Finally, if outcomes are measured in dollars, each error is in dollars, its square is in dollars squared, and the square root of the mean squared error is in dollars.
The denominator is \(n\), the number of evaluated pairs. RMSE describes the average squared prediction error for those pairs; it is not a sample variance estimate and does not use an \(n-1\) correction. Keep the squared-error average and the square-root step separate when checking work.
For each row, identify its actual outcome and the prediction for that same row.
Use \(e_i=y_i-\hat{y}_i\) consistently, and check the subtraction for each pair.
Square every error, add the squared errors, and divide by the number of complete evaluated pairs.
Take the nonnegative square root of the mean squared error, then state the result on the original outcome scale.
Worked Calculations by Hand
Worked Example: Claim Costs Recorded in Thousands of Dollars
A hypothetical insurer evaluates four predictions of claim cost. Both actual and predicted costs are recorded in thousands of dollars. Calculate RMSE and report it in thousands of dollars and dollars. The method is to form one signed error per row, square those errors, and divide by the four evaluated pairs. This avoids using a wrong unit conversion or dividing by anything other than the number of rows.
| Observation | Actual cost | Predicted cost | Error: actual − predicted | Squared error |
|---|---|---|---|---|
| 1 | 40 | 42 | −2 | 4 |
| 2 | 50 | 47 | 3 | 9 |
| 3 | 60 | 60 | 0 | 0 |
| 4 | 80 | 75 | 5 | 25 |
The squared errors sum to \(4+9+0+25=38\), in squared thousands of dollars. There are \(n=4\) complete pairs, so the mean squared error is \(38/4=9.5\). Therefore,
To convert to dollars, multiply by 1,000: \(3.082207\ldots \times 1{,}000 = \$3{,}082.207\ldots\), or \(\$3{,}082.21\) to the nearest cent. The check is that the reported RMSE is greater than zero because three rows have nonzero errors, and that its units are thousands of dollars before conversion. A common mistake is to report 9.5 as RMSE; 9.5 is the mean squared error, and it is the square root, not the mean squared error itself, that is in the original units.
Worked Example: Keeping the Denominator at the Number of Pairs
A hypothetical equipment-repair team compares predicted and actual repair costs in dollars for four jobs. Find RMSE to the nearest cent. The method is the same row-by-row calculation, with \(n=4\). The common mistake to avoid is treating the four errors as a sample for a variance estimate and dividing by \(n-1\).
| Job | Actual cost | Predicted cost | Error: actual − predicted | Squared error |
|---|---|---|---|---|
| A | $1,200 | $1,100 | $100 | $10,000 |
| B | $1,800 | $1,900 | −$100 | $10,000 |
| C | $2,500 | $2,200 | $300 | $90,000 |
| D | $3,500 | $3,900 | −$400 | $160,000 |
The sum of squared errors is \(10{,}000+10{,}000+90{,}000+160{,}000=270{,}000\) dollars squared. Divide by \(n=4\), giving \(270{,}000/4=67{,}500\) dollars squared. Thus,
The denominator check is decisive: four jobs were evaluated, so the denominator is 4. The incorrect \(n-1\) calculation would give \(\sqrt{270{,}000/3}=\sqrt{90{,}000}=\$300\), which is not the RMSE for these four prediction errors. The signed errors include both positive and negative values, but their squared contributions are all nonnegative.
Reproducing the Calculation in R
In R, store the actual outcomes and predictions in vectors in the same order. Subtract the prediction vector from the actual vector to obtain the errors, then square, average, and take the square root. A direct calculation uses the base R functions mean and sqrt; it does not require a package.
R applies subtraction and squaring element by element. The mean of the resulting squared errors divides their sum by the number of values, so the expression uses \(n\), not \(n-1\). The square root is applied after the mean, not to each error before averaging.
Worked Example: Verifying a Five-Row Calculation in R
A hypothetical analyst evaluates five predicted service costs, with both actual and predicted values measured in thousands of dollars. Calculate the RMSE by hand, then reproduce it in R. The method explicitly checks the row order before using the vector expression; the common mistake is pairing one prediction with a different observation.
| Row | Actual | Predicted | Actual − predicted | Squared error |
|---|---|---|---|---|
| 1 | 12 | 10 | 2 | 4 |
| 2 | 15 | 16 | −1 | 1 |
| 3 | 18 | 15 | 3 | 9 |
| 4 | 20 | 22 | −2 | 4 |
| 5 | 25 | 23 | 2 | 4 |
The squared errors sum to \(4+1+9+4+4=22\). There are five paired rows, so the mean squared error is \(22/5=4.4\), and the RMSE is \(\sqrt{4.4}=2.097617\ldots\) thousand dollars.
Enter the vectors in the table’s row order and calculate the result as follows. In R, the displayed output from the final expression is approximately 2.097618.
actual = c(12, 15, 18, 20, 25)
predicted = c(10, 16, 15, 22, 23)
errors = actual - predicted
squared_errors = errors^2
mean(squared_errors)
sqrt(mean(squared_errors))
The intermediate R values are errors equal to c(2, -1, 3, -2, 2), squared_errors equal to c(4, 1, 9, 4, 4), and mean(squared_errors) equal to 4.4. The result is therefore about 2.0976 thousand dollars, or about $2,097.62. The hand and R calculations agree. Reversing the subtraction would reverse every signed error, but would not change RMSE because \((-e_i)^2=e_i^2\). The sign convention still matters when examining the direction of errors; here it is stated explicitly so the intermediate errors can be checked.
Checks That Prevent Misreporting
Before reporting an RMSE, check the calculation in a fixed order. Confirm that each prediction is paired with the actual value from the same observation. Confirm that the number of complete pairs is the denominator. Confirm that the square root is taken after averaging the squared errors. Finally, state the measurement unit and any conversion used.
The most common calculation errors are easy to diagnose. A negative RMSE cannot be correct, because squares and the principal square root are nonnegative. If the result is in squared dollars rather than dollars, the square-root step has been omitted. If it is far larger or smaller than the errors themselves, check whether the outcome was recorded in dollars or thousands of dollars, and whether a conversion was applied twice or not at all.
A sign reversal alone does not alter RMSE, because the errors are squared. By contrast, misaligning actual and predicted values can change the size of the errors and therefore the RMSE. If some pairs are incomplete, first identify the rows for which both values are available and define the evaluation set; do not let a software setting silently determine which rows are included. The resulting value describes the included pairs, so the number of pairs should be checked and reported when the evaluation set matters.
Check Your Understanding
For each problem, calculate RMSE using the stated evaluation rows and units. Check your work against the final answer.
- A hypothetical model predicts weekly costs in thousands of dollars. The actual values are 8, 11, and 14; the predictions are 7, 13, and 14. What is RMSE in thousands of dollars? (Answer: \(\sqrt{5/3}\approx 1.2910\) thousand dollars.)
- For four hypothetical repairs, actual costs in dollars are 100, 120, 150, and 130, while predictions are 110, 115, 160, and 125. Calculate RMSE to the nearest cent. (Answer: \(\sqrt{62.5}\approx \$7.91\).)
- A model predicts the number of policies processed in a shift. The actual counts are 4, 6, 9, and 11; the predictions are 5, 5, 10, and 10. Calculate the sum of squared errors, the mean squared error, and RMSE. (Answer: sum of squared errors \(=4\), mean squared error \(=1\), RMSE \(=1\) policy.)
- In thousands of dollars, the five actual values are 10, 14, 18, 22, and 26, and the corresponding predictions are 8, 16, 18, 19, and 27. Calculate RMSE. State the denominator and the final unit. (Answer: \(\sqrt{18/5}=\sqrt{3.6}\approx 1.8974\) thousand dollars; denominator \(=5\).)
- A hypothetical evaluation table has four rows. Three complete pairs have actual costs of $500, $700, and $900, with corresponding predictions of $450, $800, and $850. The fourth row has an actual cost but no prediction. If only complete pairs are evaluated, calculate RMSE and state how many pairs are included. (Answer: \(\sqrt{5{,}000}\approx \$70.71\), using 3 complete pairs.)