Why Root Mean Squared Error Matters
A prediction error is the difference between an observed outcome and its prediction. A model can make several small errors and one much larger miss; a single summary should reflect the whole set of errors while remaining interpretable on the scale of the outcome. Root mean squared error, usually abbreviated RMSE, does this by squaring the errors, averaging them, and then taking the square root.
RMSE is useful when evaluating predictions of a numerical outcome, such as a hypothetical insurer’s predicted claim cost or a service provider’s forecast of monthly demand. The square root returns the summary to the outcome’s units, while squaring means that larger errors contribute disproportionately. This tutorial is aligned with the SOA Exam PA (Predictive Analytics) syllabus.
The definition applies to numerical outcomes and predictions matched case by case on the same evaluation set. The outcome and its predictions must use the same scale and units. For example, an observed amount in dollars cannot be directly compared with a prediction in thousands of dollars without first converting one of them. RMSE is not defined by this formula for an empty set of cases.
The errors may be positive or negative, depending on whether a prediction is below or above its observation. Squaring removes the sign, so positive and negative errors do not cancel. RMSE is a nonnegative summary of error size; it does not say whether predictions tend to be too high or too low.
How to Calculate and Interpret RMSE
The calculation has three essential operations, in order: find each error, square each error and average the squares, then take the square root. Keeping the operations in this order matters. The final value is expressed in the same units as the observed outcome because taking the square root reverses the squared units.
Subtract the prediction from its corresponding observed outcome: \(e_i=y_i-\hat{y}_i\). Retain the sign at this stage so the error is calculated consistently.
Square every error, add the squared values, and divide by the number of cases \(n\). This intermediate average is MSE, in squared outcome units.
Take the nonnegative square root of MSE. State the result in the original outcome’s units and in the context of the evaluation cases.
Worked Example: Predicting Hypothetical Repair Costs
A hypothetical repair program evaluates predictions for four jobs. Costs and predictions are in dollars. Calculate RMSE and interpret it. Name the method and identify a calculation mistake to avoid.
| Job | Observed cost \(y_i\) | Predicted cost \(\hat{y}_i\) |
|---|---|---|
| 1 | $12 | $10 |
| 2 | $18 | $20 |
| 3 | $25 | $21 |
| 4 | $35 | $39 |
Method. Apply the RMSE formula to the four matched observed-predicted pairs. The signed errors are \(12-10=2\), \(18-20=-2\), \(25-21=4\), and \(35-39=-4\) dollars. Squaring gives \(2^2=4\), \((-2)^2=4\), \(4^2=16\), and \((-4)^2=16\) dollars squared.
The sum of squared errors is \(4+4+16+16=40\). There are \(n=4\) jobs, so \(\operatorname{MSE}=40/4=10\) dollars squared. Therefore, \(\operatorname{RMSE}=\sqrt{10}=3.162277\ldots\) dollars, or $3.16 when rounded to the nearest cent.
The result summarizes the prediction errors for these four jobs on a dollar scale. It does not mean that every prediction is exactly $3.16 away from its observation. Common mistake avoided: adding signed errors would give \(2-2+4-4=0\), but that is not a measure of error size. RMSE squares each error before averaging, so positive and negative errors cannot cancel.
Why Squaring Changes the Summary
The square root in RMSE is not an extra adjustment applied to the average error. It is the final step after averaging squared errors. The reason this produces a measure in the original units follows directly from the units and the definition.
Suppose the outcome is measured in units \(U\), such as dollars or service days. Each error is then measured in \(U\), and its square is measured in \(U^2\). An average of squared errors still has units \(U^2\). Taking the square root gives units \(\sqrt{U^2}=U\), so RMSE can be interpreted against the original outcome scale.
Squaring also explains why a large error can matter more than several smaller ones. An error with magnitude \(6\) contributes \(6^2=36\) to the sum of squared errors, whereas an error with magnitude \(2\) contributes \(2^2=4\). The first contributes nine times as much to the squared-error sum, because \(36/4=9\). The final square root does not undo that relative contribution.
This also shows why RMSE is not generally the average absolute error. The average absolute error first takes each error’s absolute value and then averages those values. RMSE instead squares, averages, and takes a square root. The two summaries answer related but different questions about the same prediction errors.
Worked Example: Comparing Two Forecasts of Service Days
A hypothetical service center compares two models’ forecasts of service days for the same four locations. Their signed errors, defined as observed minus predicted days, are shown below. Calculate each RMSE, compare the results, and explain how the error pattern affects the comparison.
| Location | Model A error (days) | Model B error (days) |
|---|---|---|
| 1 | 1 | 2 |
| 2 | 1 | 2 |
| 3 | 1 | 2 |
| 4 | 5 | 2 |
Method. Square each model’s errors separately, average within the model, and take the square root. For Model A, the squared errors are \(1,1,1,25\), with sum \(28\). Thus, \(\operatorname{MSE}_A=28/4=7\) days squared and \(\operatorname{RMSE}_A=\sqrt{7}=2.645751\ldots\), or 2.65 days to two decimal places.
For Model B, the squared errors are \(4,4,4,4\), with sum \(16\). Thus, \(\operatorname{MSE}_B=16/4=4\) days squared and \(\operatorname{RMSE}_B=\sqrt{4}=2\) days. Model B has the lower RMSE, by \(2.645751\ldots-2=0.645751\ldots\), or about 0.65 day.
Model A has three errors of only one day, but its five-day error has a squared contribution of \(25\). Model B’s four errors are each two days. RMSE gives more influence to the single large miss, so it favors Model B for this evaluation set. Common mistake avoided: comparing the number of small errors alone would ignore the squared contribution of Model A’s five-day miss.
Use the Units and Evaluation Set Carefully
An RMSE value only has meaning when its units and the cases being evaluated are clear. “RMSE = 2.4” is incomplete if the outcome’s units are not stated. An RMSE of 2.4 dollars is very different from an RMSE of 2.4 thousand dollars. Similarly, the value alone does not establish whether the predictions are adequate for a particular decision; that judgment depends on the scale and purpose of the outcome.
When comparing models, calculate RMSE on the same cases and use the same outcome scale. A lower RMSE means a lower average squared error on that set of cases. If the compared values come from different case sets or different outcome units, the difference may not be a fair comparison. RMSE is also a summary, not a description of every error: it does not reveal which cases produced the misses or whether errors tend to be positive or negative.
Worked Example: Interpreting Claim-Cost RMSE in Dollars
A hypothetical insurer reviews three predicted claim costs. Both observed values and predictions are recorded in thousands of dollars. The signed errors are \(-2\), \(2\), and \(3\) thousand dollars. Calculate RMSE and report it in dollars to the nearest cent.
Method. Square the errors in thousands of dollars: \((-2)^2=4\), \(2^2=4\), and \(3^2=9\), each in squared thousands of dollars. Their sum is \(4+4+9=17\), so the MSE is \(17/3=5.666666\ldots\) squared thousands of dollars.
Taking the square root gives \(\sqrt{17/3}=2.380476\ldots\) thousand dollars. Since one thousand dollars is $1,000, the RMSE in dollars is \(2.380476\ldots \times 1{,}000=\$2{,}380.476\ldots\), which rounds to $2,380.48 to the nearest cent.
The answer is in dollars, the same units as the claim costs. The intermediate value \(5.666666\ldots\) is in squared thousands of dollars, not dollars, so it must not be interpreted as a typical claim-cost error. Common mistake avoided: reporting the MSE as the RMSE would leave the result on the squared scale and would omit the required square root.
Common Calculation and Interpretation Traps
- Taking the square root too soon. The correct operation is to average the squared errors first, then take the square root. Averaging the square roots of the squared errors instead gives an average absolute error, not RMSE.
- Letting error signs cancel. Do not average the signed errors and then take a square root. Positive and negative errors can sum to zero even when predictions are not exact.
- Forgetting the denominator. Divide the sum of squared errors by the number of evaluated cases \(n\) before taking the square root.
- Using mismatched units. Convert observations and predictions to the same units before finding their differences.
- Overstating what RMSE says. RMSE summarizes squared-error size; it does not show error direction, describe every case, or by itself establish that a model is suitable.
A useful final check is to ask whether the reported RMSE is nonnegative, uses the outcome’s units, and was calculated from the same evaluation cases for every model being compared. These checks catch many avoidable errors. The result should be reported with enough context to identify the outcome, units, and evaluation set.
Check Your Understanding
For each question, use observed minus predicted as the signed error unless the question gives errors directly.
- A hypothetical workshop observes costs of $14, $18, and $20, with predictions of $12, $21, and $19. Calculate RMSE in dollars to two decimal places. (Answer: $2.16)
- A hypothetical depot records four weekly demands: 40, 50, 60, and 70 units. Its predictions are 42, 47, 64, and 67 units. Calculate RMSE to two decimal places. (Answer: 3.08 units)
- For the same four evaluation cases, Model A has errors of \(1,1,1,5\) days and Model B has errors of \(2,2,2,2\) days. Calculate both RMSE values and identify which is lower. (Answer: Model A: 2.65 days; Model B: 2.00 days; Model B is lower)
- A hypothetical claims team has three errors of \(-2,2,3\) thousand dollars. Calculate RMSE in thousands of dollars to four decimal places, then convert it to dollars to the nearest cent. (Answer: 2.3805 thousand dollars, or $2,380.48)
- Two models are evaluated on the same five cases, with squared errors summing to 125 for Model C and 80 for Model D. Calculate both RMSE values and the difference, to two decimal places, and state which model has lower RMSE. (Answer: Model C: 5.00 units; Model D: 4.00 units; difference: 1.00 unit; Model D is lower)