Why the Choice Between MAE and RMSE Matters
When predictions are numeric, a model can be evaluated by comparing each prediction with its observed value. But a single summary score must combine errors that may differ in size. The choice of summary affects which model appears better: mean absolute error (MAE) treats error size proportionally, while root mean squared error (RMSE) gives unusually large errors greater influence.
The appropriate metric depends on the task. If the main concern is the average size of a typical miss, MAE may be easier to interpret. If large misses deserve substantially more attention, RMSE may better reflect that concern. Neither metric is automatically best: the choice should follow the consequences of error and the scale on which decisions are made.
The sign convention for \(e_i\) identifies whether a prediction was above or below the observed value, but the sign disappears in both metrics. Consequently, MAE and RMSE describe error magnitude, not whether a model tends to overpredict or underpredict. Their calculation also presumes that the predictions and outcomes are on the same scale and refer to the same observations. Model scores are not a fair comparison if they are calculated on different cases or different target scales.
How RMSE Emphasizes Large Errors
MAE adds error magnitudes directly. Doubling an error doubles its contribution to the total absolute error. RMSE first squares each error, so doubling an error makes its squared contribution four times as large. Taking the square root at the end restores the original units, but does not undo the greater influence of large errors.
The relationship can also be seen by comparing a large error with several smaller ones. An error of magnitude \(k\) contributes \(k\) to the sum used in MAE and \(k^2\) to the sum used in RMSE. For example, an error of 8 contributes 8 to the absolute-error total but 64 to the squared-error total. This does not mean RMSE is always larger than MAE for every possible set of errors; rather, large errors have a greater effect on RMSE relative to smaller errors.
RMSE is the square root of mean squared error. Because the square-root function is strictly increasing for nonnegative values, two models evaluated on the same observations are ranked the same by mean squared error and RMSE. The square root changes the reported scale, not that ranking. MAE may rank the models differently because it adds absolute errors instead of squared errors.
Worked Example: Calculating Both Metrics for One Model
A hypothetical service team predicts the number of requests received in each of four periods. The observed and predicted counts are shown below. Calculate MAE and RMSE, then interpret the scores.
| Period | Observed \(y_i\) | Predicted \(\hat{y}_i\) | Error \(e_i=y_i-\hat{y}_i\) | \(|e_i|\) | \(e_i^2\) |
|---|---|---|---|---|---|
| 1 | 10 | 9 | 1 | 1 | 1 |
| 2 | 12 | 13 | -1 | 1 | 1 |
| 3 | 8 | 7 | 1 | 1 | 1 |
| 4 | 15 | 8 | 7 | 7 | 49 |
Step 1: Find MAE. The absolute errors sum to \(1+1+1+7=10\). There are four periods, so \(\operatorname{MAE}=10/4=2.5\) requests.
Step 2: Find RMSE. The squared errors sum to \(1+1+1+49=52\). The mean squared error is \(52/4=13\), so \(\operatorname{RMSE}=\sqrt{13}=3.606\) requests, rounded to three decimal places.
Check and interpretation. The fourth period's error is much larger than the other three. It contributes 7 of the 10 total absolute-error units and 49 of the 52 squared-error units. That period therefore has especially strong influence on RMSE. Both scores are in requests, not requests squared. This calculation avoids the common mistake of treating the mean squared error, 13, as RMSE.
When the Metrics Rank Models Differently
A model with several small errors and one large error can have lower MAE but higher RMSE than a model with more consistent errors. Which model is preferable depends on how important the large miss is to the application. MAE is often useful when each additional unit of error has roughly the same significance. RMSE is useful when unusually large errors should count disproportionately in the evaluation.
Worked Example: A Large Miss Changes the Preferred Model
Two hypothetical models forecast daily equipment demand on the same five days. Their prediction-error magnitudes, in units, are:
- Model A: \(1, 1, 1, 1, 9\)
- Model B: \(3, 3, 3, 3, 3\)
Step 1: Calculate Model A's MAE. Its absolute errors sum to \(1+1+1+1+9=13\). Thus, \(\operatorname{MAE}_A=13/5=2.6\) units.
Step 2: Calculate Model A's RMSE. Its squared errors sum to \(1+1+1+1+81=85\). The mean squared error is \(85/5=17\), so \(\operatorname{RMSE}_A=\sqrt{17}=4.123\) units, rounded to three decimal places.
Step 3: Calculate Model B's MAE. Its absolute errors sum to \(3+3+3+3+3=15\). Thus, \(\operatorname{MAE}_B=15/5=3\) units.
Step 4: Calculate Model B's RMSE. Its squared errors sum to \(9+9+9+9+9=45\). The mean squared error is \(45/5=9\), so \(\operatorname{RMSE}_B=\sqrt{9}=3\) units.
Conclusion. MAE favors Model A because \(2.6<3\); RMSE favors Model B because \(4.123>3\). If the large demand miss on one day is particularly costly, RMSE may better reflect the evaluation goal. If average absolute error is the main concern and that one miss is not disproportionately costly, MAE may be more relevant. The mistake to avoid is choosing a metric after seeing which model it favors rather than matching the metric to the consequences of error.
Interpret the Metric on the Decision Scale
Both metrics are expressed in the outcome's units, which helps connect the score to a practical decision. For a forecast in dollars, the scores are in dollars; for a forecast in thousands of dollars, the scores are in thousands of dollars. The reported unit matters: a score of 1.2 in thousands of dollars means $1,200, not $1.20. Reporting a metric without naming its units can make an otherwise correct result misleading.
Changing the measurement unit by a positive factor changes both metrics by that same factor. If all errors are converted from thousands of dollars to dollars, each error is multiplied by 1,000. Absolute errors are multiplied by 1,000, and squared errors by \(1{,}000^2\); the square root in RMSE then leaves a factor of 1,000. The model ranking on either metric is unchanged by this conversion, but the values must be interpreted on the unit scale used by decision makers.
Worked Example: Reading Errors Reported in Thousands of Dollars
For the same four hypothetical claims, two models have errors measured in thousands of dollars. Model A's error magnitudes are \(0.5, 0.5, 0.5, 2.5\); Model B's are \(1, 1, 1, 1\). Calculate both metrics for each model and report the results in dollars.
Step 1: Model A. Its absolute errors sum to \(0.5+0.5+0.5+2.5=4\), so \(\operatorname{MAE}_A=4/4=1\) thousand dollars. Its squared errors sum to \(0.25+0.25+0.25+6.25=7\), so \(\operatorname{RMSE}_A=\sqrt{7/4}=\sqrt{1.75}=1.322876\) thousand dollars.
Step 2: Model B. Its absolute errors sum to \(1+1+1+1=4\), so \(\operatorname{MAE}_B=4/4=1\) thousand dollars. Its squared errors sum to \(1+1+1+1=4\), so \(\operatorname{RMSE}_B=\sqrt{4/4}=1\) thousand dollars.
Step 3: Convert units. Multiply each score in thousands by 1,000. Both models have MAE of $1,000. Model A's RMSE is $1,322.88, and Model B's RMSE is $1,000.00, rounded to the nearest cent.
Conclusion. MAE sees the models as tied on average absolute error, while RMSE penalizes Model A's larger deviation more. If the cost of a large claim-prediction miss is a key concern, Model B has the lower RMSE. This interpretation is based on hypothetical claim errors, not actual company experience.
Practical Selection and Common Traps
Before calculating either metric, state what one unit of prediction error means in the application. Then decide whether the evaluation should reflect average absolute miss size or put extra emphasis on large deviations. A useful selection process is:
Confirm that predictions and observed outcomes use the same units and scale, and identify the practical meaning of one unit of error.
If unusually large errors warrant extra penalty, RMSE may be appropriate. If error size should accumulate proportionally, MAE may be easier to align with the objective.
Calculate the chosen metric for each candidate model on the same observations and report the units. Explain why the metric matches the decision, rather than treating a lower score as a complete decision by itself.
A common calculation trap is to take the square root of the mean absolute error. For the four errors \(1,-1,1,7\), that incorrect operation gives \(\sqrt{(1+1+1+7)/4}=\sqrt{2.5}=1.581\). RMSE instead squares each signed error, averages the squares, and then takes the square root: \(\sqrt{(1+1+1+49)/4}=\sqrt{13}=3.606\). The two procedures are not interchangeable.
- Do not report mean squared error as RMSE. Mean squared error is in squared units; RMSE takes its square root and returns to the outcome's units.
- Do not assume the lowest score is universally best. A lower score is meaningful only under the selected metric and a fair, same-case comparison.
- Do not treat either metric as a measure of error direction. The absolute value and square remove the sign. A separate check of overprediction and underprediction may be needed for a particular task.
- Do not assume RMSE is always preferable because it penalizes extremes. That emphasis is useful only when the evaluation objective gives large errors extra importance.
Check Your Understanding
For each problem, use the stated errors as observed outcomes minus predictions. Round metric values to three decimal places unless the question specifies otherwise.
- A model has errors \(-2, 2, -4, 4\) on four observations. Calculate MAE and RMSE. (Answer: MAE \(=3\); RMSE \(=\sqrt{10}=3.162\).)
- On the same five days, Model A has error magnitudes \(0,0,0,12,0\), and Model B has magnitudes \(4,4,4,4,4\). Which model has lower MAE, and which has lower RMSE? (Answer: Model A has MAE \(12/5=2.4\) and RMSE \(\sqrt{144/5}=5.367\); Model B has MAE \(4\) and RMSE \(4\). MAE favors A; RMSE favors B.)
- Three forecast errors are \(-1,2,-2\) thousand dollars. Calculate MAE and RMSE in dollars, rounding to the nearest cent. (Answer: MAE \(= \$1{,}666.67\); RMSE \(=\sqrt{3}\) thousand dollars \(=\$1{,}732.05\).)
- A model has six errors \(1,-1,2,-2,3,-3\). Calculate MAE and RMSE, and state which is larger. (Answer: MAE \(=2\); RMSE \(=\sqrt{14/3}=2.160\); RMSE is larger.)
- For errors \(1,1,1,7\), a candidate calculates RMSE as \(\sqrt{(1+1+1+7)/4}\). Identify the error in the method and give the correct RMSE. (Answer: The candidate averaged absolute errors instead of squared errors. Correct RMSE \(=\sqrt{(1+1+1+49)/4}=\sqrt{13}=3.606\).)