Median Absolute Error for Skewed Errors

Why Use a Median Error Summary?

A few predictions can miss by far more than the rest. In that situation, the mean absolute error (MAE) may be much larger than the error experienced on a typical case, because every absolute error contributes to the mean. The median absolute error (MedAE) provides a different view: it reports the middle absolute error after the errors are ordered.

Neither summary is universally better. MAE reflects the average size of the observed misses, including extreme ones. MedAE describes the center of the observed absolute-error distribution and is less sensitive to the magnitude of a small number of extreme misses. They answer related but distinct questions, so reporting both can be informative when the error distribution is strongly right-skewed.

Definition: For $n$ observed responses $y_i$ and corresponding predictions $\hat{y}_i$, the signed residual is $e_i=y_i-\hat{y}_i$, and the absolute error is $a_i=|e_i|=|y_i-\hat{y}_i|$. The median absolute error is the sample median of $a_1,\ldots,a_n$. The mean absolute error is $\frac{1}{n}\sum_{i=1}^{n}a_i$. Both are expressed in the same units as the response.

These are empirical summaries of a specified set of observed-predicted pairs. Their calculation requires that each pair be included in the comparison and that the response and prediction use the same scale and units. No probability distribution assumption is needed. Unless a weighting scheme is explicitly specified, each observation contributes equally.

Calculating the Median Absolute Error

First calculate the absolute errors, then sort them from smallest to largest. If $n$ is odd, the median is the single middle value. If $n$ is even, the median is the arithmetic mean of the two middle values. In either case, the MAE is the sum of all absolute errors divided by $n$.

\operatorname{MedAE}= \begin{cases} a_{((n+1)/2)}, & n\text{ odd},\\[4pt] \dfrac{a_{(n/2)}+a_{(n/2+1)}}{2}, & n\text{ even}, \end{cases} \qquad \operatorname{MAE}=\frac{1}{n}\sum_{i=1}^{n}a_i

Here, $a_{(j)}$ denotes the $j$th value in the ordered list of absolute errors. The subscript in parentheses indicates a rank after sorting; it does not refer to a particular observation before sorting.

Worked Example: A Large Miss Among Repair-Cost Predictions

A hypothetical model predicts repair costs, measured in thousands of dollars, for seven claims. The observed and predicted costs are shown below. Calculate MedAE and MAE, and explain why they differ.

Claim	Observed ($000)	Predicted ($000)	Absolute error ($000)
1	18	17	1
2	22	20	2
3	25	23	2
4	31	28	3
5	34	30	4
6	40	35	5
7	90	50	40

Step 1: Calculate each absolute error. For claim 1, $|18-17|=1$. The remaining errors are $|22-20|=2$, $|25-23|=2$, $|31-28|=3$, $|34-30|=4$, $|40-35|=5$, and $|90-50|=40$, all in thousands of dollars.

Step 2: Order the errors. They are already ordered: $1,2,2,3,4,5,40$. There are seven observations, so the middle rank is $(7+1)/2=4$. The fourth value is 3, giving MedAE $=3$ thousand dollars.

Step 3: Calculate MAE. The sum is $1+2+2+3+4+5+40=57$. Thus MAE $=57/7=8.142857\ldots$ thousand dollars. Rounded to four decimal places, MAE is 8.1429 thousand dollars.

Check and interpretation. The median absolute error is 3 thousand dollars, while the average absolute error is about 8.1429 thousand dollars. The 40-thousand-dollar miss is much larger than the other six errors and raises the mean substantially. MedAE describes the center of this set of misses; it does not imply that every claim is predicted within 3 thousand dollars.

Why the Median Is Less Sensitive to Extreme Magnitudes

The median depends on the rank of each error, not on the total size of all errors. In the seven-claim example, the fourth ordered value determines MedAE. If the largest error changed from 40 to 400 while all other errors stayed the same, the ordered list would be $1,2,2,3,4,5,400$. The fourth value would still be 3, even though MAE would increase.

This stability is not a claim that the median can never change when observations change. If enough errors change, or if an observation crosses the central ranks, the median can move. The point is narrower: changing the magnitude of an already extreme value need not change the median, whereas it changes the mean whenever the value changes.

A short order-statistic argument makes the distinction precise. With seven ordered values, the median is the fourth. Replacing just the largest value by any other value that remains largest leaves the first six ranks unchanged, so the fourth value is unchanged. More generally, when fewer than half of the observations are replaced, the median cannot be driven arbitrarily far solely by making those replacements arbitrarily large: the unaltered observations still occupy at least half of the data and constrain the central rank. The exact median may shift within the range of unaltered central values, so “robust” does not mean “immune to every change.”

Interpretation: MedAE summarizes a central absolute error and is relatively insensitive to the size of a small number of extreme errors. MAE summarizes average absolute error and responds to every error’s magnitude. A large gap between the two can be a signal to inspect the distribution of errors, but the gap alone does not explain why the extreme misses occurred.

Worked Example: An Even Number of Errors

A hypothetical forecasting team evaluates eight monthly demand predictions. The absolute errors, in thousands of units, are $0,1,2,3,4,5,6,30$. Calculate both summaries. Then determine the effect of changing the largest error from 30 to 300.

Step 1: Find MedAE for the original errors. The values are already ordered. With $n=8$, the two middle ranks are 4 and 5, whose values are 3 and 4. Therefore, MedAE $=(3+4)/2=3.5$ thousand units.

Step 2: Find MAE for the original errors. Their sum is $0+1+2+3+4+5+6+30=51$. Hence MAE $=51/8=6.375$ thousand units.

Step 3: Recalculate after the largest error changes. The ordered errors become $0,1,2,3,4,5,6,300$. The middle two values are still 3 and 4, so MedAE remains 3.5 thousand units. The new sum is $0+1+2+3+4+5+6+300=321$, giving MAE $=321/8=40.125$ thousand units.

Check and interpretation. The median is based on the two central ranks and is unchanged in this particular change; the mean rises from 6.375 to 40.125 thousand units. This illustrates sensitivity to the extreme value’s magnitude, not a general guarantee that MedAE will be unchanged whenever one error changes.

Comparing Models with Both Summaries

When comparing two models, calculate both metrics on the same observations and response scale. MedAE can help describe the typical central miss, while MAE reveals whether large errors materially affect the average. If the metrics favor different models, report the disagreement rather than selecting one metric without considering the decision context.

Worked Example: Two Models with the Same Median Error

Two hypothetical models predict the same seven claim severities, measured in thousands of dollars. Their absolute errors are:

Model A: $0,1,2,3,4,5,40$
Model B: $2,2,2,3,3,3,10$

Calculate MedAE and MAE for each model. State what each metric indicates.

Step 1: Calculate Model A’s MedAE and MAE. The fourth of its seven ordered errors is 3, so MedAE is 3 thousand dollars. The sum is $0+1+2+3+4+5+40=55$, so MAE $=55/7=7.857142\ldots$, or 7.8571 thousand dollars to four decimal places.

Step 2: Calculate Model B’s MedAE and MAE. Its fourth ordered error is also 3, so MedAE is 3 thousand dollars. The sum is $2+2+2+3+3+3+10=25$, so MAE $=25/7=3.571428\ldots$, or 3.5714 thousand dollars to four decimal places.

Step 3: Compare. The MedAE values tie: both models have a central absolute error of 3 thousand dollars. Model B has the smaller MAE because its total absolute error is 25 rather than 55 thousand dollars. Model A’s 40-thousand-dollar miss makes its average error considerably larger.

Check and interpretation. These seven cases do not establish how either model will perform on new data. They do show that equal MedAE values can conceal an important difference in average error. The comparison is valid as stated because both models are assessed on the same claims and in the same units.

Common Mistakes and a Practical Calculation

A frequent error is to take the median of signed residuals and then take its absolute value. That is not MedAE: the absolute value must be applied to each residual before finding the median. For example, if signed residuals are $-10,-2,1,3,30$, their median is 1, whose absolute value is 1. But the absolute errors are $10,2,1,3,30$, which order to $1,2,3,10,30$; MedAE is 3.

Another common error is to select one middle value when $n$ is even. The sample median is the mean of the two middle ordered values. Also check the units: an error of 3 in a response recorded in thousands is 3 thousand response units, not 3 individual units.

In R, with observed responses stored in y and predictions in y_hat, the calculations are:

\texttt{median(abs(y - y\_hat))} \qquad\text{and}\qquad \texttt{mean(abs(y - y\_hat))}

Both expressions must use matching observations in the same order. Before interpreting the results, confirm that missing values or mismatched rows have not caused the two summaries to be calculated on different sets of cases.

Key takeaway: Calculate absolute errors first, then summarize their ordered distribution. Use MedAE to describe the central absolute miss and MAE to describe the average absolute miss. When a few errors are extreme, showing both makes their influence on the average visible without hiding the typical error.

Check Your Understanding

For each problem, calculate the requested summaries from the stated errors and interpret the comparison where asked.

A hypothetical model has absolute errors of 1, 4, 2, 9, and 3 dollars. Calculate MedAE and MAE. (Answer: MedAE = $3; MAE = $3.80.)
A forecast has six absolute errors, in thousands of units: 2, 5, 1, 8, 4, and 10. Calculate MedAE and MAE. (Answer: MedAE = 4.5 thousand units; MAE = 5 thousand units.)
A model has eight ordered absolute errors of 1, 2, 2, 3, 4, 5, 24, and 80. Calculate both metrics and explain which is more affected by the two largest errors. (Answer: MedAE = 3.5; MAE = 15.125; MAE is more affected by their magnitudes.)
On the same five claims, Model A has absolute errors 1, 2, 2, 4, and 31 thousand dollars; Model B has errors 3, 3, 3, 3, and 5 thousand dollars. Calculate both metrics and state which model has the smaller median and which has the smaller mean. (Answer: A has MedAE 2 and MAE 8 thousand dollars; B has MedAE 3 and MAE 3.4 thousand dollars. A has the smaller median, while B has the smaller mean.)
A set of six absolute errors is 1, 2, 3, 4, 5, and 100. Calculate MedAE and MAE, and compare the median with the median before the 100 error was added, when the errors were 1, 2, 3, 4, and 5. (Answer: With six errors, MedAE = 3.5 and MAE = 19.1667; before adding 100, MedAE = 3. The extreme error changes the median here because it changes the sample size and central ranks.)

What You'll Learn