Absolute Loss and the Median Prediction

Why the Choice of Prediction Depends on the Error Measure

A prediction can miss an outcome by a small amount or by a large amount, and an error measure determines how those misses are summarized. Mean absolute error (MAE) treats a miss of 5 outcome units as five times the error of a miss of 1 unit. When we must make the same constant prediction for every outcome in a set, choosing that constant to minimize MAE leads to a specific summary of the outcomes: a median.

This result is useful for constructing a simple baseline prediction and for understanding what an MAE-based fitting procedure rewards. It is about minimizing absolute error over a fixed set of outcomes. It does not say that a median-based prediction must perform best on new data, or that every prediction from a model should be the median of all observed outcomes.

Definition: For observed outcomes $y_1,\ldots,y_n$ and a single constant prediction $c$, the total absolute error is $$ L(c)=\sum_{i=1}^{n}|y_i-c|. $$ The mean absolute error for these outcomes is $$ \operatorname{MAE}(c)=\frac{1}{n}\sum_{i=1}^{n}|y_i-c|=\frac{L(c)}{n}. $$ A median of the outcomes is a value with at least half the observations at or below it and at least half at or above it. For an odd number of observations, the median is the middle value after sorting. For an even number, every value between the two middle observations is a median; their midpoint is the conventional sample median.

Here $c$ is a prediction in the same units as the outcome. For example, if outcomes are measured in minutes, then $c$, each absolute error, total absolute error, and MAE are also measured in minutes. Since $n$ is fixed and positive, minimizing total absolute error and minimizing MAE give exactly the same choices: dividing every candidate total by $n$ does not change which one is smallest.

Why a Median Minimizes Absolute Error

Sort the outcomes from smallest to largest, writing them as $y_{(1)}\leq y_{(2)}\leq\cdots\leq y_{(n)}$. Consider any two values $a\leq b$ from this ordered list. For every proposed prediction $c$, the triangle inequality gives

|a-c|+|b-c|\geq |b-a|=b-a.

If $c$ lies between $a$ and $b$, the two errors add to exactly $b-a$. Thus, for a pair of outcomes, any prediction between them attains the smallest possible combined absolute error for that pair.

Pair the smallest outcome with the largest, the second-smallest with the second-largest, and continue inward. When $n$ is even, these pairs are $(y_{(i)},y_{(n+1-i)})$ for $i=1,\ldots,n/2$. A prediction $c$ between the two middle outcomes, $y_{(n/2)}$ and $y_{(n/2+1)}$, lies between the two outcomes in every pair. It therefore minimizes the error for every pair at once, and hence minimizes their sum. When $n$ is odd, pair the outer outcomes in the same way and leave the middle observation $y_{((n+1)/2)}$ unpaired. The middle observation lies between the values in every pair, so it minimizes all paired errors; predicting that middle value also makes the unpaired observation's error zero. This proves the result for both sample sizes.

Theorem: Among all constant predictions $c$, the values minimizing $\sum_{i=1}^{n}|y_i-c|$, and therefore the values minimizing MAE, are the medians of the outcomes. For odd $n$, this is the middle ordered outcome. For even $n$, every $c$ in the closed interval from the lower middle outcome to the upper middle outcome minimizes the total.

The even-sample case deserves care. The conventional median is often reported as the midpoint of the two central observations, but that midpoint is only one of the minimizing predictions. Every prediction in the interval between those observations has the same minimum total absolute error. Predictions outside the interval have a larger total.

Worked Example: Finding the Best Constant for Five Outcomes

Suppose five hypothetical delivery times, in minutes, are 4, 7, 8, 10, and 31. We want one constant prediction for all five. The values are already ordered, and because $n=5$, the middle outcome is 8 minutes. Calculate MAE at 8 minutes and compare it with a nearby candidate, 10 minutes.

At $c=8$ minutes, the absolute errors are $4,1,0,2,$ and $23$ minutes. Therefore,

L(8)=|4-8|+|7-8|+|8-8|+|10-8|+|31-8| =4+1+0+2+23=30\text{ minutes},

and

\operatorname{MAE}(8)=\frac{30}{5}=6\text{ minutes}.

At $c=10$ minutes, the total is

L(10)=6+3+2+0+21=32\text{ minutes}, \qquad \operatorname{MAE}(10)=\frac{32}{5}=6.4\text{ minutes}.

The median prediction has the smaller MAE. The theorem guarantees it minimizes MAE over all constant predictions, not only over the two candidates calculated here. The relatively large outcome of 31 minutes contributes an error of 23 minutes at the median prediction, but it does not pull the median as far as it can pull some other summaries.

Worked Example: A Range of Best Predictions for Four Outcomes

Consider the hypothetical outcomes 2, 5, 9, and 14 units. The two middle values are 5 and 9, so every constant prediction from 5 through 9 units minimizes total absolute error. Check three predictions in this interval.

For $c=5$, the total is $3+0+4+9=16$ units, so the MAE is $16/4=4$ units. For $c=7$, the total is $5+2+2+7=16$ units, again giving an MAE of 4 units. For $c=9$, the total is $7+4+0+5=16$ units, also giving an MAE of 4 units.

The same minimum can be seen by pairing the outer values. For any $c$ between 5 and 9, the pair 2 and 14 contributes $14-2=12$ units, while the pair 5 and 9 contributes $9-5=4$ units. Thus the total is $12+4=16$ units throughout the interval. Outside it, at least one pair's error is larger than its minimum. The conventional sample median is $(5+9)/2=7$, but 7 is not the only MAE-minimizing constant.

Using the Result in Prediction

Suppose a baseline procedure ignores all predictors and assigns one constant to every case. If it is fitted by minimizing MAE on a training set, its best constant is a median of the training outcomes. This gives a clear rule for constructing an MAE-based constant baseline. It also explains why the target of a fitting procedure depends on the loss it is designed to minimize.

Model evaluation is a separate step. A median calculated from training outcomes minimizes absolute error on those training outcomes among constants. It does not guarantee the smallest MAE on a separate validation or test set. To compare prediction procedures, calculate their errors on the evaluation outcomes and compare their MAEs there. A model that makes different predictions for different cases is not restricted to a single constant, so the constant-median theorem alone does not decide which such model has lower evaluation MAE.

Worked Example: Training a Constant Baseline and Evaluating Predictions

A team uses seven hypothetical training outcomes, in minutes: 18, 19, 20, 22, 41, 43, and 44. A constant baseline trained to minimize MAE predicts the median, 22 minutes. Its training total and MAE are

L(22)=4+3+2+0+19+21+22=71\text{ minutes}, \qquad \operatorname{MAE}(22)=\frac{71}{7}\approx10.14\text{ minutes}.

For comparison, a constant prediction of 20 minutes has training total $2+1+0+2+21+23+24=73$ minutes and MAE $73/7\approx10.43$ minutes. The median-based baseline has lower training MAE, as the theorem predicts.

Now consider three separate hypothetical evaluation outcomes: 17, 25, and 39 minutes. The trained constant baseline predicts 22 minutes for each, giving absolute errors $5,3,$ and $17$ minutes. Its evaluation MAE is $(5+3+17)/3=8.00$ minutes. Suppose another prediction procedure gives predictions of 18, 26, and 35 minutes on those same cases. Its errors are $1,1,$ and $4$ minutes, so its evaluation MAE is $(1+1+4)/3=2.00$ minutes. This comparison favors the second procedure on this evaluation set. It does not contradict the median result: 22 minimized training MAE among constant predictions for the training outcomes, while the comparison evaluates two procedures on different outcomes.

Common Mistakes and Practical Meaning

Assuming the median is just another name for the mean. The result here is specifically about absolute error. It identifies a median as a minimizing constant; it does not establish which constant minimizes a different error measure.
Reporting only the midpoint as the answer when $n$ is even. The midpoint is a conventional median, but every value between the two middle outcomes gives the same minimum total absolute error.
Thinking a training-set optimum must win on new data. The median minimizes the observed training-set MAE among constants. New outcomes can have a different distribution, so evaluation MAE must be calculated separately.
Choosing a prediction without specifying the loss. A prediction is not “best” in isolation. The median result applies when the objective is total or mean absolute error, all outcomes have equal weight, and the prediction is a single constant.

Absolute loss is useful when a typical miss in the outcome's original units is the quantity of interest. Its median-minimization property gives a principled constant benchmark rather than an arbitrary one. For more flexible prediction procedures, MAE remains a way to summarize performance, while the median theorem supplies a specific result only for the constant-prediction problem proved here.

Check Your Understanding

Use the definition and result to answer each question.

For outcomes 3, 6, 8, 12, and 20, what constant prediction minimizes MAE? What is its total absolute error?
For outcomes 1, 4, 10, and 13, identify the full range of MAE-minimizing constant predictions.
Explain why dividing total absolute error by $n$ does not change which constant minimizes it.
A constant prediction is the median of a training set but has higher MAE than another procedure on a test set. Does this contradict the median result? Explain.
For what kind of loss and prediction choice does the median-minimization result apply?

Absolute Loss and the Median Prediction

What You'll Learn