A Worked Mean Absolute Error Calculation

From Individual Predictions to One Error Summary

A prediction metric condenses a set of case-by-case comparisons into a number. To make that number useful, the calculation must preserve the link between each observed outcome and the prediction made for that same case. Mean absolute error, abbreviated MAE, does this by measuring the size of each prediction error without letting its direction change that size, then averaging those magnitudes.

This tutorial focuses on carrying out and interpreting that calculation. The examples use small, hypothetical sets of predictions so that every contribution to the final average can be checked. In an applied analysis, the cases should be ones appropriate for the question being asked. For example, to summarize performance on new cases, predictions should be evaluated on cases whose outcomes were not used to fit the model or choose its settings. The arithmetic of MAE is the same regardless of how the predictions were produced.

The observed value and prediction must refer to the same case, and they must be expressed on the same outcome scale and in the same units. There must be at least one complete observed-predicted pair. MAE itself requires no assumption that errors are normally distributed or that a regression relationship is linear: it is a descriptive calculation on the supplied pairs. However, a summary intended to describe future prediction performance is only as relevant as the cases and prediction process it summarizes.

Worked Example: Predicting Clinic Wait Times

Why Take Absolute Values Before Averaging?

Let \(e_i=y_i-\hat{y}_i\) denote the signed error for case \(i\), using observed minus predicted as the convention. A positive \(e_i\) means the observation exceeded the prediction; a negative \(e_i\) means the prediction exceeded the observation. But MAE is intended to summarize the size of the errors, not their direction. Taking \(|e_i|\) makes each contribution nonnegative before the contributions are averaged.

The absolute value is defined as \( |e_i|=e_i\) when \(e_i\geq 0\), and \( |e_i|=-e_i\) when \(e_i<0\). Thus the contribution is always a distance from prediction to observation on the outcome scale. For instance, errors of \(3\) minutes and \(-3\) minutes each contribute \(3\) minutes to MAE.

The calculation follows directly from the definition. If the absolute errors are \(a_1,\ldots,a_n\), where \(a_i=|y_i-\hat{y}_i|\), their arithmetic mean is their total divided by the number of values. Therefore:

This also explains the units. Subtracting an outcome from a prediction expressed in the same units produces an error in those units; taking an absolute value does not change the units; and averaging values in a common unit keeps that unit. MAE is therefore reported in the units of the outcome, such as minutes, grams, or dollars. It is not a percentage unless it has been separately transformed into a percentage-based quantity.

Worked Example: Predicting Blood Glucose

Worked Example: Converting the Result to Another Unit

Reading and Checking an MAE Calculation

A useful calculation is easy to audit. Keep the observed outcome, prediction, and absolute error together for each case, either in a table or in an explicitly ordered list. Then check the following:

• Pairing: each prediction is matched with the observation for the same case.
• Scale: the observed value and prediction use the same outcome scale and units before subtraction.
• Magnitude: each contribution is nonnegative because it is an absolute value.
• Count: the denominator is the number of complete pairs included in the sum.
• Context: the reported unit matches the outcome, and the interpretation says this is an average over the cases evaluated.

As a quick arithmetic check, MAE must be at least zero. It is zero only if every prediction exactly matches its observed outcome, since a sum of nonnegative absolute errors can equal zero only when each term is zero. MAE cannot exceed the largest absolute error in the set: the average of values all between zero and that maximum must also lie between those limits. These checks can flag an incorrect total or an implausible final value, although they do not replace checking each case.

Common Mistakes

• Using signed differences in the average. A difference can be negative, but MAE uses its absolute value. For example, an observed value of 104 and prediction of 110 contribute 6 units, not \(-6\) units.
• Dividing by the wrong number. MAE is the mean of the case-level absolute errors, so divide by the number of paired cases used in the sum. Do not use \(n-1\); that divisor belongs to certain sample-variance calculations, not this arithmetic mean.
• Changing units partway through. Do not subtract minutes from hours or combine errors expressed on different scales. Put all pairs in consistent units first.
• Describing the average as a typical error for every case. MAE is the average magnitude across the set. Individual errors can be smaller or larger, as the tables show.
• Overgeneralizing from the cases evaluated. A calculated MAE describes the supplied pairs. A claim about future performance also depends on whether the evaluation cases and prediction process are relevant to the intended use.