Mean Prediction Error and Systematic Bias

Why the Direction of Prediction Errors Matters

Two prediction models can have equally sized errors but behave differently in an important way: one may tend to predict too high, while the other tends to predict too low. Measures such as mean absolute error (MAE) and root mean squared error (RMSE) summarize error size, but they do not retain its direction. Mean prediction error provides that directional information by averaging signed errors.

The sign depends on the error convention, so it must be stated before interpreting a result. Here, prediction error is defined as predicted value minus observed value. With this convention, positive errors are overpredictions and negative errors are underpredictions.

Definition: For evaluation case $i$, let $y_i$ be the observed outcome and $\hat{y}_i$ the corresponding prediction. The signed prediction error is $e_i=\hat{y}_i-y_i$. For $n$ paired predictions and outcomes, the mean prediction error (also called mean signed error) is $$ \mathrm{MPE}=\frac{1}{n}\sum_{i=1}^{n}e_i =\frac{1}{n}\sum_{i=1}^{n}(\hat{y}_i-y_i). $$ An MPE above zero indicates average overprediction on these cases; an MPE below zero indicates average underprediction. Its units are the same as the outcome’s units.

MPE is not the same as MAE or RMSE. MAE averages the absolute values of the errors, and RMSE takes the square root of their average squared values. Since both use error magnitudes rather than signed errors, neither identifies the direction of average error. MPE answers a different question: how far above or below the observed outcomes are the predictions on average?

Calculating and Interpreting MPE

The calculation begins by pairing each prediction with its observed outcome and subtracting in the defined order, prediction minus observation. Only after calculating every signed error should the errors be summed and divided by the number of cases. Keep the outcome’s units attached to the final result: an MPE of $-1.5$ dollars is not an error of $-1.5$ percent.

Worked Example: Calculating Average Underprediction

Suppose a hypothetical model predicts the weekly output of four machines. The outcomes and predictions are measured in units produced per week.

Machine	Observed output $y_i$	Predicted output $\hat{y}_i$	Error $\hat{y}_i-y_i$
1	20	18	$-2$
2	30	28	$-2$
3	40	39	$-1$
4	50	48	$-2$

For example, machine 1 has error $18-20=-2$ units per week. The four signed errors sum to $-2-2-1-2=-7$, so

\mathrm{MPE}=\frac{-7}{4}=-1.75\text{ units per week}.

The negative sign indicates that the predictions were below the observed outputs on average. In this small evaluation set, the model underpredicted by an average of 1.75 units per machine per week. The statement describes the average; it does not mean that every prediction was exactly 1.75 units too low.

One useful identity makes the direction interpretation especially clear. Define the average prediction as $\overline{\hat{y}}=\frac{1}{n}\sum_{i=1}^{n}\hat{y}_i$, and the average observed outcome as $\bar{y}=\frac{1}{n}\sum_{i=1}^{n}y_i$. Then

\begin{aligned} \mathrm{MPE} &=\frac{1}{n}\sum_{i=1}^{n}(\hat{y}_i-y_i)\\ &=\frac{1}{n}\sum_{i=1}^{n}\hat{y}_i-\frac{1}{n}\sum_{i=1}^{n}y_i\\ &=\overline{\hat{y}}-\bar{y}. \end{aligned}

The second equality follows by distributing the sum across the subtraction. Thus, positive MPE means the average prediction exceeds the average observed outcome; negative MPE means it falls below the average observed outcome. This algebra justifies the interpretation, but it does not by itself explain why the difference occurred or whether it will persist in new cases.

Reading the Sign and Magnitude in Context

The sign tells the direction of average error under the stated convention. The magnitude tells how far the average signed error is from zero, measured in outcome units. Both parts matter. Reporting only “MPE is 2.8” omits the units and the practical setting; reporting “MPE is positive” omits the size.

Worked Example: Average Overprediction in Daily Demand

A hypothetical shop uses a model to predict the number of customer visits per day. For five evaluation days, the observed and predicted counts are as follows.

Day	Observed visits	Predicted visits	Error (predicted minus observed)
1	50	53	$+3$
2	64	67	$+3$
3	72	75	$+3$
4	84	86	$+2$
5	90	93	$+3$

The errors total $3+3+3+2+3=14$ visits across five days. Therefore,

\mathrm{MPE}=\frac{14}{5}=2.8\text{ visits per day}.

Because the value is positive, the model overpredicted on average by 2.8 visits per day on these evaluation days. This is a directional summary, not a claim that every daily prediction was high by exactly 2.8 visits. The actual errors here range from 2 to 3 visits.

The unit is visits per day, not a percentage. To express error as a percentage would require a different calculation and a clearly specified denominator; MPE alone is not a percentage-error measure.

What MAE and RMSE Can Conceal

MAE and RMSE are useful when the goal is to summarize how large prediction misses are. However, taking absolute values or squares removes the plus-or-minus direction. Consequently, two models can have the same MAE and RMSE even when one consistently predicts above the observed values and the other consistently predicts below them. MPE complements those size measures by retaining direction.

Worked Example: Same Error Sizes, Opposite Direction

Consider two hypothetical models evaluated on the same four cases, with outcomes measured in units. Model A predicts 98, 118, 138, and 158; Model B predicts 102, 122, 142, and 162. The observed outcomes are 100, 120, 140, and 160.

Case	Observed	Model A prediction	Model A error	Model B prediction	Model B error
1	100	98	$-2$	102	$+2$
2	120	118	$-2$	122	$+2$
3	140	138	$-2$	142	$+2$
4	160	158	$-2$	162	$+2$

For Model A, every error is calculated as prediction minus observation; for case 1, for example, $98-100=-2$. Its mean prediction error is

\mathrm{MPE}_A=\frac{-2-2-2-2}{4}=-2\text{ units}.

Model A therefore underpredicts by 2 units on average. For Model B, case 1 has error $102-100=+2$, and the other three errors are also $+2$. Thus,

\mathrm{MPE}_B=\frac{2+2+2+2}{4}=2\text{ units}.

Model B overpredicts by 2 units on average. For either model, the absolute errors are all 2, so $\mathrm{MAE}=2$ units. The squared errors are all 4, giving

\mathrm{RMSE}=\sqrt{\frac{4+4+4+4}{4}}=2\text{ units}.

The MAE and RMSE are identical for these models, but their MPE values have opposite signs. This example shows why an error-size summary alone cannot tell whether predictions tend to be too high or too low.

Using MPE Responsibly

Mean prediction error should be calculated from predictions and outcomes for the same evaluation cases, using a consistent prediction-minus-observation convention and the same outcome scale. For assessing performance beyond the data used to fit or select a model, the predictions should come from an appropriate validation or test procedure. The interpretation is limited to the cases and evaluation process represented by the calculation.

A nonzero MPE is evidence of a directional average difference in that evaluation set. It does not establish a cause, prove that the same difference will occur in the population of future cases, or show how errors vary across different values of the outcome or predictors. It is also important to distinguish a descriptive evaluation-set MPE from a formal statistical definition of bias, which concerns an expected error over a specified process or population.

MPE is most informative alongside MAE or RMSE: the latter describe typical error size in different ways, while MPE describes net direction and signed average size. Read each metric for the question it answers rather than treating one as a substitute for the others.

Common Mistake

Reversing the subtraction. If error is defined as prediction minus observation, then $102-100=+2$, not $-2$. A positive error is an overprediction under this convention. Reversing the order changes every sign and requires reversing the interpretation too.
Calling positive MPE “better.” Positive means overprediction, not better performance. Whether a direction is practically important depends on the context and consequences of high versus low predictions.
Using MPE as an error-size measure by itself. A value near zero is not enough to conclude that individual predictions are accurate. Consult an error-size measure such as MAE or RMSE as well.
Dropping the units. An MPE of $2.8$ in a count outcome means 2.8 outcome units on average, not 2.8 percent.

Key takeaway: With error defined as prediction minus observation, positive MPE indicates average overprediction and negative MPE indicates average underprediction. Its magnitude is expressed in outcome units. MAE and RMSE summarize error size but, by themselves, do not show this direction.

Check Your Understanding

For each question, use prediction minus observation as the signed-error convention.

Observed outcomes are 12, 18, and 25; predictions are 14, 17, and 24. Calculate the three signed errors and the MPE. Interpret the sign and units if the outcomes are measured in kilograms.
A model has an MPE of $-4$ minutes on a set of travel-time predictions. What does the sign indicate, and what does the magnitude describe?
Explain why two models can have identical MAE and RMSE but different MPE values.
For one case, the prediction is 102 and the observed outcome is 100. Find the signed error and state whether it is an overprediction or underprediction.
Why does an MPE near zero not, by itself, demonstrate that individual predictions are accurate?

Case	Observed	Model A prediction	Model A error	Model B prediction	Model B error
1	100	98	\(-2\)	102	\(+2\)
2	120	118	\(-2\)	122	\(+2\)
3	140	138	\(-2\)	142	\(+2\)
4	160	158	\(-2\)	162	\(+2\)