Mean Error and Directional Bias

Why Direction Matters

A prediction can be too high or too low. A summary that retains this direction helps identify whether predictions tend to fall below or above the values that are eventually observed. For example, a forecasting process that consistently underpredicts claim amounts may require investigation even if some predictions are close to their outcomes.

The sign is meaningful only after the error convention has been stated. This tutorial uses observed value minus predicted value. Under this convention, a positive error means the prediction was too low, while a negative error means it was too high. Reversing the subtraction reverses both interpretations.

Definition: For observation $i$, let $y_i$ be the observed value and $\hat{y}_i$ its prediction. Define the signed error as $e_i=y_i-\hat{y}_i$. For $n$ observations, the mean signed error, also called the mean error under this convention, is

\overline{e}=\frac{1}{n}\sum_{i=1}^{n} e_i =\frac{1}{n}\sum_{i=1}^{n}(y_i-\hat{y}_i).

A positive $\overline{e}$ indicates average underprediction on the observations being summarized; a negative $\overline{e}$ indicates average overprediction. A value of zero indicates no net signed difference across those observations.

The mean signed error is reported in the same units as the outcome. If the outcome is dollars, the mean error is dollars; if it is days, the mean error is days. State the sign convention and the units when reporting it, so a reader can interpret the result without guessing.

Calculation and Interpretation

The calculation has two essential parts: calculate each signed error using the stated subtraction order, then average those errors. The errors must correspond to the same observations, and the ordinary mean gives each observation equal weight. A positive mean does not say that every prediction was too low; it summarizes the net direction after positive and negative errors are combined.

Fix the convention.
Write “observed minus predicted” before calculating. Under this convention, positive means underprediction.

Find each signed error.
For every observation, subtract its prediction from its observed value, keeping the sign.

Sum and divide by the number of observations.
Include every observation once in the sum and divide by $n$, not by the number of segments unless each segment is the unit being averaged.

Interpret the result in context.
Report the direction, the mean amount, and the outcome’s units. Do not infer that every prediction has the same direction.

Worked Example: Mean Error for Monthly Claim Forecasts

A hypothetical analyst predicts paid amounts for three claims. The observed amounts are $120, $90, and $150; the corresponding predictions are $110, $100, and $140. Calculate the mean signed error and explain its direction.

Method. Use observed minus predicted, so the errors are measured in dollars and a positive result indicates underprediction.

For the first claim, $e_1=120-110=10$. For the second, $e_2=90-100=-10$. For the third, $e_3=150-140=10$. Thus the signed errors are $10,-10,10$ dollars.

Their sum is $10+(-10)+10=10$ dollars. There are $n=3$ claims, so

\overline{e}=\frac{10}{3}=3.333\ldots\text{ dollars}.

Rounded to the nearest cent, the mean signed error is $3.33 per claim. It is positive, so the predictions underpredicted by an average of $3.33 across these three claims. This does not mean that all three predictions were too low: the second prediction was $10 too high.

Check. The mean observed amount is $(120+90+150)/3=120$ dollars. The mean predicted amount is $(110+100+140)/3=350/3=116.666\ldots$ dollars. Their difference is $120-116.666\ldots=3.333\ldots$ dollars, agreeing with the calculated mean error. The common mistake to avoid is dropping the negative sign on the second error, which would incorrectly inflate the result.

Why the Mean Error Equals a Difference of Means

The mean signed error can also be understood as the difference between the mean observed value and the mean predicted value, provided both means use the same observations and equal weights. This identity follows directly from distributing the sum across subtraction:

\begin{aligned} \overline{e} &=\frac{1}{n}\sum_{i=1}^{n}(y_i-\hat{y}_i)\\ &=\frac{1}{n}\left(\sum_{i=1}^{n}y_i-\sum_{i=1}^{n}\hat{y}_i\right)\\ &=\frac{1}{n}\sum_{i=1}^{n}y_i-\frac{1}{n}\sum_{i=1}^{n}\hat{y}_i\\ &=\overline{y}-\overline{\hat{y}}. \end{aligned}

The equality holds because the same $n$ observations appear in both sums, so each sum is divided by the same count. It is a useful check on arithmetic and a useful interpretation: mean signed error measures the gap between the average observed outcome and the average prediction. It does not require the individual errors to be similar.

This identity does not justify comparing means calculated from different sets of observations or from different weighting schemes. If one mean excludes observations that the other includes, the subtraction no longer represents the mean of the paired errors. Check that the observations and inclusion rules match before using the identity.

Worked Example: A Zero Mean Can Hide Opposing Errors

A hypothetical service team forecasts the number of daily support requests for five days. The signed errors, calculated as observed minus predicted, are $-8,-4,-6,12,6$ requests. Calculate the mean signed error and interpret what it does and does not show.

Method. Add the signed errors, retaining their signs, and divide by the five days. The negative errors represent overprediction; the positive errors represent underprediction.

The sum is

$$ (-8)+(-4)+(-6)+12+6=-18+18=0. $$

With $n=5$, the mean signed error is $0/5=0$ requests per day. The result shows that positive and negative errors cancel exactly in the signed average. It does not show that every forecast was exact: each of the five listed errors is nonzero.

Check. The three negative errors sum to $-18$ requests, and the two positive errors sum to $18$ requests, so the total is zero. The mean observed count therefore equals the mean predicted count on these five days, consistent with the identity above. The common mistake is to interpret a zero mean signed error as “no prediction error.” It means no net signed difference, not that individual predictions match their observations.

Combining Segments Correctly

Mean errors may be reported separately for groups, such as regions or product types. To obtain an overall observation-level mean, weight each group mean by its number of observations. A small group and a large group should not automatically contribute equally: the ordinary observation-level mean gives each observation equal weight, not each group.

Key result: If segment $j$ has $n_j$ observations and mean signed error $\overline{e}_j$, then the overall observation-level mean signed error is

\overline{e}_{\mathrm{overall}} =\frac{\sum_j n_j\overline{e}_j}{\sum_j n_j}.

This follows because the sum of the errors in segment $j$ is $n_j\overline{e}_j$. Adding those segment sums gives the total error sum, which is divided by the total number of observations.

Worked Example: Unequal Segment Sizes and Overall Direction

A hypothetical portfolio has two segments. Segment A contains two observations with signed errors $+6$ and $+10$ thousand dollars. Segment B contains four observations, each with signed error $-2$ thousand dollars. Find the overall observation-level mean signed error and its direction.

Method. Use the six individual errors, or equivalently weight each segment mean by its observation count. The sign convention is observed minus predicted, so a positive overall result indicates underprediction.

Segment A has error sum $6+10=16$ thousand dollars and mean error $16/2=8$ thousand dollars. Segment B has error sum $4(-2)=-8$ thousand dollars and mean error $-8/4=-2$ thousand dollars.

The total error sum is $16+(-8)=8$ thousand dollars, and the total number of observations is $2+4=6$. Therefore,

\overline{e}_{\mathrm{overall}} =\frac{8}{6} =\frac{4}{3} =1.333\ldots\text{ thousand dollars}.

Rounded to three decimal places, the overall mean signed error is $1.333$ thousand dollars, or approximately $1,333.33 per observation. It is positive, indicating average underprediction across the six observations.

Check and trap. The weighted segment calculation gives $\{2(8)+4(-2)\}/6=(16-8)/6=8/6$, the same answer. A common wrong calculation takes the unweighted average of the two segment means: $(8+(-2))/2=3$ thousand dollars. That gives each segment equal weight, rather than each observation, and is not the requested overall observation-level mean. Another error is claiming the segment totals cancel: $16-8=8$, not zero.

Directional Bias Is Not Overall Error Size

Mean signed error is a measure of directional bias in the evaluated set. It summarizes whether the signed errors lean upward or downward overall. Because positive and negative errors cancel, its magnitude can be small even when individual predictions differ substantially from their observed values. The zero-mean example showed exact cancellation despite five nonzero errors.

Consequently, mean signed error answers a focused question: “On average, are predictions below or above observed values, and by how much in signed units?” It does not, by itself, answer “How far away are predictions from observations overall?” A report should not treat a small mean signed error as proof that individual predictions are accurate.

There is also a distinction between a realized mean error and a persistent model tendency. A positive mean on one evaluation set describes underprediction on that set. It may be evidence worth investigating, especially if it recurs across suitable evaluation samples, but one result alone does not establish that the model will systematically underpredict in every population or future period. Changes in the data, the evaluation sample, or the prediction process can affect the observed mean.

Common Exam Traps

Reversing the subtraction. With $e_i=y_i-\hat{y}_i$, positive means underprediction. If using prediction minus observation instead, the interpretation reverses; state the convention rather than mixing them.
Removing signs before averaging. The mean signed error uses the signed values. Replacing negative errors with positive values changes the quantity and removes the cancellation that defines directional bias.
Using the number of segments as the denominator. For an observation-level mean, divide by the total number of observations. Weight segment means by their counts if working from grouped results.
Calling a zero result perfect prediction. A zero mean can result from positive and negative errors cancelling. It does not imply that each $e_i=0$.
Overstating what a sign proves. A positive sample mean indicates underprediction on the evaluated observations under the stated convention; it is not, by itself, proof of a persistent tendency in all future data.

Key takeaway: Define error as observed minus predicted, keep every sign, and divide the total signed error by the number of observations. Interpret a positive mean as underprediction and a negative mean as overprediction, while remembering that cancellation can conceal substantial individual errors.

Check Your Understanding

For each question, use signed error equal to observed value minus predicted value. Interpret the sign using that convention.

For three hypothetical daily demand observations, the observed values are $42,50,61$ units and the predictions are $45,48,59$ units. Calculate the mean signed error and state its direction. (Answer: $1/3=0.3333$ units; average underprediction.)
A five-day forecast has signed errors $-5,-5,0,4,6$ units. Calculate the mean signed error and explain whether a zero result, if obtained, would mean every prediction was exact. (Answer: $0$ units; the errors cancel, and zero mean does not mean every prediction was exact.)
Segment A has two observations and a mean signed error of $+6$ dollars. Segment B has three observations and a mean signed error of $-1$ dollar. Find the overall observation-level mean error and direction. (Answer: $\{2(6)+3(-1)\}/5=9/5=1.8$ dollars; average underprediction.)
Region A has five observations with mean signed error $-4$ dollars. Region B has three observations with mean signed error $+6$ dollars. Calculate the combined observation-level mean and explain why simply averaging the two regional means gives a different result. (Answer: $\{5(-4)+3(6)\}/8=-2/8=-0.25$ dollars; average overprediction. The unweighted regional average is $1$ dollar because it gives each region equal weight.)
For three hypothetical monthly totals, observed values are $80, $100, and $120, while predictions are $100, $90, and $110. Calculate the mean signed error and state one conclusion it does not support. (Answer: $\{(-20)+10+10\}/3=0$ dollars; no net signed difference, but it does not show that each monthly prediction was exact.)

What You'll Learn