Why Keep Observations, Predictions, and Errors Separate?
A numeric prediction is not the outcome that eventually occurs. For an individual case, an insurer might predict a claim amount of $1,200, while the observed claim amount is $900. Reporting only the prediction hides the difference; reporting only the difference hides the level of the outcome. Careful error analysis begins by keeping three quantities distinct: the observed value, the predicted value, and the error calculated from them.
This tutorial establishes a consistent sign convention and applies it to individual observations. The calculations are the starting point for later summaries of prediction performance; here, the goal is to calculate and interpret each case’s signed difference, not to combine errors into an overall metric.
The Sign Convention and Its Interpretation
The subtraction order matters. Throughout this tutorial, the error is observed minus predicted, not predicted minus observed. A positive error therefore indicates an underprediction of that particular outcome, while a negative error indicates an overprediction. “Underprediction” and “overprediction” describe the prediction relative to the observed value; they do not, by themselves, say whether the prediction method is generally biased.
An error has the same units as the outcome. If \(y_i\) is a claim amount in dollars, then \(e_i\) is in dollars. If \(y_i\) is a count of visits, the error is in visits. The error is not a percentage unless a separate calculation defines it that way. It can be fractional even when the observed outcome is a count, because a model may predict an expected count such as 2.4 visits.
The arithmetic definition requires only that the observed value and prediction refer to the same case, outcome, and measurement scale. No probability distribution or particular regression model is needed to subtract one from the other. The quality of a prediction may depend on how it was produced, but the individual signed difference is defined once \(y_i\) and \(\hat{y}_i\) are available.
Worked Example: Individual Claim Amount Predictions
A hypothetical insurer predicts the next-year claim amounts for three policies. The observed and predicted amounts are shown below. Calculate each signed error and interpret its sign.
| Policy | Observed \(y_i\) ($) | Predicted \(\hat{y}_i\) ($) |
|---|---|---|
| A | 1,500 | 1,200 |
| B | 800 | 1,000 |
| C | 2,400 | 2,400 |
Method. Apply \(e_i=y_i-\hat{y}_i\) separately to each policy. The common mistake to avoid is reversing the subtraction, which would reverse every sign and its interpretation.
For policy A, $$ e_A = 1{,}500 - 1{,}200 = 300 \text{ dollars}. $$ The observed claim is $300 greater than predicted, so the prediction was too low by $300.
For policy B, $$ e_B = 800 - 1{,}000 = -200 \text{ dollars}. $$ The observed claim is $200 less than predicted, so the prediction was too high by $200.
For policy C, $$ e_C = 2{,}400 - 2{,}400 = 0 \text{ dollars}. $$ The prediction equals the observed amount.
Check. Adding each error to its prediction recovers the observation: \(1{,}200+300=1{,}500\), \(1{,}000+(-200)=800\), and \(2{,}400+0=2{,}400\). All three calculations are consistent with observed minus predicted.
Prediction Errors and Residuals
The word residual is commonly used for the difference between an observed response and a model’s fitted value on data used to fit the model. For a fitted case \(i\), write the fitted value as \(\hat{y}_i\); its residual is \(r_i=y_i-\hat{y}_i\). This uses the same subtraction order as the signed prediction error defined above.
For a new case not used to fit the model, the difference between its later-observed outcome and its model prediction is usually called a prediction error. The distinction is about the case’s role in model fitting, not a different subtraction rule. In either setting, observed minus predicted gives a positive difference when the model’s value is too low. A prediction calculated on training data is a fitted value; a prediction for a held-out or future case is an out-of-sample prediction.
It is essential to match the observation and prediction correctly. The prediction must correspond to the same case and the same target quantity as the observed value. Comparing a monthly predicted amount with an annual observed amount, or pairing one person’s observation with another person’s prediction, does not produce a meaningful individual error.
Why the Error Formula Works
The definition follows from expressing an observation as its predicted part plus the amount by which it differs from that prediction. Rearranging this identity gives the signed error directly.
For any paired observed value \(y_i\) and prediction \(\hat{y}_i\), write the observation as the prediction plus a difference: \(y_i=\hat{y}_i+(y_i-\hat{y}_i)\).
Define \(e_i=y_i-\hat{y}_i\). Substituting this definition into the previous equality gives \(y_i=\hat{y}_i+e_i\).
If \(e_i>0\), then \(y_i>\hat{y}_i\), so the prediction is below the observation. If \(e_i<0\), then \(y_i<\hat{y}_i\), so the prediction is above the observation. If \(e_i=0\), the two values are equal.
This also provides a quick arithmetic check: adding the signed error to the prediction must recover the observed value. If it does not, either the subtraction or the sign has been handled incorrectly.
Worked Residuals from a Fitted Regression Line
For a model that predicts a numeric response from an input \(x\), a fitted value is obtained by inserting the case’s input into the fitted prediction equation. The residual is then observed response minus that fitted value. The next example uses a specified line only to calculate fitted values and residuals; it does not require assumptions about a distribution for the outcomes.
Worked Example: Residuals for Fitted Service Times
A hypothetical service team fits the line \(\hat{y}=10+2x\), where \(x\) is the number of items in a request and \(y\) is service time in minutes. For three requests, \(x\) is 1, 3, and 5, and the observed service times are 13, 14, and 22 minutes. Find each fitted value and residual.
Method. First calculate \(\hat{y}_i=10+2x_i\), then calculate \(r_i=y_i-\hat{y}_i\). A common mistake is to treat the fitted value as the residual; they are different quantities.
For the request with \(x=1\), $$ \hat{y}_1=10+2(1)=12 \text{ minutes}, \qquad r_1=13-12=1 \text{ minute}. $$ The model underpredicts this observed service time by 1 minute.
For the request with \(x=3\), $$ \hat{y}_2=10+2(3)=16 \text{ minutes}, \qquad r_2=14-16=-2 \text{ minutes}. $$ The model overpredicts this observed service time by 2 minutes.
For the request with \(x=5\), $$ \hat{y}_3=10+2(5)=20 \text{ minutes}, \qquad r_3=22-20=2 \text{ minutes}. $$ The model underpredicts this observed service time by 2 minutes.
Check. The fitted values plus residuals recover the observations: \(12+1=13\), \(16+(-2)=14\), and \(20+2=22\) minutes. The signs and units are consistent.
Calculating Errors in a Data Workflow
In a data set, observed outcomes and predictions are often stored in separate vectors or columns. The calculation is element-by-element: the first observation is paired with the first prediction, the second with the second, and so on. For example, in R, if observed and predicted contain paired numeric values in the same order, the signed errors can be calculated as observed - predicted. Before interpreting results, verify that the rows are aligned and the units agree.
Worked Example: A Predicted Number of Annual Visits
A hypothetical benefits analyst predicts annual clinic visits for three members. The model’s predicted values are 2.5, 4.0, and 1.2 visits. After the year ends, the observed counts are 3, 2, and 0 visits. Calculate the individual prediction errors and identify which prediction was closest to its observation in absolute difference, without calculating a group performance metric.
Method. Subtract the prediction from the observed count for each member. Fractional predictions are valid: a model may predict an expected count even though the realized count is an integer. The trap is to round the predicted value before calculating the error, which changes the requested difference.
For member 1, $$ e_1=3-2.5=0.5 \text{ visits}. $$ The observed count exceeds the prediction by 0.5 visit.
For member 2, $$ e_2=2-4.0=-2.0 \text{ visits}. $$ The observed count is 2.0 visits below the prediction.
For member 3, $$ e_3=0-1.2=-1.2 \text{ visits}. $$ The observed count is 1.2 visits below the prediction.
To identify the closest individual prediction, compare the magnitudes of the three differences: \(|0.5|=0.5\), \(|-2.0|=2.0\), and \(|-1.2|=1.2\) visits. The smallest magnitude is 0.5, so member 1’s prediction is closest to that member’s observed count. This comparison concerns these cases only; it is not a general summary of model performance.
Check. Reconstructing the observations gives \(2.5+0.5=3\), \(4.0+(-2.0)=2\), and \(1.2+(-1.2)=0\).
Common Sign and Matching Traps
A frequent error is silently switching to predicted minus observed. For example, if the observation is 70 and the prediction is 82, then observed minus predicted is \(70-82=-12\): the model overpredicted by 12 units. The reversed calculation \(82-70=12\) is not the signed error under the convention used here; it has the opposite sign.
Other errors arise before the subtraction: using the wrong row, mixing currencies or time periods, or treating a fitted value as though it were an observed value. A sound calculation records the outcome, the prediction, their units, and the subtraction order. Interpret the sign only after those details are confirmed.
- Positive does not mean “good.” It means the observation is greater than the prediction. Whether that is desirable depends on the application.
- Negative does not mean “bad.” It means the prediction exceeds the observation for that case.
- Do not round early. Preserve the prediction’s given precision through subtraction, then round only if the question requests it.
- Do not confuse a residual with a prediction. The fitted value is \(\hat{y}_i\); the residual is \(y_i-\hat{y}_i\).
Check Your Understanding
For each problem, use observed minus predicted and keep the stated units.
- A hypothetical appliance plan predicts a repair cost of $460 for a household; the observed repair cost is $525. Find the signed error and state whether the prediction was too high or too low. (Answer: \(525-460=65\) dollars; the prediction was too low.)
- A model predicts 18.5 minutes for a delivery, but the observed time is 21 minutes. Calculate the prediction error and verify it by reconstruction. (Answer: \(21-18.5=2.5\) minutes; \(18.5+2.5=21\), so the prediction was too low.)
- A fitted line gives \(\hat{y}=6+3x\). For a case with \(x=4\), the observed response is 15. Calculate the fitted value and residual, then interpret the sign. (Answer: \(\hat{y}=6+3(4)=18\); \(r=15-18=-3\), so the fitted value is 3 units too high.)
- A hypothetical model predicts quarterly payments of $1,050, $900, and $1,300; the corresponding observations are $980, $1,050, and $1,300. Calculate all three signed errors and identify any exact match. (Answer: \(-70\), \(150\), and \(0\) dollars, respectively; the third prediction is an exact match.)
- A model predicts 3.6 events for a case, while 5 events are observed. A colleague reports the error as \(-1.4\) by calculating \(3.6-5\). Correct the signed error and explain the direction. (Answer: \(5-3.6=1.4\) events; the positive error means the prediction was too low.)