Why Calculate Residuals?
A fitted regression line gives a predicted response for each predictor value. For an individual case, the observed response may differ from that prediction. The residual records the size and direction of this difference as a number, using the same units as the response. Calculating residuals lets us describe, case by case, how observed data relate to a fitted line.
This tutorial concerns the arithmetic for an individual case. Start with a predictor value and its observed response, calculate the response fitted by the line at that predictor value, and subtract. No probability model or distributional assumption is needed for this calculation: the fitted line and the observed data values are enough. The predictor and response must refer to the same case, and the equation and recorded responses must use compatible units.
Definition and Basic Relationship
The symbols distinguish three quantities that should not be confused. The value \(y_i\) is observed in the data. The value \(\hat{y}_i\) is calculated from the fitted line at \(x_i\). The residual \(e_i\) is calculated from those two values. A residual is therefore specific to an individual case and a particular fitted line.
The subtraction order is part of the definition: observed response minus fitted response. It is not fitted response minus observed response. The result is measured in the response’s units, because both quantities being subtracted are responses in those units. If response is measured in minutes, the residual is in minutes; if response is measured in points, the residual is in points.
From the Fitted Line to a Residual
The calculation has two stages. First, substitute the case’s predictor value into the fitted equation to find \(\hat{y}_i\). Then subtract that fitted response from the case’s observed response \(y_i\). Keep the case’s observed response separate from the line’s prediction until the subtraction.
The definition can also be rearranged to show how the three quantities fit together. Adding \(\hat{y}_i\) to both sides of \(e_i=y_i-\hat{y}_i\) gives
This identity is a useful check: the observed response must equal the fitted response plus the residual. It does not mean that the residual is another prediction. The fitted response comes from the line; the residual is the difference needed to relate that prediction to the observation.
Worked Example: One Commute-Time Residual
Suppose a hypothetical fitted line predicts commute time \(y\), in minutes, from distance \(x\), in kilometres:
For one commuter, the distance is \(x_i=2\) km and the observed commute time is \(y_i=23\) minutes. Calculate the fitted response and residual.
Fitted response. Substitute the observed distance into the line:
Residual. Subtract the fitted response from the observed response:
The three values satisfy the check \(y_i=\hat{y}_i+e_i\), since \(23=21+2\). The residual is in minutes, matching the response. It is not a distance residual: the predictor is measured in kilometres, but the subtraction compares two commute times.
Calculating Residuals for Several Cases
When a data set contains several cases, apply the same calculation separately to each one. Use each case’s own \(x_i\) to calculate its fitted response, then use that case’s own \(y_i\) in the subtraction. Organizing the work in columns makes it easier to keep the observed, fitted, and residual values distinct.
Worked Example: Residuals for Three Study-Time Observations
Consider a hypothetical fitted line relating study time \(x\), in hours, to quiz score \(y\), in points:
Three hypothetical students have the following study times and observed scores. Calculate each fitted score and residual.
| Case | Study time \(x_i\) (hours) | Observed score \(y_i\) (points) |
|---|---|---|
| A | 2 | 55 |
| B | 3 | 56 |
| C | 4 | 64 |
For case A, the fitted score is \(42+5(2)=52\) points, so \(e_A=55-52=3\) points. For case B, the fitted score is \(42+5(3)=57\) points, so \(e_B=56-57=-1\) point. For case C, the fitted score is \(42+5(4)=62\) points, so \(e_C=64-62=2\) points.
| Case | Observed response \(y_i\) (points) | Fitted response \(\hat{y}_i\) (points) | Residual \(e_i=y_i-\hat{y}_i\) (points) |
|---|---|---|---|
| A | 55 | 52 | 3 |
| B | 56 | 57 | -1 |
| C | 64 | 62 | 2 |
Check each row by adding the fitted response and residual: \(52+3=55\), \(57+(-1)=56\), and \(62+2=64\). Each sum recovers that case’s observed score. The residuals are all expressed in points because quiz score is the response.
Details That Keep the Calculation Accurate
A residual depends on both the observed case and the fitted line. If the line changes, its fitted response at \(x_i\) may change, and so may the residual for the same observed case. For a fixed line, cases with different predictor values generally receive different fitted responses; do not use one case’s fitted response for another case merely because the observations appear near each other in a table.
Use the equation as given and retain enough precision during the calculation. If a fitted value is rounded before subtraction, the reported residual may differ slightly from the value calculated with unrounded coefficients. In hand calculations, show the fitted-value step and state any rounding used. Report enough digits to make the arithmetic transparent without implying more precision than the data or fitted equation supports.
The residual is an observed difference from a fitted value. It is not the same quantity as the response itself, and it is not the same as the fitted value. For example, if an observed response is 56 points and its fitted response is 57 points, the residual is \(-1\) point—not 56, 57, or 1 hour. The calculation uses the response values, so its units follow the response rather than the predictor.
Worked Example: Sales Residuals With Explicit Units
Suppose a hypothetical fitted line relates weekly advertising expenditure \(x\), measured in hundreds of dollars, to the number of orders \(y\):
For a week with \(x_i=15\), the observed number of orders is \(y_i=33\). Find the fitted response and residual, and check their relationship.
Substitute \(x_i=15\) into the fitted equation:
Now subtract the fitted response from the observed response:
The check gives \(30+3=33\) orders, the observed count. Although \(x_i\) is measured in hundreds of dollars, the residual is measured in orders, because it is a difference between two values of \(y\). The units of the predictor do not determine the residual’s units.
Common Mistakes and a Reliable Check
Most errors in residual calculations come from mixing up the two response values or their units. The following checks help catch those errors before interpreting or reporting a result.
- Reversing the subtraction. Calculating \(\hat{y}_i-y_i\) gives the negative of the defined residual. Use \(e_i=y_i-\hat{y}_i\).
- Subtracting the predictor. A residual is not \(y_i-x_i\). First calculate the fitted response from \(x_i\), then compare that response with \(y_i\).
- Using a fitted response as if it were observed. The fitted value comes from the equation; it does not replace the recorded \(y_i\) in the residual definition.
- Reporting predictor units. Residuals use response units. A residual for a commute-time response is in minutes, even when distance is the predictor.
- Pairing values from different cases. Calculate \(\hat{y}_i\) using \(x_i\), and subtract it from the observed response \(y_i\) for that same case.
A compact checking routine is to label the case, write down \(x_i\) and \(y_i\), calculate \(\hat{y}_i\) from the line, and then evaluate \(y_i-\hat{y}_i\). Finally, verify \(y_i=\hat{y}_i+e_i\) and report the response units. This checks the substitution, subtraction order, and interpretation of units without requiring any extra regression calculation.
Check Your Understanding
Use the definition and the observed-equals-fitted-plus-residual check for each question.
- A fitted line is \(\hat{y}=8+3x\). For a case with \(x_i=4\) and \(y_i=21\), calculate the fitted response and residual, including units if \(y\) is measured in metres.
- For a case, the observed response is 18 and the fitted response is 20. Write the residual calculation in the correct order and state its value.
- A fitted response is 46 points and the residual is \(-2\) points. What is the observed response?
- If the predictor is measured in kilometres and the response is measured in minutes, what are the units of a residual? Explain briefly.
- Explain why two cases with the same observed response need not have the same residual when they have different predictor values.