Why Small Calculation Errors Matter
A simple regression calculation can look convincing and still be wrong because one value was put in the wrong place, a subtraction was reversed, or a coefficient was given the wrong meaning. These errors can change the sign of a residual, produce a prediction with impossible units, or lead to an incorrect description of how two variables are associated.
The most reliable checks begin with the roles of the variables. In a simple regression, \(x\) is the predictor and \(y\) is the response. A fitted line uses the predictor to calculate a predicted response. Keeping those roles clear makes the arithmetic easier to audit.
The Calculation Setup
The equation is a calculation rule: insert a predictor value for \(x\), then calculate the corresponding fitted response. The line by itself does not say that changing \(x\) causes a change in \(y\). Nor do the arithmetic checks in this tutorial require assumptions about a probability distribution for the errors. They require a correctly identified predictor and response, a supplied fitted line, and values recorded in consistent units.
Before calculating, write down what each quantity represents and its units. The intercept has the units of the response; the slope has response units per predictor unit; and a residual has the units of the response. These facts provide useful checks on both substitutions and interpretations.
Worked Checks for Common Errors
Worked Example: Reversing the Residual Sign
Suppose a hypothetical fitted line predicts trail-completion time \(y\), in minutes, from trail distance \(x\), in kilometres:
For one hiker, the distance is \(8\) km and the observed completion time is \(36\) minutes. Calculate the fitted time and residual, then interpret the sign.
Fitted value. Substitute the predictor \(x=8\):
Residual. Subtract the fitted time from the observed time:
The hiker's observed time is 4 minutes below the fitted time. A common mistake is to calculate \(40-36=4\) and call that the residual. That subtraction gives the difference in the opposite direction, not the defined residual. The sign follows from the order \(y-\hat{y}\): observed minus fitted. Here the negative sign correctly indicates that the line overpredicted this hiker's time.
Worked Example: Swapping the Slope and Intercept
In a hypothetical class, a fitted line predicts quiz score \(y\), in points, from study time \(x\), in hours:
Identify the intercept and slope, give their units, and calculate the predicted score for a student who studied \(6\) hours.
The constant term is the intercept, so \(b_0=42\) points. The coefficient multiplying \(x\) is the slope, so \(b_1=5.2\) points per hour. The predicted score is:
The intercept describes the line's predicted score at \(x=0\) hours: \(42\) points. The slope describes the fitted change in score for a one-hour increase in study time: \(5.2\) points, on average according to this line. Saying that the slope is 42 points per hour and the intercept is 5.2 points swaps their roles and gives the wrong units. The fitted association does not, by itself, establish that additional study time causes a score increase.
Worked Example: Substituting the Response Instead of the Predictor
Suppose a hypothetical fitted line predicts response \(y\) from predictor \(x\):
For an observation with predictor value \(x=3\) and observed response \(y=11\), find the fitted value and residual. Both variables are measured in their respective recorded units.
The line uses the predictor as its input, so substitute \(x=3\):
Then use the observed response to calculate the residual:
A mistaken calculation might put the observed response \(y=11\) into the line's \(x\) position, giving \(4+2(11)=26\). That is not the fitted response for this observation: the line takes an \(x\)-value as input, and \(11\) is the observed \(y\)-value. The fitted value is \(10\), and the residual is \(1\) response unit.
Worked Example: Restoring Missing Units
A hypothetical fitted line predicts mass \(y\), in kilograms, from height \(x\), in centimetres:
Calculate the predicted mass at \(x=170\) cm and state the units of the intercept and slope.
Substitute the height in centimetres:
The intercept is \(48\) kg, and the slope is \(0.32\) kg per centimetre. The multiplication \(0.32\text{ kg/cm}\times170\text{ cm}\) gives \(54.4\) kg, so it can be added to the intercept, also measured in kilograms. Omitting units hides this check. Calling the slope simply “0.32 kilograms” is incomplete; the value describes fitted mass change per centimetre of height.
Why the Residual Sign Works
The residual sign is not a matter of convention chosen separately for each data point. It follows from the definition and the relative positions of the observation and fitted value on the response scale. If the observed response is greater than the fitted response, then \(y_i-\hat{y}_i>0\). If the observed response is smaller, then \(y_i-\hat{y}_i<0\).
For example, if the fitted value is \(40\) minutes and the observed value is \(36\) minutes, then \(36-40=-4\) minutes. The negative residual means the observation falls below the fitted value. Reversing the subtraction changes the sign and therefore reverses that interpretation. It does not make the observation itself higher or lower; it only reports the difference in the wrong direction.
A Short Audit Before Reporting a Result
A calculation can be checked in a few deliberate steps. First name which quantity is the predictor and which is the response. Next identify what the fitted equation returns, then check the substitution, arithmetic, and units. Finally, compare the sign and size of the answer with the values given.
Write “predictor \(x\)” and “response \(y\)” in the context of the problem. Do not infer their roles from which value happens to be larger.
The value multiplying \(x\) is the slope; the constant term is the intercept. The line takes \(x\) as input and returns \(\hat{y}\).
For a fitted value, substitute \(x\). For a residual, calculate \(y-\hat{y}\), using the observed response for \(y\).
The fitted value and residual use response units, the intercept uses response units, and the slope uses response units per predictor unit.
This audit also catches answers that are arithmetically neat but conceptually wrong. A residual cannot be interpreted as a predictor difference, and the slope is not a predicted response at a particular \(x\)-value. If an answer's units do not match the quantity being reported, revisit the setup before trusting the arithmetic.
Common Mistakes and Corrections
| Mistake | Why it is wrong | Correction |
|---|---|---|
| Calculating \(\hat{y}-y\) as the residual | It reverses the defined subtraction and flips the sign. | Use \(e=y-\hat{y}\): observed response minus fitted response. |
| Calling the constant term the slope | The constant is the intercept; it does not multiply \(x\). | Identify the coefficient of \(x\) as the slope and the constant as the intercept. |
| Inserting an observed \(y\)-value where \(x\) belongs | The fitted line uses the predictor as its input. | Substitute the observed predictor to find \(\hat{y}\); then use observed \(y\) to find the residual. |
| Reporting a slope without units | A bare number hides the scale of the change being described. | State response units per predictor unit, such as points per hour. |
| Treating a fitted association as proof of cause | A regression calculation alone does not establish causation. | Describe the fitted relationship without making an unsupported causal claim. |
A useful habit is to write the meaning beside the answer, not just the number. “The residual is \(-4\) minutes” is more informative than “the residual is \(-4\),” because the units confirm that it is a difference in observed and fitted response. Likewise, “\(5.2\) points per hour” makes clear what the slope measures and which variable is changing.
Check Your Understanding
Answer each question by identifying the quantity involved and showing the relevant calculation or reasoning.
- A fitted line predicts response in dollars from a predictor measured in hours. What are the units of the slope, intercept, and residual?
- For one observation, \(y=18\) and \(\hat{y}=21\). Calculate the residual and explain what its sign says.
- For \(\hat{y}=7+4x\), identify the intercept and slope. What predictor value belongs in the equation when calculating a fitted response?
- Suppose an observation has \(x=5\) and \(y=27\), and the fitted line is \(\hat{y}=3+4x\). Calculate the fitted value and residual.
- Why is \(y-\hat{y}\) the correct residual calculation rather than \(\hat{y}-y\)?