Why the Direction of Prediction Matters
A regression line summarizes how a response variable tends to vary with an explanatory variable. Its direction is part of its definition: the line is fitted to predict one variable from the other. If the question changes from predicting a response from an explanatory measurement to predicting that measurement from the response, the least-squares calculation changes too.
This matters whenever paired measurements could be used for either prediction. For example, a researcher might use hours of study to predict a test score, or use a test score to predict hours of study. These are different questions, even though they use the same pairs of observations. The two fitted lines generally are not obtained by simply reversing the equation of one line.
The word “vertical” describes deviations in the response variable, not necessarily vertical distance on a page. When predicting \(y\) from \(x\), each observed point is compared with the line at that point’s \(x\)-value, and the difference in \(y\)-units is squared. When predicting \(x\) from \(y\), the differences are measured in \(x\)-units instead.
Two Slopes From the Same Paired Data
For paired observations \((x_i,y_i)\), the least-squares slope for predicting \(y\) from \(x\) is \(b_1=r\frac{s_y}{s_x}\), where \(r\) is the sample correlation and \(s_x\) and \(s_y\) are the sample standard deviations of the two variables. The slope for predicting \(x\) from \(y\) is \(a_1=r\frac{s_x}{s_y}\). Each slope has its own units: the first is \(y\)-units per \(x\)-unit, and the second is \(x\)-units per \(y\)-unit.
These formulas also show why reversing the variables does not usually give the reciprocal slope. The reciprocal of the slope for predicting \(y\) from \(x\) is \(\frac{s_x}{r s_y}\), when \(r\ne 0\). The reverse-regression slope is instead \(r\frac{s_x}{s_y}\). The two are generally different because correlation is generally not \(1\) or \(-1\).
Worked Example: Fitting Both Lines to the Same Data
Suppose, hypothetically, that five students’ study times and test scores were recorded as follows. The scores are on a scale for which higher values represent higher scores.
| Study time, \(x\) (hours) | Test score, \(y\) (points) |
|---|---|
| 1 | 1 |
| 2 | 2 |
| 3 | 2 |
| 4 | 4 |
| 5 | 6 |
Each row is a paired observation: both measurements come from the same student. For this descriptive calculation, the pairs must be recorded together and \(x\) must vary. A straight-line summary is being used as a useful description of the association; no claim that study time causes a score is being made.
The means are \(\bar{x}=3\) hours and \(\bar{y}=3\) points. The sums of squared deviations and the sum of cross-products are
For predicting score from study time, the least-squares slope is \(b_1=S_{xy}/S_{xx}\), so
The line passes through the point of the sample means, so its intercept is \(b_0=\bar{y}-b_1\bar{x}=3-(1.2)(3)=-0.6\) points. Thus the fitted line for predicting score is
For predicting study time from score, the slope uses \(S_{yy}\) in the denominator:
Its intercept is \(a_0=\bar{x}-a_1\bar{y}=3-(0.75)(3)=0.75\) hours, giving
The slopes are 1.2 points per hour and 0.75 hours per point. They are not reciprocals: the reciprocal of 1.2 is about 0.833 hours per point, not 0.75.
Why Reversing the Equation Is Not Enough
Rearranging the first line from the example to make \(x\) the subject gives \(x=(y+0.6)/1.2\), or \(x=0.5+0.833y\). This is the algebraic inverse of the fitted score-on-study-time line. It is not the least-squares line for predicting study time from score, which was \(\hat{x}=0.75+0.75y\). Algebraic inversion answers a different question: it finds the \(x\)-value that lies on the first fitted line at a specified \(y\)-value. The reverse regression instead minimizes squared errors in \(x\).
The relationship between the two slopes makes the distinction precise. Define \(S_{xx}=\sum(x_i-\bar{x})^2\), \(S_{yy}=\sum(y_i-\bar{y})^2\), and \(S_{xy}=\sum(x_i-\bar{x})(y_i-\bar{y})\). When both variables vary, the two least-squares slopes are \(b_1=S_{xy}/S_{xx}\) and \(a_1=S_{xy}/S_{yy}\). The sample correlation is \(r=S_{xy}/\sqrt{S_{xx}S_{yy}}\). Therefore,
This is a short derivation of the product rule for the two slopes. Since \(r^2\) is between 0 and 1, the product of the slopes is generally less than 1. In the example, the product is \((1.2)(0.75)=0.9\). The correlation is \(r=12/\sqrt{(10)(16)}=0.9487\), so \(r^2\approx0.9\), as the rule predicts.
If \(r\ne0\), the reverse-regression slope would be the reciprocal of the original slope only if \(r^2=1\), which occurs when the paired observations fall exactly on a straight line. If the points do not fall exactly on a line, the two fitted lines are generally distinct. When \(r=0\), both slopes are zero; one line is horizontal in the \(y\)-on-\(x\) direction and the other is horizontal in the \(x\)-on-\(y\) direction. They are still not obtained by inverting one another.
Worked Example: Comparing Predictions in the Two Directions
Use the two fitted lines from the hypothetical student data: \(\hat{y}=-0.6+1.2x\) and \(\hat{x}=0.75+0.75y\). Suppose the question is to predict a score for a student who studied for 4 hours. Substituting \(x=4\) in the score-on-study-time line gives
This prediction is in points because the response is test score. The reverse line answers a different question. If a score of 4 points is given and the aim is to predict study time, substitute \(y=4\):
By contrast, algebraically inverting the first line at a score of 4 gives \(x=(4+0.6)/1.2=3.833\) hours. That value is not the least-squares prediction of study time from score. The difference is not an arithmetic error; it comes from minimizing squared deviations in different variables.
Both predictions are summaries of the observed association, not proof that changing study time would change a score. Also, a fitted value is an estimate rather than a guarantee that an individual student will have exactly that score or study time.
When the Lines Are Exact Inverses
The special case of a perfect straight-line relationship helps explain the general rule. If every paired point lies exactly on a line with nonzero slope, then the regression of \(y\) on \(x\) reproduces every observed \(y\)-value without error. Its correlation has magnitude 1, so \(r^2=1\). The product rule then says the two slopes multiply to 1, and the lines pass through the same mean point. In that case, the reverse line is exactly the algebraic inverse of the first.
Worked Example: A Perfect Linear Relationship
Consider hypothetical paired measurements \((x,y)\) of \((1,2),(2,4),(3,6),(4,8)\). Here \(y=2x\) for every pair. The means are \(\bar{x}=2.5\) and \(\bar{y}=5\). The sums are \(S_{xx}=5\), \(S_{yy}=20\), and \(S_{xy}=10\). Thus the slope for predicting \(y\) from \(x\) is
The first line is \(\hat{y}=2x\). For predicting \(x\) from \(y\), the slope is
The reverse line is \(\hat{x}=0.5y\), exactly the algebraic inverse of \(\hat{y}=2x\). Here \(b_1a_1=(2)(0.5)=1\), consistent with \(r^2=1\). This exact agreement occurs because all four points lie on the same straight line; it is not the usual situation when paired data have scatter around a line.
Common Mistakes and Why Direction Matters
A frequent mistake is to fit a line for predicting \(y\) from \(x\), rearrange its equation, and call the result the regression line for predicting \(x\) from \(y\). That treats the original line as if it minimized errors in both directions. Least squares does not do that: each regression minimizes squared differences in its response variable, so switching responses changes the calculation.
A second mistake is to interpret a slope without naming its direction or units. A slope of 1.2 points per hour describes the fitted change in predicted score for an additional hour of study. A slope of 0.75 hours per point describes the fitted change in predicted study time for an additional score point. These are different descriptions of the same paired data, not interchangeable causal effects.
The direction also matters when selecting a line for a practical prediction. Choose the response variable to match the quantity the question asks you to predict, and use the regression fitted for that response. Do not switch the explanatory and response variables just because the algebra of the first equation seems convenient. Regression describes patterns in the observed data; by itself, it does not establish that one variable causes the other.
Check Your Understanding
Answer each question using the distinction between predicting \(y\) from \(x\) and predicting \(x\) from \(y\).
- What quantity is minimized when fitting a least-squares line to predict \(y\) from \(x\)?
- If the slope for predicting \(y\) from \(x\) is 0.8, must the reverse-regression slope be 1.25? Explain.
- For the paired data in the first worked example, which fitted line should be used to predict study time from a test score?
- What does the product of the two regression slopes equal, and what does that imply when the data do not lie exactly on a line?
- Under what condition are the two regression lines exact inverses?