From Paired Observations to a Fitted Line
A regression equation is often reported as if it appeared all at once: \(\hat{y}=a+bx\). In practice, fitting the line from raw data involves several linked calculations. First, identify the paired measurements and their roles. Then calculate sample summaries, use the correlation and standard deviations to obtain the slope, and use the sample means to obtain the intercept. The result is a line that summarizes the linear relationship in the observed data.
This tutorial works through that calculation. The examples use small hypothetical data sets so each step can be checked by hand. A fitted line describes an association between the explanatory variable \(x\) and response variable \(y\); it does not, by itself, establish that changes in \(x\) cause changes in \(y\).
The Calculation Plan
A calculation from raw data is appropriate when every \(x\)-value is paired with the \(y\)-value measured on the same individual or unit, and both variables are quantitative. Before fitting a line, inspect the scatterplot. A linear summary is most useful when the overall pattern is reasonably straight and no observation stands out as an unusual point that could dominate the fit. Scatterplot diagnostics help assess these features; a calculation alone cannot establish that a line is a suitable summary.
For the numerical work, define the sample means and sample standard deviations as follows:
The sample correlation \(r\) summarizes the direction and strength of the linear association. One way to calculate it directly from the observations is
Once \(r\), \(s_x\), and \(s_y\) are known, the slope and intercept are
The slope has units of \(y\) per unit of \(x\), while the intercept has the units of \(y\). Report the fitted equation as \(\hat{y}=b_0+b_1x\), retaining enough precision during calculations to avoid unnecessary rounding error.
Why the Slope Formula Matches the Raw-Data Calculation
The slope formula using \(r\) can be connected directly to the centered raw-data sums. This identity is useful for checking a hand calculation; it does not require deriving the least-squares criterion.
Let \(S_{xx}=\sum_{i=1}^{n}(x_i-\bar{x})^2\), \(S_{yy}=\sum_{i=1}^{n}(y_i-\bar{y})^2\), and \(S_{xy}=\sum_{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y})\). By the definition of sample correlation, \(r=S_{xy}/\sqrt{S_{xx}S_{yy}}\). The sample standard deviations are \(s_x=\sqrt{S_{xx}/(n-1)}\) and \(s_y=\sqrt{S_{yy}/(n-1)}\). Therefore,
The factors involving \(S_{yy}\) and \(n-1\) cancel, leaving the centered-data slope calculation \(S_{xy}/S_{xx}\). This provides a useful independent check on the result. In the examples below, the formula involving \(r\), \(s_x\), and \(s_y\) will be used to fit the line.
Worked Example: Fitting a Line to Five Paired Scores
Suppose five hypothetical students have the following paired values, where \(x\) is hours spent on a practice set and \(y\) is score on a short assessment out of 10.
| Student | \(x\) (hours) | \(y\) (points) |
|---|---|---|
| A | 1 | 2 |
| B | 2 | 3 |
| C | 3 | 5 |
| D | 4 | 4 |
| E | 5 | 6 |
Check the data. The variables are quantitative and each hour value is paired with the score for the same student. The points follow a generally increasing, roughly linear pattern, with no point visibly separated from the rest in this small illustrative set. A line is a reasonable descriptive summary for these observations.
Calculate the means. The mean practice time is \(\bar{x}=(1+2+3+4+5)/5=3\) hours. The mean score is \(\bar{y}=(2+3+5+4+6)/5=4\) points.
Calculate the centered sums. The \(x\)-deviations are \(-2,-1,0,1,2\), and the \(y\)-deviations are \(-2,-1,1,0,2\). Thus \(S_{xx}=(-2)^2+(-1)^2+0^2+1^2+2^2=10\), \(S_{yy}=(-2)^2+(-1)^2+1^2+0^2+2^2=10\), and \(S_{xy}=(-2)(-2)+(-1)(-1)+(0)(1)+(1)(0)+(2)(2)=9\).
Calculate the sample standard deviations and correlation. With \(n-1=4\), \(s_x=\sqrt{10/4}=1.581\) hours and \(s_y=\sqrt{10/4}=1.581\) points. The correlation is \(r=9/\sqrt{10\cdot10}=0.900\).
Calculate the slope and intercept. The slope is \(b_1=r(s_y/s_x)=0.900(1.581/1.581)=0.900\) points per hour. The intercept is \(b_0=\bar{y}-b_1\bar{x}=4-(0.900)(3)=1.30\) points.
The fitted equation is \(\hat{y}=1.30+0.900x\), where \(x\) is hours of practice and \(\hat{y}\) is the predicted assessment score in points. The slope describes the fitted increase of 0.900 points per additional hour of practice within the context of these data. It does not prove that practice caused the scores to increase.
A Negative Association and a Units Check
The same calculation works when the relationship decreases as \(x\) increases. The sign of \(r\) determines the sign of the slope, because standard deviations are positive. It is also useful to check the units at each stage: the ratio \(s_y/s_x\) has units of response per explanatory unit, as a slope should.
Worked Example: A Decreasing Pattern
Consider five hypothetical measurements of \(x\), a setting on a device, and \(y\), the resulting output in units. The paired values are \((1,9),(2,8),(3,8),(4,5),(5,5)\). Assume a scatterplot shows a generally decreasing, roughly linear pattern without an observation separated from the others; the line will summarize this small set descriptively.
Means and deviations. The means are \(\bar{x}=(1+2+3+4+5)/5=3\) and \(\bar{y}=(9+8+8+5+5)/5=7\). The \(x\)-deviations are \(-2,-1,0,1,2\), and the \(y\)-deviations are \(2,1,1,-2,-2\).
Centered sums. Multiplying paired deviations gives \(S_{xy}=(-2)(2)+(-1)(1)+(0)(1)+(1)(-2)+(2)(-2)=-11\). Squaring deviations gives \(S_{xx}=10\) and \(S_{yy}=2^2+1^2+1^2+(-2)^2+(-2)^2=14\).
Standard deviations and correlation. Here \(s_x=\sqrt{10/4}=1.581\) units of \(x\), and \(s_y=\sqrt{14/4}=\sqrt{3.5}=1.871\) output units. The correlation is \(r=-11/\sqrt{10\cdot14}=-0.930\) to three significant figures.
Slope and intercept. The slope is \(b_1=(-0.930)(1.871/1.581)\), approximately \(-1.10\) output units per unit of \(x\). Using the unrounded values gives \(-1.10\) as well, since \(S_{xy}/S_{xx}=-11/10=-1.10\). The intercept is \(b_0=7-(-1.10)(3)=10.3\) output units.
The fitted equation is \(\hat{y}=10.3-1.10x\). The negative slope indicates that the fitted output decreases by 1.10 units for each one-unit increase in the device setting. The intercept is the line's predicted output at \(x=0\); whether that value is meaningful in practice depends on whether \(x=0\) is relevant to the setting, not merely on the arithmetic.
Reporting a Fitted Equation in Context
A numerical equation is incomplete unless the variables and units are clear. The next example uses nonconsecutive explanatory values to show that the method depends on the observations, not on having \(x=1,2,3,\ldots\). It also illustrates a case where the intercept is a mathematical part of the fitted line but may not describe an observed condition.
Worked Example: Fitting a Line With Measurement Units
Suppose a hypothetical set of five devices is tested at \(x=2,4,6,8,10\) hours of use, and their recorded output \(y\), in units, is \(10,13,15,19,18\), respectively. The pairs are \((2,10),(4,13),(6,15),(8,19),(10,18)\). Their scatterplot has an overall increasing, approximately linear pattern, with no isolated point dominating the pattern, so fitting a descriptive line is reasonable.
Means. The mean time is \(\bar{x}=(2+4+6+8+10)/5=6\) hours. The mean output is \(\bar{y}=(10+13+15+19+18)/5=15\) units.
Centered sums. The \(x\)-deviations are \(-4,-2,0,2,4\), and the \(y\)-deviations are \(-5,-2,0,4,3\). Therefore, \(S_{xx}=16+4+0+4+16=40\), \(S_{yy}=25+4+0+16+9=54\), and \(S_{xy}=20+4+0+8+12=44\).
Sample summaries. The sample standard deviations are \(s_x=\sqrt{40/4}=\sqrt{10}=3.162\) hours and \(s_y=\sqrt{54/4}=\sqrt{13.5}=3.674\) units. The correlation is \(r=44/\sqrt{40\cdot54}=44/\sqrt{2160}=0.947\).
Fitted coefficients. The slope is \(b_1=0.947(3.674/3.162)\), approximately \(1.10\) output units per hour. The centered-sum check gives \(S_{xy}/S_{xx}=44/40=1.10\). The intercept is \(b_0=15-(1.10)(6)=8.40\) output units.
The fitted equation is \(\hat{y}=8.40+1.10x\), with \(x\) measured in hours and \(\hat{y}\) in output units. The line predicts an increase of about 1.10 output units per additional hour over the observed range. Although the equation has an intercept of 8.40 units, the data only cover 2 to 10 hours; the intercept is not itself a measured output at zero hours.
Keeping the Calculation Reliable
A compact workflow reduces errors and makes the calculation easy to audit. Keep the paired values together, show how the sample summaries were obtained, and delay rounding until the coefficients are calculated. If software or a calculator supplies \(r\), \(s_x\), and \(s_y\), the same formulas apply; the raw-data calculations provide a way to understand and check those summaries.
State what \(x\) and \(y\) measure, identify their units, and verify that each pair belongs to the same individual or unit.
Use the scatterplot to check that a roughly linear summary is sensible and that no unusual point appears to dominate the relationship.
Find \(\bar{x}\), \(\bar{y}\), \(s_x\), \(s_y\), and \(r\), retaining adequate precision.
Use \(b_1=r(s_y/s_x)\), then \(b_0=\bar{y}-b_1\bar{x}\). Write \(\hat{y}=b_0+b_1x\) and interpret the slope with units and context.
The arithmetic for describing a sample does not require a probability model. If a later analysis uses regression to make inferential claims, additional assumptions and methods are needed; calculating a fitted line alone does not supply them. In particular, a high absolute correlation does not guarantee that a line is appropriate if the scatterplot is curved or contains an influential unusual observation.
Common Mistakes
- Pairing values incorrectly. Sorting the \(x\)-values and \(y\)-values separately can break the original pairings and change the correlation and fitted line. Keep each \(x_i\) with its corresponding \(y_i\).
- Using the wrong standard deviation ratio. The slope is \(r(s_y/s_x)\), not \(r(s_x/s_y)\). Its units must be units of \(y\) per unit of \(x\).
- Using the sample mean for the intercept incorrectly. The intercept is \(b_0=\bar{y}-b_1\bar{x}\), not just \(\bar{y}\) or \(\bar{x}\).
- Rounding too early. Rounding \(r\), \(s_x\), or \(s_y\) aggressively before calculating the slope can introduce avoidable differences. Keep extra digits until the final reported coefficients.
- Overinterpreting the equation. A fitted line describes an observed linear association. It does not establish causation, and its predictions may not be meaningful beyond the range or context of the observed data.
Check Your Understanding
Try these questions before moving on.
- For five paired observations, \(\bar{x}=4\), \(\bar{y}=12\), \(s_x=2\), \(s_y=3\), and \(r=0.80\). Calculate the fitted slope and intercept.
- Why must the \(x\)- and \(y\)-values remain paired when calculating correlation from raw data?
- If \(r\) is negative and both sample standard deviations are positive, what must be the sign of the fitted slope? Explain using the slope formula.
- A fitted line predicts a response in kilograms from an explanatory variable measured in centimeters. What are the units of the slope?
- What does an intercept represent mathematically, and why might it lack a practical interpretation?