Least Squares as a Squared-Error Criterion

From Observations to a Fitted Line

In simple linear regression, a line summarizes how a response $Y$ varies with a predictor $X$. Given $n$ observed pairs $(x_i,y_i)$, a candidate line assigns a predicted response to each observed predictor value. Least squares provides a precise rule for choosing among candidate lines: it selects the intercept and slope that make the squared discrepancies between observed and fitted responses as small as possible in total.

The criterion is a calculation on the observed data. It does not, by itself, require that the responses follow a particular probability distribution. A probability model may be added when making inferential claims, but it is distinct from the definition of the least-squares fit.

Definition: For observed data $(x_i,y_i)$, $i=1,\ldots,n$, a candidate line with intercept $a$ and slope $b$ gives the fitted value $\hat{y}_i=a+bx_i$ at predictor value $x_i$. The residual for observation $i$ is $e_i=y_i-\hat{y}_i$, the observed response minus its fitted value. The sum of squared residuals for the candidate line is $$ \operatorname{SSE}(a,b)=\sum_{i=1}^{n} e_i^2 =\sum_{i=1}^{n}(y_i-a-bx_i)^2. $$ The least-squares intercept and slope, denoted $\hat{\beta}_0$ and $\hat{\beta}_1$, are values of $a$ and $b$ that minimize this sum over all candidate lines.

A fitted value is the value assigned by a particular line at an observed $x_i$; it is not necessarily the observed response. A residual records the difference for that observation. The sign convention matters: a positive residual means the observed response is above the fitted value, while a negative residual means it is below.

The objective uses the residuals for all observations together. It is not enough for a line to fit one point well, or to have small discrepancies for most observations if another discrepancy is very large. Each squared residual contributes a nonnegative amount to the total, and the line with the smaller total has the smaller least-squares criterion.

Why Square the Residuals?

Adding signed residuals would allow positive and negative discrepancies to cancel. For instance, residuals of $4$ and $-4$ sum to zero even though neither observation is fitted exactly. Squaring avoids that cancellation: both contribute $16$ to the sum of squares. Squaring also gives a large discrepancy substantially more influence than a small one; a residual of $6$ contributes $36$, while a residual of $2$ contributes $4$.

The least-squares criterion is therefore a specific way of measuring total discrepancy, not a claim that every observation is close to the line or that the line explains every feature of the data. The chosen line is the one with the smallest sum of squared residuals among all lines considered. Whether that line is suitable for a particular purpose still depends on the context and on the regression assumptions relevant to that purpose.

Worked Example: Comparing Candidate Lines for a Claim-Review Sample

A hypothetical insurer records review hours $x$ and completed files per shift $y$ for three shifts. The observations are $(0,2)$, $(1,3)$, and $(2,5)$. Compare the candidate lines $L_1:\hat{y}=2+x$ and $L_2:\hat{y}=1+2x$ using the least-squares criterion. The data are treated as fixed observations; no probability distribution, interest rate, or mortality assumption is used in this calculation.

Method. For each candidate, calculate the fitted value at each observed $x$, subtract it from the observed $y$ to obtain the residual, square each residual, and add the squares.

For $L_1$, the fitted values at $x=0,1,2$ are $2,3,4$. The residuals are $2-2=0$, $3-3=0$, and $5-4=1$. Thus $$ \operatorname{SSE}(L_1)=0^2+0^2+1^2=1. $$

For $L_2$, the fitted values are $1,3,5$. The residuals are $2-1=1$, $3-3=0$, and $5-5=0$. Thus $$ \operatorname{SSE}(L_2)=1^2+0^2+0^2=1. $$

The two candidate lines tie under the least-squares criterion: each has a sum of squared residuals equal to $1$. Neither is better by this criterion alone. This comparison does not establish that either is the least-squares line over all possible intercepts and slopes; it only compares these two candidates.

Check. The residuals for each line sum to $1$, but their squared sums are also $1$. The reported totals agree with the individual residual calculations. The common mistake avoided here is to assume that the line with a different-looking pattern of residuals must have a smaller total; the squared sums decide.

Minimization Means Choosing the Smallest Total

For every candidate intercept and slope, the data determine one value of $\operatorname{SSE}(a,b)$. Least squares chooses the pair that gives the smallest value across the full set of lines. The minimization is over both parameters at once: changing the intercept shifts all fitted values, while changing the slope changes them according to the predictor values.

A short one-parameter calculation shows how minimizing a sum of squares works. Suppose the slope $b$ is held fixed, and only the intercept $a$ may vary. Define $z_i=y_i-bx_i$. Then each residual is $z_i-a$, so

\sum_{i=1}^{n}(z_i-a)^2 = \sum_{i=1}^{n}(z_i-\bar{z})^2+n(a-\bar{z})^2, \qquad \bar{z}=\frac{1}{n}\sum_{i=1}^{n}z_i.

To verify the identity, write $z_i-a=(z_i-\bar{z})+(\bar{z}-a)$, square, and sum. The cross-term is $2(\bar{z}-a)\sum_i(z_i-\bar{z})=0$, because deviations from their mean sum to zero. The first term on the right does not depend on $a$; the second is nonnegative and is smallest when $a=\bar{z}$. Thus, for a fixed slope, the intercept minimizing the squared residuals is the mean of the adjusted responses $y_i-bx_i$.

When both intercept and slope are free, the least-squares choice must minimize the criterion over both. The next stage of the calculation derives the equations that characterize that joint minimum; here, the key idea is the objective being minimized and how to evaluate it.

Worked Example: Minimizing Over the Intercept When the Slope Is Fixed

A hypothetical service center records cases handled $x$ and minutes of follow-up work $y$ on three shifts: $(1,4)$, $(2,5)$, and $(3,8)$. Suppose the slope is fixed at $b=2$. Find the intercept that minimizes the sum of squared residuals, and compare its criterion value with the candidate intercept $a=1$. No distributional, interest-rate, or mortality assumptions are needed for this data calculation.

Method. With $b=2$, calculate $z_i=y_i-2x_i$. The adjusted responses are $4-2(1)=2$, $5-2(2)=1$, and $8-2(3)=2$. Their mean is $$ \bar{z}=\frac{2+1+2}{3}=\frac{5}{3}. $$

By the fixed-slope minimization result, the minimizing intercept is $a=5/3$. The fitted values are $5/3+2=11/3$, $5/3+4=17/3$, and $5/3+6=23/3$. The residuals are $$ 4-\frac{11}{3}=\frac{1}{3},\qquad 5-\frac{17}{3}=-\frac{2}{3},\qquad 8-\frac{23}{3}=\frac{1}{3}. $$

The sum of squared residuals is $$ \operatorname{SSE}\left(\frac{5}{3},2\right) =\left(\frac{1}{3}\right)^2+\left(-\frac{2}{3}\right)^2+\left(\frac{1}{3}\right)^2 =\frac{1+4+1}{9}=\frac{2}{3}. $$

For the comparison intercept $a=1$, the fitted values are $3,5,7$, so the residuals are $1,0,1$. Therefore $\operatorname{SSE}(1,2)=1^2+0^2+1^2=2$, which exceeds $2/3$. The minimizing intercept is $5/3$, not $1$, when the slope is fixed at $2$.

Check. The residuals at the minimizing intercept sum to $1/3-2/3+1/3=0$, and the directly calculated sum of squares is $2/3$. The common mistake avoided is to select an intercept by looking at one observation rather than minimizing the total across all three.

Large Residuals and the Squared-Error Penalty

Because the criterion squares each residual, one observation far from a candidate line can have a substantial effect on its total. This is a property of the criterion, not a computational error. It also explains why the line with the smallest sum of squared residuals need not have the smallest sum of absolute residuals.

Worked Example: A Large Discrepancy Changes the Candidate Ranking

A hypothetical team records inspections completed $x$ and processing units $y$ on four shifts: $(0,1)$, $(1,2)$, $(2,3)$, and $(3,10)$. Compare $L_A:\hat{y}=1+x$, $L_B:\hat{y}=2x$, and $L_C:\hat{y}=-1+3x$ by their sums of squared residuals. Treat these as observed values without imposing a response distribution, interest rate, or mortality assumption.

Method. Calculate fitted values and residuals in the order of the four observations, then square and sum.

For $L_A$, the fitted values are $1,2,3,4$, giving residuals $0,0,0,6$. Hence $\operatorname{SSE}(L_A)=0+0+0+36=36$.

For $L_B$, the fitted values are $0,2,4,6$, giving residuals $1,0,-1,4$. Hence $\operatorname{SSE}(L_B)=1+0+1+16=18$.

For $L_C$, the fitted values are $-1,2,5,8$, giving residuals $2,0,-2,2$. Hence $\operatorname{SSE}(L_C)=4+0+4+4=12$. Among these three candidates, $L_C$ has the smallest sum of squared residuals.

As a check on the effect of squaring, each candidate has a sum of absolute residuals equal to $6$: for $L_A$, $0+0+0+6=6$; for $L_B$, $1+0+1+4=6$; and for $L_C$, $2+0+2+2=6$. The squared-error criterion nevertheless ranks $L_C$ best among the three. The common mistake avoided is to treat the candidate with the smallest total absolute discrepancy as automatically best by least squares.

What the Criterion Does—and Does Not—Say

Least squares defines a fitting rule, not a causal conclusion. A line can minimize squared residuals for observed data without showing that changing $X$ causes a change in $Y$. The interpretation of a fitted line depends on the context, the way the data were collected, and the assumptions used for the analysis.

The criterion also does not require normally distributed errors simply to calculate fitted values, residuals, or a sum of squares. If a statistical model is used for inference, its assumptions must be stated for that inference. For example, a model might write $Y_i=\beta_0+\beta_1x_i+\varepsilon_i$, where $\varepsilon_i$ is an unobserved error. The residual $e_i=y_i-\hat{y}_i$, in contrast, is calculated from observed data and an estimated line. It is not the unobserved error itself.

Key takeaway: At each observed predictor value, a candidate line produces a fitted value; subtracting that value from the observation gives a residual. Least squares chooses the intercept and slope that minimize the sum of the squared residuals across the data. Always calculate the stated criterion, and distinguish a best candidate in a shortlist from a minimum over all possible lines.

Check Your Understanding

For each problem, compare the specified candidate lines using the least-squares criterion. The final answer is given for checking.

A hypothetical analyst records $(x,y)=(0,2),(1,4),(2,7)$. Compare $L_1:\hat{y}=1+2x$ and $L_2:\hat{y}=2+x$. Which has the smaller sum of squared residuals? (Answer: $L_1$, with $\operatorname{SSE}=6$, versus $10$ for $L_2$.)
For observations $(0,3)$ and $(2,7)$, compare $L_1:\hat{y}=2+2x$ and $L_2:\hat{y}=3+x$. Give the fitted values and identify the line with the smaller sum of squared residuals. (Answer: $L_1$ has fitted values $2,6$ and $\operatorname{SSE}=2$; $L_2$ has fitted values $3,5$ and $\operatorname{SSE}=4$, so $L_1$ is better by this criterion.)
For $(1,5),(2,8),(3,9)$, compare $L_1:\hat{y}=2+2x$ and $L_2:\hat{y}=1+2.5x$. Calculate the residuals for each and select the lower sum of squares. (Answer: $L_1$ has residuals $1,2,1$ and $\operatorname{SSE}=6$; $L_2$ has residuals $1.5,2,0.5$ and $\operatorname{SSE}=6.5$, so $L_1$ is better.)
For observations $(1,4)$ and $(2,4)$, use the candidate line $\hat{y}=2+x$. Find both fitted values, both residuals, and the sum of squared residuals. (Answer: fitted values $3,4$; residuals $1,0$; $\operatorname{SSE}=1$.)
A candidate line has residuals $3,-1,-2$. Compare the sum of signed residuals with the sum of squared residuals, and explain whether cancellation occurs in the signed sum. (Answer: the signed sum is $3-1-2=0$, while the sum of squares is $9+1+4=14$; the signed sum cancels even though the observations are not all fitted exactly.)

What You'll Learn