Deriving the Least-Squares Intercept

Why the Intercept Depends on Both Sample Means and Slope

In simple linear regression, the intercept is the fitted value of the response when the predictor is zero. Its estimate is not generally the average response by itself. The fitted slope also matters: if the predictor values are not centered at zero, changing the slope changes the height the fitted line needs to have at $x=0$ in order to fit the data overall.

The least-squares intercept is obtained by minimizing the sum of squared residuals. Differentiating that objective with respect to the intercept gives a simple equation: at the minimizing line, the residuals sum to zero. Solving this equation expresses the intercept estimate in terms of the sample means and the estimated slope.

Definition: For observed pairs $(x_i,y_i)$, $i=1,\ldots,n$, the simple linear least-squares line is $\hat{y}_i=\hat{\beta}_0+\hat{\beta}_1x_i$. The least-squares estimates $\hat{\beta}_0$ and $\hat{\beta}_1$ are values that minimize the sum of squared residuals $$ \operatorname{SSE}(\hat{\beta}_0,\hat{\beta}_1) =\sum_{i=1}^{n}\left[y_i-(\hat{\beta}_0+\hat{\beta}_1x_i)\right]^2. $$ Here, $\hat{\beta}_0$ is the estimated intercept and $\hat{\beta}_1$ is the estimated slope.

This calculation is an algebraic property of least squares. It does not require normal errors, constant error variance, or any other probabilistic assumption. Those conditions matter for other questions about a regression model, but they are not needed to derive the minimizing intercept. For a uniquely determined joint least-squares line, the observed predictor values must vary; if every $x_i$ is the same, infinitely many intercept-slope pairs can give the same fitted value at that one predictor value.

Deriving the Minimizing Intercept

First hold the slope fixed and choose the intercept that minimizes the sum of squared residuals. This isolates the intercept part of the minimization. At the joint least-squares solution, the same equation must hold with the minimizing slope substituted into it.

Write the objective.
For a fixed slope value $b_1$, define $Q(b_0)=\sum_{i=1}^{n}[y_i-(b_0+b_1x_i)]^2$, where $b_0$ is the intercept being varied.

Differentiate with respect to the intercept.
Using the chain rule gives $\frac{dQ}{db_0}=-2\sum_{i=1}^{n}[y_i-(b_0+b_1x_i)]$.

Set the derivative to zero.
At a minimum, $\sum_{i=1}^{n}[y_i-(b_0+b_1x_i)]=0$. Rearranging gives $\sum_{i=1}^{n}y_i=nb_0+b_1\sum_{i=1}^{n}x_i$.

Solve for the intercept and use the estimated slope.
Divide by $n$, and then set $b_1=\hat{\beta}_1$. The resulting intercept is $\hat{\beta}_0=\bar{y}-\hat{\beta}_1\bar{x}$.

\hat{\beta}_0=\bar{y}-\hat{\beta}_1\bar{x}, \qquad \bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_i, \qquad \bar{y}=\frac{1}{n}\sum_{i=1}^{n}y_i.

The minimizing condition is a minimum, not a maximum: for fixed $b_1$, $Q(b_0)$ is a quadratic in $b_0$ whose coefficient on $b_0^2$ is $n>0$. Its second derivative is $2n>0$, so the stationary point is the unique minimum in the intercept direction.

What the Equation Tells Us

The equation $\hat{\beta}_0=\bar{y}-\hat{\beta}_1\bar{x}$ makes the dependence explicit. The intercept equals the response mean only when the estimated slope is zero or the predictor mean is zero. In general, the slope term $\hat{\beta}_1\bar{x}$ adjusts the response mean back to the fitted line’s value at $x=0$.

An equivalent form is $\bar{y}=\hat{\beta}_0+\hat{\beta}_1\bar{x}$. This says that the fitted line passes through the point $(\bar{x},\bar{y})$, called the sample centroid. It does not say that the intercept is the centroid’s response coordinate: the intercept is the line’s fitted response at $x=0$, whereas $\bar{y}$ is the observed response average at the sample’s average predictor value.

Key result: Once the least-squares slope is known, the minimizing intercept is determined by the two sample means. The line passes through $(\bar{x},\bar{y})$, but its intercept is generally not $\bar{y}$.

The slope is not derived here. It may be supplied by a regression fit or calculated separately. This separation is useful: regardless of how the minimizing slope was obtained, the intercept must satisfy the equation above whenever the joint least-squares fit exists.

Worked Example: Calculating an Intercept From a Fitted Slope

A hypothetical portfolio study records each policy’s annual exposure $x$, measured in policy-years, and annual claim count $y$. For three policies, the observations are $(1,4)$, $(2,6)$, and $(3,8)$. A least-squares fit reports slope $\hat{\beta}_1=2$ claims per policy-year. Find the intercept and check the result using the fitted residuals.

Method. Use the intercept minimizing equation with the reported slope. The slope is given, so there is no need to recalculate it.

Step 1: Find the sample means. The predictor mean is $\bar{x}=(1+2+3)/3=6/3=2$ policy-years. The response mean is $\bar{y}=(4+6+8)/3=18/3=6$ claims.

Step 2: Calculate the intercept. $\hat{\beta}_0=\bar{y}-\hat{\beta}_1\bar{x}=6-(2)(2)=2$ claims. The fitted line is $\hat{y}=2+2x$.

Step 3: Check the fit. At $x=1,2,3$, the fitted values are $4,6,8$. The residuals, observed minus fitted, are $4-4=0$, $6-6=0$, and $8-8=0$; their sum is $0$, as required by the intercept minimizing equation.

Interpretation and trap avoided. The intercept is $2$, not the response mean $6$. Setting the intercept equal to $\bar{y}$ would ignore that the fitted slope contributes $2\bar{x}=4$ claims at the sample mean predictor value.

Worked Example: Using Means From a Regression Output

In a hypothetical study of repair work, a technician records job duration $x$ in hours and labor cost $y$ in tens of dollars. For three jobs, the data are $(2,10)$, $(4,13)$, and $(6,18)$. A least-squares output gives $\hat{\beta}_1=2$ tens of dollars per hour. Find the intercept, write the fitted line, and verify the sum-of-residuals condition.

Step 1: Calculate the means. The predictor mean is $\bar{x}=(2+4+6)/3=12/3=4$ hours. The response mean is $\bar{y}=(10+13+18)/3=41/3$ tens of dollars.

Step 2: Apply the minimizing equation. $\hat{\beta}_0=41/3-(2)(4)=41/3-24/3=17/3$ tens of dollars. Thus the fitted line is $\hat{y}=17/3+2x$.

Step 3: Calculate fitted values and residuals. At $x=2$, $\hat{y}=17/3+4=29/3$, so the residual is $10-29/3=1/3$. At $x=4$, $\hat{y}=17/3+8=41/3$, so the residual is $13-41/3=-2/3$. At $x=6$, $\hat{y}=17/3+12=53/3$, so the residual is $18-53/3=1/3$.

Step 4: Check. The residual sum is $1/3-2/3+1/3=0$. Also, at $\bar{x}=4$, the fitted value is $17/3+8=41/3=\bar{y}$, confirming that the line passes through the sample centroid.

Common mistake avoided. Using $\bar{y}=41/3$ as the intercept would make the fitted value at $\bar{x}=4$ equal to $41/3+8$, not $41/3$. The response mean is the fitted value at the predictor mean, not the fitted value at zero.

Worked Example: Intercept in a Hypothetical Risk Model

An insurer studies annual claim costs for a hypothetical group of four commercial accounts. Let $x$ be years since the account’s last major equipment update, and let $y$ be annual claim cost in hundreds of dollars. The observations are $(0,12)$, $(2,14)$, $(4,19)$, and $(6,19)$. A fitted least-squares slope is reported as $\hat{\beta}_1=1.3$ hundreds of dollars per year. Determine the estimated intercept, explain its units, and check the residual sum.

Step 1: Find the predictor mean. $\bar{x}=(0+2+4+6)/4=12/4=3$ years.

Step 2: Find the response mean. $\bar{y}=(12+14+19+19)/4=64/4=16$ hundreds of dollars.

Step 3: Derive the intercept. $\hat{\beta}_0=16-(1.3)(3)=16-3.9=12.1$ hundreds of dollars. The fitted equation is $\hat{y}=12.1+1.3x$, with both fitted response and intercept measured in hundreds of dollars.

Step 4: Check with residuals. At $x=0,2,4,6$, the fitted values are $12.1,14.7,17.3,19.9$. The residuals are $12-12.1=-0.1$, $14-14.7=-0.7$, $19-17.3=1.7$, and $19-19.9=-0.9$. Their sum is $-0.1-0.7+1.7-0.9=0$.

Interpretation and trap avoided. The intercept is the fitted cost at $x=0$, here $12.1$ hundreds of dollars. It is not the average observed cost, which is $16$ hundreds of dollars. The zero-predictor interpretation is meaningful only if $x=0$ makes sense in the study; the algebraic intercept equation still applies whether or not that value is substantively useful.

Common Mistakes and a Practical Check

Using the response mean as the intercept. The correct relation is $\hat{\beta}_0=\bar{y}-\hat{\beta}_1\bar{x}$. Omitting the slope adjustment is valid only in special cases, such as $\bar{x}=0$ or $\hat{\beta}_1=0$.
Changing the sign. The adjustment is subtracted: $\bar{y}-\hat{\beta}_1\bar{x}$. A plus sign generally fails to make the fitted value at $\bar{x}$ equal $\bar{y}$.
Mixing units. The intercept has the response’s units, and the product $\hat{\beta}_1\bar{x}$ must also have response units. Check that the slope’s units times the predictor’s units give the response’s units.
Confusing the intercept with a guaranteed observed value. The intercept is the fitted value at $x=0$. If zero lies outside the observed predictor range, it may be an extrapolated value, but it remains the algebraic intercept of the line.
Expecting every residual to be zero. Least squares requires the residuals to sum to zero when an intercept is included; it does not require each individual residual to vanish.

A quick check is to substitute the calculated intercept into $\hat{\beta}_0+\hat{\beta}_1\bar{x}$. It should equal $\bar{y}$. Alternatively, calculate the fitted residuals and confirm their sum is zero, allowing for small rounding differences if the slope or intercept has been rounded.

The intercept equation is aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus. In a calculation, identify the reported or estimated slope, compute both sample means, retain consistent units, and then apply the minimizing equation.

Check Your Understanding

For each problem, use the least-squares intercept relation and keep the units consistent.

A fitted line has slope $\hat{\beta}_1=1.2$. The observed predictor values are $3,5,7$, and the response values are $8,11,14$. Find $\bar{x}$, $\bar{y}$, and the intercept. (Answer: $\bar{x}=5$, $\bar{y}=11$, $\hat{\beta}_0=5$)
A least-squares output reports slope $1.5$ for data with predictor values $1,3,5,7$ and responses $8,10,15,15$. Find the intercept and the fitted value at the predictor mean. (Answer: $\hat{\beta}_0=6$; the fitted value at $\bar{x}=4$ is $12$)
For a hypothetical risk study, $x$ is years of account tenure and $y$ is annual cost in thousands of dollars. The predictor values are $1,2,3,4$, the response values are $6,7,8,9$, and the fitted slope is $0.8$ thousand dollars per year. Find the intercept, then state the sum-of-residuals condition it must satisfy. (Answer: $\bar{x}=2.5$, $\bar{y}=7.5$, $\hat{\beta}_0=5.5$ thousand dollars; the residuals sum to $0$)
A fitted line has intercept $9$, predictor mean $4$, and response mean $17$. What slope is required for the line to satisfy the least-squares intercept equation? (Answer: $\hat{\beta}_1=2$)
A candidate calculates the intercept as $\bar{y}+\hat{\beta}_1\bar{x}$, with $\bar{x}=6$, $\bar{y}=20$, and $\hat{\beta}_1=-0.5$. Give the correct intercept and explain the sign error. (Answer: $\hat{\beta}_0=23$; the correct equation subtracts $\hat{\beta}_1\bar{x}$, so $20-(-0.5)(6)=23$)

What You'll Learn