The Simple Linear Regression Model

Introduction

A scatterplot can suggest a straight-line pattern, but a model states more precisely how that pattern is represented. In simple linear regression, one predictor contributes a straight-line component to a response, while an error term accounts for the difference between that component and an individual observation. Keeping those pieces distinct is essential: the model is not the same thing as the data.

For example, an analyst might study whether a hypothetical policy feature is associated with claim cost. The predictor values and recorded claim costs are data. A regression model is a mathematical description relating them; it includes parameters and errors that are not simply additional columns of observed data.

Definition: The simple linear regression model for observation $i$ is

Y_i=\beta_0+\beta_1x_i+\varepsilon_i,\qquad i=1,\ldots,n.

Here, $x_i$ is the predictor value for observation $i$, $Y_i$ is its response, $\beta_0$ is the intercept, $\beta_1$ is the slope, and $\varepsilon_i$ is the error term. There is one predictor, and the systematic part of the model, $\beta_0+\beta_1x_i$, is a straight line in $x_i$.

Reading the Model Equation

The equation has two components. The expression $\beta_0+\beta_1x_i$ is the line component: it specifies the response value represented by the model at predictor value $x_i$, before adding the error. The term $\varepsilon_i$ represents the observation-specific departure from that component. The response includes both.

Predictor, $x_i$: the value of the one explanatory variable recorded for observation $i$. Its units matter because they determine how to interpret a one-unit change.
Response, $Y_i$: the variable the analysis is intended to describe or predict. Its units are the units used for the outcome.
Intercept, $\beta_0$: the constant term in the line component.
Slope, $\beta_1$: the coefficient multiplying the predictor in the line component. It specifies how much that component changes for a one-unit increase in $x_i$.
Error, $\varepsilon_i$: the term that allows an individual response to differ from the line component.

The model parameters $\beta_0$ and $\beta_1$ describe the line in the model. They are not labels for individual observations. In an application, their values are generally unknown. This tutorial uses given parameter values to illustrate the model equation; it does not estimate them from data.

The $Y_i$ in the equation denotes the response as represented in the model. After data are collected, the recorded response is written $y_i$. Thus a data set can be represented by the pairs $(x_i,y_i)$, for $i=1,\ldots,n$. The observations are the recorded predictor and response values; the model is the equation proposed to describe their relationship.

Important distinction: The observed pair $(x_i,y_i)$ is data. The equation $Y_i=\beta_0+\beta_1x_i+\varepsilon_i$ is a model. The intercept and slope are model parameters, and the error term represents an observation’s departure from the line component. A scatter of observed points need not lie exactly on a straight line.

Using the Equation for an Individual Observation

When the parameters and the error value for an observation are specified, the model equation gives its response by substitution. Conversely, if the parameters and an observed response are known, rearranging the equation gives the error value required for that observation. In either direction, keep units consistent: the predictor, response, intercept, slope, and error must be used on their stated scales.

Worked Example: Calculating Responses From a Model

A hypothetical insurer studies annual claim cost $Y_i$, measured in thousands of dollars, against a policy feature score $x_i$, measured in points. For this illustration, use the model $Y_i=12+3.5x_i+\varepsilon_i$. For two policies, the score and error values are: policy A has $x_A=2$ and $\varepsilon_A=-1$; policy B has $x_B=4$ and $\varepsilon_B=2$. Calculate each response and identify the line component and error contribution.

Method. Substitute each policy’s predictor into the line component $12+3.5x_i$, then add that policy’s error. The intercept is 12 thousand dollars, the slope is 3.5 thousand dollars per score point, and the stated error values are in thousands of dollars. No probability distribution is assigned to the errors in this example.

For policy A, the line component is $12+3.5(2)=12+7=19$. Adding the error gives $Y_A=19+(-1)=18$, or \$18,000.

For policy B, the line component is $12+3.5(4)=12+14=26$. Adding the error gives $Y_B=26+2=28$, or \$28,000.

Check. Policy B has a score 2 points higher than policy A. The line component rises by $3.5(2)=7$ thousand dollars. The error changes by $2-(-1)=3$ thousand dollars, so the observed response rises by $7+3=10$ thousand dollars, matching $28-18=10$. The common mistake is to omit the error and report the line component as the observed response.

Why an Observed Difference Is Not Necessarily the Slope

The slope describes the change in the line component per predictor unit. It does not, by itself, say that every pair of observed responses will differ by exactly the slope times the predictor difference. The error terms can also differ from observation to observation.

This follows directly from the model. For two observations $i$ and $j$, write the model equation for each, then subtract the first equation from the second:

\begin{aligned} Y_j&=\beta_0+\beta_1x_j+\varepsilon_j,\\ Y_i&=\beta_0+\beta_1x_i+\varepsilon_i,\\ Y_j-Y_i &=(\beta_0-\beta_0)+\beta_1(x_j-x_i)+(\varepsilon_j-\varepsilon_i)\\ &=\beta_1(x_j-x_i)+(\varepsilon_j-\varepsilon_i). \end{aligned}

The intercept cancels, but the difference between the errors remains. Therefore, an observed response difference equals the slope component plus the difference in errors. Only when the two errors are equal does that error difference vanish.

Worked Example: Separating the Slope Component From an Observed Change

In a hypothetical portfolio study, the response is an annual administrative cost in hundreds of dollars, and the predictor is the number of covered locations. The model is $Y_i=8+1.2x_i+\varepsilon_i$. Policy A has $x_A=5$ and error $\varepsilon_A=-1$; policy B has $x_B=9$ and error $\varepsilon_B=2$. Find both model responses, their observed difference, and the contribution due to the slope.

Method. Calculate each response from the model, then compare the response difference with the slope component. The parameters are $\beta_0=8$ hundred dollars and $\beta_1=1.2$ hundred dollars per location; the given errors are in hundreds of dollars. No error distribution is specified.

For A, the line component is $8+1.2(5)=8+6=14$, so $Y_A=14+(-1)=13$ hundred dollars.

For B, the line component is $8+1.2(9)=8+10.8=18.8$, so $Y_B=18.8+2=20.8$ hundred dollars.

The observed response difference is $20.8-13=7.8$ hundred dollars. The predictor difference is $9-5=4$ locations, so the slope component is $1.2(4)=4.8$ hundred dollars. The error difference is $2-(-1)=3$ hundred dollars, and $4.8+3=7.8$, as required.

Check. The responses and the difference are all on the hundred-dollar scale. The common mistake is to divide the observed response difference by the predictor difference and call the result the model slope: $7.8/4=1.95$. That is the observed change per location for this pair, not the slope $1.2$, because the errors differ.

Model Versus Recorded Data

The distinction between an equation and data can be made explicit using observed values. Suppose the model parameters are supplied and a response has been recorded. The equation can then be rearranged to find the error value consistent with that observation:

\varepsilon_i=y_i-(\beta_0+\beta_1x_i).

This rearrangement does not turn the model into the data. The observed $y_i$ is recorded; the expression $\beta_0+\beta_1x_i$ is the model’s line component; their difference is the error implied by the specified parameters. If the parameters are not known, the data alone do not tell us which line component to subtract. Estimation of the parameters is a separate task.

Worked Example: Reading a Data Table Through a Specified Model

A hypothetical review records response $y_i$, in thousands of dollars, and predictor $x_i$, in rating points, for three accounts. The specified model is $Y_i=25+4x_i+\varepsilon_i$, where the intercept is 25 thousand dollars and the slope is 4 thousand dollars per point. The observed pairs are $(1,30)$, $(3,39)$, and $(6,52)$. For each account, calculate the model line component and the error implied by the stated parameters. Do not estimate a different line.

Method. For each row, calculate $25+4x_i$, then subtract that quantity from the recorded response $y_i$. The table values are observations, while the equation and its parameters are the stipulated model. No probability distribution is assigned.

For the first account, $x_1=1$ and $y_1=30$. The line component is $25+4(1)=29$, so $\varepsilon_1=30-29=1$ thousand dollars.

For the second account, $x_2=3$ and $y_2=39$. The line component is $25+4(3)=37$, so $\varepsilon_2=39-37=2$ thousand dollars.

For the third account, $x_3=6$ and $y_3=52$. The line component is $25+4(6)=49$, so $\varepsilon_3=52-49=3$ thousand dollars.

Check. Adding each implied error back to its line component gives $29+1=30$, $37+2=39$, and $49+3=52$, exactly the recorded responses. The common mistake is to treat the line component as if it were the observed response; here, for example, the third account’s line component is 49, not 52.

What the Model Does—and Does Not—Assert

The simple linear regression model provides a structured way to express a one-predictor linear relationship with observation-specific deviations. It does not require every recorded point to lie on the line. If $\varepsilon_i=0$, the response equals the line component for that observation; if $\varepsilon_i\ne0$, the response differs from it by that amount.

The equation alone, as introduced here, does not specify a probability distribution for the errors or impose conditions such as independence, a particular average error, or a common error variance. Those are additional modeling assumptions, not consequences of writing the equation. Likewise, the existence of a regression relationship does not by itself establish that changing the predictor causes the response to change.

A useful habit is to label every quantity before calculating: which values are observed, which are specified model parameters, and which are error values? Mixing these categories can lead to errors such as calling a recorded difference the slope, treating a line component as an observed response, or treating an unknown parameter as if it were already calculated from the data.

Key takeaway: In $Y_i=\beta_0+\beta_1x_i+\varepsilon_i$, the intercept and slope define the line component, and the error accounts for the individual departure from it. The recorded data are the observed predictor-response pairs $(x_i,y_i)$; they are not the model equation itself.

Check Your Understanding

For each problem, distinguish the stated model from the observations and keep the units consistent.

A hypothetical claim model is $Y_i=10+2x_i+\varepsilon_i$, where $Y_i$ is in thousands of dollars and $x_i$ is in points. If $x_i=3$ and $\varepsilon_i=-2$, find the line component and response. (Answer: line component $=16$ thousand dollars; response $=14$ thousand dollars.)
A model has slope 0.8 and two observations have predictor values 2 and 7. What is the difference between their line components? (Answer: $0.8(7-2)=4$ response units.)
Under $Y_i=5+3x_i+\varepsilon_i$, an observation has $x_i=4$ and recorded response $y_i=19$. Find the line component and the implied error. (Answer: line component $=17$; implied error $=2$.)
A model has slope 1.5. Two observations have predictor values 4 and 8, with errors $-1$ and $2$, respectively. Find the response difference from the model, and explain why it is not just the slope times the predictor difference. (Answer: $1.5(8-4)+(2-(-1))=6+3=9$ response units; the errors differ by 3.)
A stipulated model is $Y_i=20+2x_i+\varepsilon_i$. The observed pairs are $(2,26)$ and $(5,29)$. Find each implied error and compare the observed response difference with the slope component. (Answer: implied errors are 2 and $-1$; observed difference $=3$, while the slope component is $2(5-2)=6$, offset by the error difference $-3$.)

What You'll Learn