The Linearity Assumption in Simple Regression

What the Linearity Assumption Means

A simple regression line is useful only if a straight-line trend is a reasonable description of how the average response changes with the predictor. The key word is average: individual responses can vary substantially, even when their conditional mean follows a straight line. This tutorial focuses on that distinction and on recognizing when the mean pattern bends instead.

This material is aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus. The earlier material on the simple linear regression model and conditional means gives the notation used here.

Definition: In simple linear regression, the linearity assumption is that the conditional mean response is an affine function of the predictor over the range being modeled. In notation, for predictor $X$ and response $Y$, $$ E(Y\mid X=x)=\beta_0+\beta_1x. $$ Here $\beta_0$ is the intercept and $\beta_1$ is the change in conditional mean response for a one-unit increase in $x$. Equivalently, the model can be written $Y=\beta_0+\beta_1X+\varepsilon$, with $E(\varepsilon\mid X=x)=0$ over that range.

The assumption concerns the shape of the conditional mean, not the shape of the distribution of $X$, and not whether every observed point lies on a line. Given $X=x$, an individual response can differ from its mean because of its error term. The model does not require those errors to be zero; it requires their conditional mean to be zero so that the regression line represents the conditional mean.

“Linear” here includes an intercept, so the function is technically an affine function. It may have a positive slope, a negative slope, or a zero slope. The relationship is still linear if the conditional mean is constant as $x$ changes.

Constant Change and Curvature

A straight-line conditional mean has a constant rate of change. For any two predictor values $x_1$ and $x_2$, its mean responses differ by $\beta_1(x_2-x_1)$. Thus, if $x$ increases by the same amount each time, the conditional mean should also change by the same amount each time. Systematically increasing or decreasing increments suggest curvature rather than a straight-line mean pattern.

E(Y\mid X=x_2)-E(Y\mid X=x_1) =\beta_1(x_2-x_1)

For equally spaced predictor values, a related check uses second differences. If the mean responses at $x-h$, $x$, and $x+h$ come from a straight line, the two adjacent changes are equal, so the second difference is zero. This is a pattern check, not a claim that a few estimated group means prove a population model exactly.

\begin{aligned} \mu(x+h)-\mu(x)&=\beta_1h,\\ \mu(x)-\mu(x-h)&=\beta_1h,\\ \mu(x+h)-2\mu(x)+\mu(x-h)&=0, \end{aligned}

The justification follows directly by substituting $\mu(x)=\beta_0+\beta_1x$. For example, $\mu(x+h)=\beta_0+\beta_1x+\beta_1h$, while $\mu(x)=\beta_0+\beta_1x$; subtracting gives $\beta_1h$. The same subtraction applies from $x-h$ to $x$, so subtracting the two equal changes gives zero. Curvature produces unequal changes across equal steps and generally a nonzero second difference.

Worked Example: Checking Equal Changes in Conditional Means

A hypothetical insurer groups policies by the number of covered locations, $x$, and estimates the mean annual service cost per policy, in dollars. The estimated conditional means are 12, 17, 22, and 27 for $x=0,2,4,6$, respectively. Assess whether these values follow a linear pattern and, if so, calculate the fitted mean at $x=5$.

Method. Compare changes in mean response with changes in the predictor, then find the line from its intercept and slope.

The predictor increases by 2 at each step. The changes in the means are $17-12=5$, $22-17=5$, and $27-22=5$. Each equal predictor increase of 2 is associated with the same mean increase of 5. The rate of change is therefore $5/2=2.5$ dollars per additional location.

At $x=0$, the conditional mean is 12, so the intercept is 12. The line is $\hat{\mu}(x)=12+2.5x$. At $x=5$, $\hat{\mu}(5)=12+2.5(5)=12+12.5=24.5$ dollars.

Check. The line gives means 12, 17, 22, and 27 at $x=0,2,4,6$, matching all four supplied values. The calculated mean at $x=5$ is between the values at $x=4$ and $x=6$, as expected.

Common mistake avoided. Dividing the total mean change, $27-12=15$, by the number of table rows would give the wrong rate. The predictor spans 6 units, so the rate is $15/6=2.5$ dollars per location.

Recognizing Systematic Curvature

A curved pattern does not mean that the data are useless or that every straight line is impossible to calculate. It means a straight line may fail to represent the conditional mean adequately across the range. A line can still summarize an average trend, but its errors may be systematically positive in some portions of the range and negative in others. Such a pattern is different from the unsystematic scatter expected around a suitable mean line.

When the predictor values are equally spaced, inspect successive differences. A steady increase in those differences suggests the mean is bending upward; a steady decrease suggests it is bending downward. For unequal spacing, compare the change in mean per unit change in predictor rather than comparing raw changes alone.

Worked Example: Detecting an Upward Bend

A hypothetical benefits administrator estimates mean monthly processing cost, in dollars, at predictor levels $x=1,2,3,4,5$. The estimated conditional means are 8, 11, 16, 23, and 32. Determine whether the pattern supports a straight-line conditional mean, and compare a line through the two endpoints with the intermediate means.

Method. Because the $x$-values are equally spaced by 1, calculate successive changes and second differences.

The successive changes are $11-8=3$, $16-11=5$, $23-16=7$, and $32-23=9$. These are not constant; they increase by 2 each time. The second differences are $5-3=2$, $7-5=2$, and $9-7=2$. Their consistent positive value indicates an upward bend rather than a constant rate of change.

The line through the endpoints has slope $(32-8)/(5-1)=24/4=6$ and intercept $8-6(1)=2$. It predicts means $2+6x$. At $x=2$, this gives 14, compared with the supplied mean of 11; at $x=3$, it gives 20, compared with 16; at $x=4$, it gives 26, compared with 23. The line matches at $x=1$ and $x=5$ by construction, but misses the intermediate values by $-3$, $-4$, and $-3$, respectively, where each difference is supplied mean minus line prediction.

Conclusion and check. The successive increases are 3, 5, 7, and 9, so the rate is not constant. A straight line through the endpoints does not reproduce the middle means; the evidence points to systematic curvature. The common mistake is to treat agreement at the endpoints as evidence of linearity. The intermediate pattern matters.

Observed Responses and the Mean Pattern

In practice, a table may show individual responses rather than known conditional means. At a given predictor value, different individuals can have different outcomes. Consequently, a few observed points above or below a line do not by themselves establish curvature. The question is whether the average response pattern changes systematically as the predictor changes.

A scatterplot can help reveal whether a straight-line trend is plausible: a cloud with a roughly straight central pattern differs from one whose center bends. If responses are grouped at predictor values, group averages can also help describe the pattern, while recognizing that those sample averages are estimates of conditional means. The evidence should be considered across the predictor range, not from a single unusual point.

Worked Example: Verifying a Linear Pattern at Unequal Predictor Steps

In a hypothetical study of document-review workloads, the predictor $x$ is the number of files assigned and the response is mean review time in minutes. Estimated conditional means are 41, 50, 62, and 77 minutes at $x=2,5,9,14$. Decide whether the means are consistent with a straight line, and calculate the mean at $x=11$ under that model.

Method. Since predictor steps differ, divide each mean change by its predictor change and compare the rates.

From $x=2$ to $x=5$, the rate is $(50-41)/(5-2)=9/3=3$ minutes per file. From $x=5$ to $x=9$, it is $(62-50)/(9-5)=12/4=3$. From $x=9$ to $x=14$, it is $(77-62)/(14-9)=15/5=3$. The rates are equal, so the conditional means are consistent with a line.

Using slope 3 and the point $(2,41)$, the intercept is $41-3(2)=35$. Thus the mean line is $\hat{\mu}(x)=35+3x$. At $x=11$, the estimated mean is $35+3(11)=35+33=68$ minutes.

Check. Substitution gives 41 at $x=2$, 50 at $x=5$, 62 at $x=9$, and 77 at $x=14$. At $x=11$, the value 68 lies between the means 62 and 77, consistent with the positive slope. The common mistake avoided here is comparing raw mean changes of 9, 12, and 15 without accounting for predictor steps of 3, 4, and 5.

Scope, Interpretation, and Common Traps

The linearity assumption is about the predictor range for which the model is being used. A straight-line pattern over observed values does not guarantee that the same pattern continues far beyond them. Predictions outside the observed range require extending the line where there may be no supporting information. Even within the range, a plausible straight-line mean does not guarantee that every other modeling condition holds; linearity is one property of the conditional mean, not a complete description of the error behavior.

Keep these distinctions in mind when interpreting a regression model:

Linearity is not “all points lie on a line.” It is the conditional mean that follows the line; individual responses can vary around it.
A slope is a rate per predictor unit. With unequal predictor steps, compare mean changes per unit rather than raw changes.
Endpoint agreement is insufficient. Intermediate means can reveal strong curvature even when a line joins the endpoints exactly.
A curved trend is systematic, not just scatter. Look for a pattern in the center of the responses, rather than treating one high or low observation as decisive.
Do not extrapolate casually. A linear pattern in the observed range does not establish linearity beyond that range.

Key takeaway: A simple regression model assumes $E(Y\mid X=x)=\beta_0+\beta_1x$ over the range being modeled. Equal changes in $x$ should correspond to equal changes in the conditional mean. Changing rates of change suggest curvature and warn that a straight-line mean may not describe the pattern adequately.

Check Your Understanding

For each problem, assess the conditional-mean pattern before using a straight-line model.

For $x=0,3,6,9$, estimated means are 14, 20, 26, and 32. Find the rate of change per unit of $x$, and state whether the values are consistent with a line. (Answer: 2 units of response per unit of $x$; yes.)
At equally spaced values $x=1,2,3,4$, estimated means are 5, 9, 15, and 23. Calculate the successive changes and decide whether the pattern is linear. (Answer: Changes are 4, 6, and 8; no, they indicate increasing increments and upward curvature.)
At $x=2,6,10$, estimated means are 19, 31, and 43. Find the common rate per unit of $x$, the intercept, and the predicted mean at $x=8$. (Answer: Rate 3, intercept 13, predicted mean 37.)
At $x=4,7,12,18$, estimated means are 30, 39, 54, and 72. Compare the rates over the three intervals and assess linearity. (Answer: Rates are $9/3=3$, $15/5=3$, and $18/6=3$; the values are consistent with a line.)
For $x=1,2,3,4,5$, estimated means are 12, 16, 22, 30, and 40. A line is drawn through the first and last means. Find its predicted mean at $x=3$, compare it with the supplied mean, and interpret the discrepancy. (Answer: Endpoint slope is $(40-12)/(5-1)=7$, intercept is 5, and the prediction at $x=3$ is 26, four units above the supplied mean of 22; the changing increments indicate curvature, so endpoint agreement does not establish linearity.)

What You'll Learn