Common Traps in Simple Regression Calculations

Why Simple-Regression Calculations Go Wrong

A simple-regression calculation can be numerically correct and still answer the wrong question. Reversing the predictor and response changes the fitted regression line; confusing slope with correlation changes the interpretation; and using a mean-response interval to describe one future observation understates uncertainty. These errors are avoidable when the model, variables, and target are stated before calculating.

The examples use hypothetical actuarial settings. The methods and checks are aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.

Model: In simple linear regression, the response $Y_i$, conditional on the observed predictor value $x_i$, is modeled as $$ Y_i=\beta_0+\beta_1x_i+\varepsilon_i,\qquad i=1,\ldots,n. $$ Here $\beta_0$ and $\beta_1$ are the intercept and slope, and $\varepsilon_i$ is the unobserved error. For the usual small-sample $t$-based inference, assume the errors are independent, normally distributed with mean zero, and have common variance $\sigma^2$. The fitted line is $\hat{y}=b_0+b_1x$; the ordinary least-squares estimates $b_0,b_1$ are calculated from the observed data.

The predictor is the variable on the right-hand side, $x$, and the response is the variable being modeled, $Y$. This orientation is part of the question, not a choice to make based on which regression calculation seems easier. A regression of $Y$ on $x$ generally differs from a regression of $x$ on $Y$.

Trap 1: Reversing Predictor and Response

The least-squares line of $Y$ on $x$ minimizes the sum of squared vertical deviations in the response direction, $\sum_{i=1}^n (y_i-\hat{y}_i)^2$. Reversing the variables instead minimizes squared deviations in the $x$-direction. Those are different objectives, so the resulting slopes are generally not reciprocals.

Worked Example: Keeping the Regression Direction Straight

A hypothetical insurer records a maintenance score $x$ and a repair cost $y$, measured in thousands of dollars, for three devices. The data are $(1,2),(2,3),(3,7)$. Find the least-squares regression of cost on score. Then calculate the slope if score is instead regressed on cost, and explain why it is not the same slope reversed.

Method. Use the requested response as $Y$. For the cost-on-score line, the response is $y$ and the predictor is $x$. First calculate $\bar{x}=2$ and $\bar{y}=4$.

The centered sums are $S_{xx}=(-1)^2+0^2+1^2=2$, $S_{xy}=(-1)(-2)+0(-1)+1(3)=5$, and $S_{yy}=(-2)^2+(-1)^2+3^2=14$. Thus the slope and intercept for cost on score are $b_1=S_{xy}/S_{xx}=5/2=2.5$ and $b_0=\bar{y}-b_1\bar{x}=4-(2.5)(2)=-1$. The fitted line is $\hat{y}=-1+2.5x$, with cost measured in thousands of dollars.

For score on cost, the slope is $S_{xy}/S_{yy}=5/14\approx0.3571$ score units per thousand dollars. The reciprocal of the original slope is $1/2.5=0.4$, not $5/14$. The two numbers differ because the two regressions minimize different squared deviations.

Check. At the sample mean $x=2$, the fitted cost is $-1+2.5(2)=4$, which equals $\bar{y}$. The calculation uses the requested direction: cost on score. The common mistake it avoids is treating the reciprocal of a regression slope as the reverse-regression slope.

Trap 2: Treating Slope and Correlation as the Same Quantity

The slope describes the change in fitted response associated with a one-unit increase in the predictor. Its units are response units per predictor unit. The sample correlation $r$ describes the direction and strength of the linear association and has no units. A slope of 4.5 is not a correlation of 4.5; a correlation must lie between $-1$ and $1$.

Relationship between slope and correlation: For a data set with nonzero sample standard deviations $s_x$ and $s_y$, the least-squares slope of $Y$ on $x$ is $$ b_1=r\frac{s_y}{s_x}. $$ The intercept is $b_0=\bar{y}-b_1\bar{x}$. This relationship applies to the ordinary least-squares line with an intercept; the predictor must vary, so $s_x>0$.

The formula follows from the definitions $b_1=S_{xy}/S_{xx}$ and $r=S_{xy}/\sqrt{S_{xx}S_{yy}}$, where $S_{xx}=\sum (x_i-\bar{x})^2$, $S_{yy}=\sum (y_i-\bar{y})^2$, and $S_{xy}=\sum (x_i-\bar{x})(y_i-\bar{y})$. Since $s_x=\sqrt{S_{xx}/(n-1)}$ and $s_y=\sqrt{S_{yy}/(n-1)}$, dividing $s_y$ by $s_x$ gives $\sqrt{S_{yy}/S_{xx}}$. Therefore,

r\frac{s_y}{s_x} =\frac{S_{xy}}{\sqrt{S_{xx}S_{yy}}} \sqrt{\frac{S_{yy}}{S_{xx}}} =\frac{S_{xy}}{S_{xx}} =b_1.

This also shows why reversing the regression does not generally invert the slope. The reverse slope is $r s_x/s_y$, whereas the original slope is $r s_y/s_x$.

Worked Example: Distinguishing the Slope from the Correlation

In a hypothetical portfolio study, let $x$ be a risk score and $y$ be annual claim cost in dollars. Suppose $s_x=2$ score points, $s_y=15$ dollars, and the fitted slope of cost on score is $b_1=4.5$ dollars per score point. Find the correlation, the predicted cost change for a 3-point score increase, and the slope for score regressed on cost.

From $b_1=r(s_y/s_x)$, solve for $r$: $r=b_1(s_x/s_y)=4.5(2/15)=9/15=0.6$. The correlation is positive and within its required range. The squared correlation is $r^2=0.36$; for simple linear regression with an intercept, this is the proportion of sample response variation explained by the fitted linear relationship.

A 3-point increase in score corresponds to a fitted cost change of $3(4.5)=13.50$ dollars. This is a change in fitted response, not a claim that every individual’s cost rises by that amount.

The slope for score regressed on cost is $r(s_x/s_y)=0.6(2/15)=0.08$ score points per dollar. It is not $1/4.5\approx0.2222$, because the reverse regression has a different response and minimizes squared deviations in different units.

Check. The cost-on-score slope has units dollars per score point; the reverse slope has units score points per dollar; $r=0.6$ has no units. The common mistake avoided is assigning correlation’s interpretation or range to the slope.

Trap 3: Treating a Residual as the True Error

For observation $i$, the fitted value is $\hat{y}_i=b_0+b_1x_i$, and the residual is $e_i=y_i-\hat{y}_i$. The residual is calculable from the observed data. The model error $\varepsilon_i=y_i-(\beta_0+\beta_1x_i)$ depends on the unknown true regression coefficients, so it is not directly observed. A residual estimates the error, but in general it is not equal to that error.

Worked Example: Separating an Observed Residual from an Error

A hypothetical three-device data set contains $(x,y)=(0,1),(1,4),(2,4)$, where $x$ is a condition score and $y$ is a recorded outcome. Calculate the least-squares line and residual at $x=2$. For illustration, suppose the data-generating line is known to be $Y=1+2x+\varepsilon$; compare the model error at $x=2$ with that residual.

The means are $\bar{x}=1$ and $\bar{y}=3$. The centered sums are $S_{xx}=(-1)^2+0^2+1^2=2$ and $S_{xy}=(-1)(-2)+0(1)+1(1)=3$. Thus $b_1=3/2=1.5$ and $b_0=3-(1.5)(1)=1.5$, giving $\hat{y}=1.5+1.5x$.

At $x=2$, the fitted value is $1.5+1.5(2)=4.5$. The observed value is 4, so the residual is $e=4-4.5=-0.5$.

Under the stated illustrative data-generating line, the true conditional mean at $x=2$ is $1+2(2)=5$. The model error for this observation is therefore $\varepsilon=4-5=-1$. The residual $-0.5$ and the error $-1$ differ because the fitted line is estimated from the sample, whereas the true line is specified separately for this illustration.

Check. The residual is computed using the fitted line; the error is computed using the true line. In routine data analysis the true line is unknown, so the error cannot be calculated exactly. The common mistake avoided is labeling a residual as if it were the observed model error.

Trap 4: Choosing the Wrong Interval

An interval for the conditional mean response at $x_0$ addresses uncertainty about $E[Y\mid x_0]$. A prediction interval addresses the value of one future response at $x_0$. A future response includes both uncertainty in estimating the mean and the variation of a new error around that mean, so its interval is wider.

Interval formulas: Under the independent normal, constant-variance simple-regression model, with $n$ observations and residual standard error $s$, the standard errors at $x_0$ are $$ SE_{\text{mean}}=s\sqrt{\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}},\qquad SE_{\text{prediction}}=s\sqrt{1+\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}}. $$ A two-sided $100(1-\alpha)\%$ interval uses the multiplier $t_{1-\alpha/2,n-2}$. Use the first standard error for a mean-response interval and the second for an individual prediction interval.

The extra $1$ in the prediction standard error comes from the new response’s own error. More precisely, write the true conditional mean as $\mu_0=\beta_0+\beta_1x_0$. The prediction error is

Y_{\mathrm{new}}-\hat{y}_0 =(\mu_0-\hat{y}_0)+\varepsilon_{\mathrm{new}}.

Under the model, the new error is independent of the estimation error $\mu_0-\hat{y}_0$. The latter has variance $\sigma^2[1/n+(x_0-\bar{x})^2/S_{xx}]$, while the new error has variance $\sigma^2$. Adding the variances gives $\sigma^2[1+1/n+(x_0-\bar{x})^2/S_{xx}]$, which explains the prediction standard error. The identity uses the true mean $\mu_0$, not the estimated mean as the starting point for $Y_{\mathrm{new}}$.

Worked Example: Selecting the Interval for the Question

A hypothetical regression of claim amount on a rating score has $n=20$, $\bar{x}=6$, $S_{xx}=80$, residual standard error $s=3$, and fitted value $\hat{y}_0=21$ at $x_0=8$. Assume the model conditions for $t$-interval inference hold and use $t_{0.975,18}=2.101$. Calculate both 95% intervals: one for the mean claim amount at score 8 and one for a single future claim amount at score 8. Round endpoints to two decimals, only after using unrounded calculations.

The shared factor in the variance terms is $1/n+(x_0-\bar{x})^2/S_{xx}=1/20+(8-6)^2/80=0.05+4/80=0.10$. Thus the mean-response standard error is $3\sqrt{0.10}=0.948683\ldots$. Its margin is $2.101(0.948683\ldots)=1.993\ldots$, so the mean-response interval is $21\pm1.993\ldots$, or $[19.01,22.99]$.

For one future claim amount, the prediction standard error is $3\sqrt{1+0.10}=3\sqrt{1.1}=3.146426\ldots$. The margin is $2.101(3\sqrt{1.1})=6.610642\ldots$. Therefore the interval is $21\pm6.610642\ldots$, or $[14.39,27.61]$.

Check. The prediction interval is wider because it includes variation in a new response. Endpoints are rounded from the unrounded products; rounding the margin first can change an endpoint. The common mistake avoided is using the narrower mean-response interval to describe one future claim.

A Reliable Calculation Check

Before accepting a simple-regression answer, match the calculation to the question. State which variable is the response, verify the units of the slope, and decide whether the target is a fitted mean or one new observation. Then check whether the result is mathematically plausible: a correlation must be between $-1$ and $1$, the fitted line with an intercept passes through $(\bar{x},\bar{y})$, and a prediction interval at the same $x_0$ is wider than the corresponding mean-response interval.

These checks do not replace the model assumptions. The standard $t$-based intervals rely on independent, normal errors with constant variance; they may not be appropriate when those assumptions fail. Also, regression describes a conditional association and does not by itself establish that changing the predictor causes a change in the response.

Direction: identify $x$ and $Y$ from the wording before calculating a slope.
Units: report slope in response units per predictor unit, and correlation without units.
Residual versus error: calculate residuals from the fitted line; treat errors as unobserved model quantities.
Target: use a mean-response interval for an average at $x_0$, and a prediction interval for one future response.
Rounding: retain unrounded intermediate values and round only the final reported result.

Key takeaway: Most simple-regression traps are mismatches: the wrong response variable, the wrong interpretation of a statistic, an observed quantity confused with a model quantity, or an interval that does not match the target. Naming the target and its units first is a practical safeguard.

Check Your Understanding

For each hypothetical problem, identify the requested quantity and keep the regression direction and target clear.

A fitted line for annual benefit cost $y$, in dollars, on an eligibility score $x$ is $\hat{y}=7+1.2x$. What fitted change in cost corresponds to an increase of 3 score points? (Answer: an increase of $3.60.)
For a regression of $Y$ on $x$, suppose $b_1=2$, $s_x=3$, and $s_y=10$. Calculate $r$ and $r^2$. (Answer: $r=0.6000$, $r^2=0.3600$.)
At $x=5$, an observation has $y=18$ and fitted value $\hat{y}=16.5$. Calculate its residual and state whether the model error is thereby known. (Answer: residual $=1.5$; the model error is not known from these values.)
A regression has $n=25$, $\bar{x}=4$, $S_{xx}=100$, $s=2$, and $\hat{y}_0=12$ at $x_0=6$. Use $t_{0.975,23}=2.064$. Give the 95% mean-response and individual prediction intervals, rounded to two decimals. (Answer: mean-response interval $[10.83,13.17]$; prediction interval $[7.71,16.29]$.)
A hypothetical study gives $r=-0.5$, $s_x=2$ rating points, and $s_y=8$ dollars. Find the slope of dollars on rating points and the slope of rating points on dollars. (Answer: $-2$ dollars per rating point and $-0.125$ rating points per dollar.)

What You'll Learn