Why Regression Output Requires Careful Reading
A regression output table compresses several different ideas into a few columns. The coefficient estimates describe the fitted line; standard errors describe uncertainty in those estimates; a residual standard error summarizes the scale of the observed residuals; and a p-value addresses a specified hypothesis. These quantities are related, but they are not interchangeable.
This tutorial focuses on reading those entries in a simple linear regression. The examples use hypothetical data. As in the earlier material on simple regression, let \(x_i\) be the observed predictor for case \(i\), and let \(Y_i\) be its response.
The coefficient estimates and fitted values can be calculated from the observed data without assuming normally distributed errors. To attach the usual exact small-sample t tests to coefficients, the standard model assumptions are that the errors have conditional mean zero, are independent, have constant variance \(\sigma^2\), and are normally distributed. In notation, \(\varepsilon_i\mid x_1,\ldots,x_n\sim N(0,\sigma^2)\). Inferences also condition on the observed predictor values. If these assumptions do not hold, the usual standard errors, t reference distribution, or p-values may not be appropriate.
What the Coefficient Columns Mean
An output table commonly reports an estimate, a standard error, a t statistic, and a p-value for each coefficient. In a simple regression with an intercept, the slope estimate \(b_1\) is the fitted change in response for a one-unit increase in \(x\). The intercept estimate \(b_0\) is the fitted response at \(x=0\). Its practical meaning depends on whether zero is a meaningful predictor value and whether it is within a relevant range of the data.
The standard error of an estimate measures its estimated sampling variability under the model. It is not the size of the coefficient, and it is not the residual standard error. A coefficient’s t statistic for testing \(H_0:\beta_j=0\) is its estimate divided by its standard error. For the slope, the usual two-sided test compares \(b_1/\operatorname{SE}(b_1)\) with a t distribution having \(n-2\) degrees of freedom.
The p-value is the probability, assuming the null hypothesis and the model assumptions are true, of obtaining a test statistic at least as far from zero as the observed statistic. For a two-sided test it is \(2P(T_{n-2}\geq |t|)\). It is not the probability that the null hypothesis is true, nor is it a measure of the slope’s practical importance.
Worked Example: Reading a Slope Estimate and Its Standard Error
A hypothetical study relates the number of staff training sessions completed by an office team, \(x\), to a quality score, \(Y\), measured in points. The fitted output is based on 12 teams.
| Coefficient | Estimate | Standard error |
|---|---|---|
| Intercept | 42.00 | 5.00 |
| Training sessions | 3.50 | 0.80 |
Method. Read the response and predictor units first, then interpret the estimate and calculate a test statistic only if the question asks for one. The slope estimate is \(b_1=3.50\). In the fitted line, one additional training session corresponds to an increase of 3.50 points in fitted quality score. This is a description of the fitted association; it does not establish that training caused the score to increase.
The intercept is \(b_0=42.00\), so the fitted score at zero training sessions is 42.00 points. That interpretation is useful only if zero sessions is meaningful for the teams and falls within, or near, the range studied.
For a test of \(H_0:\beta_1=0\), the slope t statistic is \(3.50/0.80=4.375\). There are \(n-2=12-2=10\) degrees of freedom. The standard error is uncertainty in the estimated slope; it is not a statement that individual quality scores are typically only 0.80 points from the fitted line.
Check. The fitted line is \(\hat{Y}=42.00+3.50x\). At \(x=0\), it gives 42.00, matching the intercept. The common mistake to avoid is describing 3.50 as the predicted score itself; it is the fitted change in score per additional session.
Residual Standard Error and Fitted Values
Residual standard error (often abbreviated RSE) estimates the standard deviation of the errors, \(\sigma\), using the residuals from the fitted line. In simple regression with an intercept, two parameters are estimated, so the residual degrees of freedom are \(n-2\). If \(\operatorname{RSS}=\sum_{i=1}^n e_i^2\) is the residual sum of squares, then
The RSE is expressed in the same units as the response. It describes the residual scatter around the fitted line, not uncertainty in the slope and not uncertainty in a fitted mean response. Its interpretation also depends on the model being a reasonable description of the relationship and on the error scale being suitably constant.
A fitted value at a specified predictor value is obtained by substituting that value into the estimated line. It is the estimated response on the line, not necessarily the observed response of a particular case. The difference between an observed response and its fitted value is the residual.
Worked Example: Recovering RSS and Calculating a Fitted Value
A hypothetical regression relates the number of inspection shifts completed, \(x\), to a processing time, \(Y\), in hours. The output reports an intercept estimate of 10.00, a slope estimate of 2.50, a residual standard error of 3.00 hours, and 6 residual degrees of freedom.
Method. Use the reported residual degrees of freedom to recover the sample size and RSS, then substitute the requested predictor value into the fitted line. In simple regression with an intercept, \(n-2=6\), so \(n=8\).
From \(s=\sqrt{\operatorname{RSS}/6}=3.00\), square both sides: \(\operatorname{RSS}/6=9.00\). Thus \(\operatorname{RSS}=6(9.00)=54.00\) squared hours. The units are squared because RSS sums squared residuals.
The fitted line is \(\hat{Y}=10.00+2.50x\). At \(x=4\) shifts, \(\hat{Y}=10.00+2.50(4)=10.00+10.00=20.00\) hours.
Check. Dividing the recovered RSS by its 6 degrees of freedom gives \(54.00/6=9.00\), and \(\sqrt{9.00}=3.00\) hours, as reported. The fitted value has response units, hours; it is not an observed processing time and does not mean every case at \(x=4\) takes exactly 20 hours. The common mistake is to use \(n\), rather than \(n-2\), as the divisor when interpreting the RSE.
How the Coefficient Test Is Justified
The t statistic is an estimate measured in standard-error units. Under \(H_0:\beta_1=0\), the slope estimate is centered at zero. With normally distributed independent errors of constant variance, the standardized slope estimate using the true standard deviation has a standard normal distribution. Replacing the unknown error standard deviation with its estimate from the residuals introduces a t distribution.
Under \(H_0\), the sampling distribution of \(b_1/\operatorname{SE}_{\text{true}}(b_1)\) is standard normal under the normal-error model.
The error variance is estimated from \(\operatorname{RSS}/(n-2)\). The two fitted coefficients, intercept and slope, account for the two lost residual degrees of freedom.
Dividing by the estimated standard error gives \(b_1/\operatorname{SE}(b_1)\), which follows a \(t_{n-2}\) distribution under the stated assumptions. This justifies the usual t reference distribution and two-sided p-value.
The same structure applies to an intercept test, using \(b_0/\operatorname{SE}(b_0)\) for \(H_0:\beta_0=0\). A test of the intercept is not automatically substantively useful: it concerns the mean response at \(x=0\), which may be outside the setting of interest.
Worked Example: Correcting and Interpreting a Two-Sided p-Value
A hypothetical analysis relates the number of staff members in a service unit to its average weekly backlog, measured in hundreds of cases. Twenty units were observed. The slope row in the regression output is:
| Coefficient | Estimate | Standard error | t value | Pr(>|t|) |
|---|---|---|---|---|
| Staff members | -1.20 | 0.40 | -3.00 | 0.007685 |
Method. Interpret the slope in context, check the t ratio, and evaluate the p-value using a two-sided test with the correct degrees of freedom. The estimate says that one additional staff member is associated with a fitted decrease of 1.20 hundreds of backlog cases, or 120 cases, per week. It is an estimated association, not proof of a causal effect.
The displayed t statistic checks directly: \(-1.20/0.40=-3.00\). The sample size is \(n=20\), so the test has \(20-2=18\) degrees of freedom. For \(H_0:\beta_1=0\) against \(H_a:\beta_1\ne 0\), the p-value is \(2P(T_{18}\geq 3.00)=0.00768541\ldots\), which is reported to six decimal places as 0.007685.
At a 5% significance level, \(0.007685<0.05\), so the null hypothesis is rejected under the model assumptions. This is evidence that the population slope differs from zero; it does not measure the probability that the null is true. The estimated magnitude and the context still matter when describing the result.
Check. The two-sided p-value is larger than the one-sided upper-tail probability because it counts outcomes at least as extreme in either direction. Rounded to three decimal places, 0.007685 becomes 0.008, not 0.007. Reporting 0.007 truncates rather than rounds and understates the p-value. The common mistake is to report the one-sided tail or truncate the displayed value.
Descriptive Output Versus Inference
A regression table often places estimates and tests side by side, but answering an interpretation question does not always require a hypothesis test. The coefficient estimate describes the fitted line for the data. The standard error and p-value add inferential information only under the assumptions needed for their reference distributions. A small p-value does not guarantee a large or important effect, while a large p-value does not prove the slope is exactly zero.
| Output entry | What it describes | What it does not mean |
|---|---|---|
| Estimate | The fitted intercept or slope for the observed data | That the estimate is exact or causal |
| Coefficient standard error | Estimated sampling variability of that coefficient | The typical residual size around the line |
| Residual standard error | Estimated error scale, in response units | Uncertainty in the slope or fitted mean |
| Fitted value | The response value on the estimated line at a given \(x\) | The actual response of a particular case |
| p-value | Tail probability under a stated null and model | The probability that the null hypothesis is true |
When reading output, retain units and identify the hypothesis being tested. A slope measured in response units per predictor unit cannot be interpreted as a percent change unless the model and scale make that interpretation valid. A coefficient test also concerns a particular null value; if the null is \(H_0:\beta_1=c\), the numerator is \(b_1-c\), not simply \(b_1\).
Check Your Understanding
Work through each item by identifying the quantity being requested before calculating.
- A fitted line for weekly output \(Y\), in units, is \(\hat{Y}=18+2.4x\), where \(x\) is machine setup time in hours. Interpret the slope and find the fitted output at \(x=5\). (Answer: Each additional setup hour corresponds to 2.4 units higher fitted output; at 5 hours, the fitted value is 30 units.)
- A regression has \(n=15\) observations and reports \(\operatorname{RSS}=104\). Calculate the residual standard error, rounding to two decimals. (Answer: \(s=\sqrt{104/(15-2)}=\sqrt{8}=2.83\) response units.)
- A slope estimate is \(-0.90\), with standard error \(0.30\), from a regression on 22 observations. Find the t statistic and its degrees of freedom for testing a zero slope. (Answer: \(t=-0.90/0.30=-3.00\), with \(22-2=20\) degrees of freedom.)
- An output reports a slope estimate of 1.60 and a standard error of 0.50. In a sample of 10 observations, calculate the test statistic and state the null hypothesis tested by the usual reported slope p-value. (Answer: \(t=1.60/0.50=3.20\), with 8 degrees of freedom; the usual two-sided slope p-value tests \(H_0:\beta_1=0\) against \(H_a:\beta_1\ne0\).)
- A regression output reports a residual standard error of 4.2 dollars and a slope standard error of 0.7 dollars per unit of \(x\). A learner says the slope estimate must be within 0.7 dollars of the true slope and that typical residuals are 0.7 dollars. Identify both errors. (Answer: The slope standard error describes estimated sampling variability, not a guaranteed distance from the true slope; the residual standard error, 4.2 dollars, summarizes residual scale.)