From Regression Output to an Inferential Conclusion
A regression summary compresses several different questions into a few columns. A coefficient estimate describes the fitted relationship; its standard error measures estimation uncertainty; a t statistic and p-value address a specified hypothesis; and an interval gives a range of coefficient values compatible with the data and model. The ANOVA portion asks a broader question about the regression as a whole. Correct interpretation requires matching each number to the question it answers.
The examples below use hypothetical data. Inference is conditional on a regression model and its assumptions; a small p-value is not, by itself, evidence of causation or proof that the model is well specified.
The fitted coefficient \(b_j\) estimates \(\beta_j\). In a multiple regression, \(\beta_j\) describes the change in the conditional mean of \(Y\) for a one-unit increase in predictor \(j\), with the other predictors held fixed. The standard error, denoted \(SE(b_j)\), measures uncertainty in that estimate; it is not the standard deviation of individual response values.
Coefficient Tests, t Statistics, and p-Values
A common coefficient test is \(H_0:\beta_j=0\) against \(H_A:\beta_j\ne 0\). More generally, the null may specify a value \(\beta_{j,0}\). The test statistic compares the estimate's distance from that null value with its estimated standard error:
Under the null hypothesis and the model assumptions, this statistic has a Student t distribution with residual degrees of freedom \(n-p-1\). For a two-sided alternative, the p-value is the probability, assuming the null is true, of a t statistic at least as far from zero as the observed statistic:
A one-sided alternative requires the corresponding one-tail probability and must be specified before interpreting the result. A p-value is not the probability that the null hypothesis is true. It measures how unusual the observed statistic, or one more extreme, would be if the null and model assumptions held.
Worked Example: Testing a Slope in a Hypothetical Pricing Study
A hypothetical analysis relates annual claim cost, measured in thousands of dollars, to a risk score using simple linear regression. There are 23 observations. The summary reports a risk-score slope estimate \(b_1=1.20\) and \(SE(b_1)=0.40\). Test \(H_0:\beta_1=0\) against a two-sided alternative at the 5% level, and interpret a 95% confidence interval.
Method. Use the coefficient t test and the corresponding t interval. In simple regression, \(p=1\), so the residual degrees of freedom are \(23-1-1=21\).
Test statistic. Under the null value \(\beta_{1,0}=0\),
For a two-sided test with 21 degrees of freedom, the p-value is \(2P(T_{21}\ge 3.00)=0.006822\). Since \(0.006822<0.05\), reject the zero-slope null at the 5% level. The evidence is inconsistent with a zero linear slope under the model; it does not establish that the risk score causes claim cost to change.
Interval. The 95% critical value is approximately \(t_{0.975,21}=2.0796\). The margin of error is \(2.0796(0.40)=0.83184\), so the interval is
Rounded to four decimal places, the 95% confidence interval is \((0.3682,\ 2.0318)\) thousand dollars per score point. Both endpoints are positive, agreeing with rejection of zero in the two-sided 5% test. The interval describes uncertainty in the slope, not a range containing 95% of individual claim costs.
Check and mistake avoided. The test statistic uses the coefficient's standard error, \(0.40\), not the residual standard deviation of individual costs. The positive estimate and interval are also consistent: the interval excludes zero.
Confidence Intervals for Coefficients
A two-sided \(100(1-\alpha)\%\) confidence interval for a coefficient is obtained by taking the estimate plus and minus a t critical value times its standard error:
The confidence level describes the long-run coverage of the procedure when its assumptions hold: across repeated samples, the procedure would cover the fixed coefficient in approximately \(100(1-\alpha)\%\) of samples. It is not a probability statement that the fixed coefficient is random and lies in this particular interval.
For the two-sided test \(H_0:\beta_j=\beta_{j,0}\) at level \(\alpha\), the test rejects exactly when \(\beta_{j,0}\) lies outside the corresponding \(100(1-\alpha)\%\) interval. This connection is useful as a check: a 95% interval containing zero corresponds to not rejecting a two-sided zero-coefficient test at 5%, when both use the same model and standard error.
Worked Example: Interpreting a Coefficient Near the Significance Boundary
In a hypothetical multiple regression for claim-processing time, a binary predictor indicates whether a claim was submitted digitally (1 if yes, 0 if no). The estimated coefficient is \(b=-1.50\) days, with \(SE(b)=0.75\) days and 18 residual degrees of freedom. Construct 90% and 95% confidence intervals and explain their implications for two-sided tests of a zero coefficient.
Method. Apply the coefficient interval formula using the appropriate t critical value for each confidence level. For 18 degrees of freedom, the 90% critical value is approximately 1.7341 and the 95% critical value is approximately 2.1009.
For 90%, the margin is \(1.7341(0.75)=1.300575\). Thus the interval is \(-1.50\pm1.300575=(-2.800575,-0.199425)\), or \((-2.8006,-0.1994)\) days to four decimal places.
For 95%, the margin is \(2.1009(0.75)=1.575675\). Thus the interval is \(-1.50\pm1.575675=(-3.075675,0.075675)\), or \((-3.0757,0.0757)\) days to four decimal places.
The 90% interval excludes zero, so the zero-coefficient null would be rejected by the corresponding two-sided 10% test. The 95% interval includes zero, so it would not be rejected by the corresponding 5% test. In context, the estimate is a 1.50-day lower fitted mean processing time for digital submissions than for non-digital submissions, conditional on the other predictors. The intervals show substantial uncertainty about the size of that difference.
Check and mistake avoided. The confidence level changes the critical value, not the estimate or standard error. The 95% interval must be wider than the 90% interval; here it is, and it includes the 90% interval.
How the t Formula and Interval Arise
The t procedure is not just a rescaling rule. Under the stated normal-error model, the least-squares coefficient estimate is normally distributed around its true coefficient. In matrix notation, if \(X\) is the design matrix and \(\boldsymbol{\beta}\) the coefficient vector, then
The diagonal entry of \((X^\mathsf{T}X)^{-1}\) for coefficient \(j\) is denoted \(c_{jj}\). Thus the standard deviation of \(b_j\), if \(\sigma\) were known, would be \(\sigma\sqrt{c_{jj}}\). In practice, \(\sigma\) is unknown and is estimated by the residual standard deviation \(s=\sqrt{MSE}\), where \(MSE=SSE/(n-p-1)\). The reported standard error is \(SE(b_j)=s\sqrt{c_{jj}}\).
For normal errors, \(b_j-\beta_j\), divided by its true standard deviation, is standard normal. Also, \(SSE/\sigma^2\) has a chi-square distribution with \(n-p-1\) degrees of freedom and is independent of the coefficient estimate. Replacing \(\sigma\) by \(s\) therefore divides a standard normal quantity by the square root of an independent chi-square quantity divided by its degrees of freedom. That ratio has a Student t distribution. This gives the t statistic and, by taking its central \(1-\alpha\) probability, the confidence interval formula.
This exact derivation depends on the model assumptions, particularly normal errors for finite-sample t inference. If those assumptions are not appropriate, the usual standard errors, p-values, and intervals may not have their advertised behavior; the output alone cannot verify the assumptions.
Reading the ANOVA Portion of a Regression Summary
The analysis-of-variance (ANOVA) table partitions the total variation in the response around its sample mean into variation explained by the fitted regression and residual variation. With an intercept in the model, the sums of squares are \(SST=\sum_i(Y_i-\bar Y)^2\), \(SSR=\sum_i(\hat Y_i-\bar Y)^2\), and \(SSE=\sum_i(Y_i-\hat Y_i)^2\), with \(SST=SSR+SSE\). Their degrees of freedom are \(n-1\), \(p\), and \(n-p-1\), respectively.
The corresponding mean squares are \(MSR=SSR/p\) and \(MSE=SSE/(n-p-1)\). The overall F statistic is
It tests the joint null hypothesis \(H_0:\beta_1=\cdots=\beta_p=0\) against the alternative that at least one slope is nonzero. Under the null and model assumptions, it has an F distribution with numerator degrees of freedom \(p\) and denominator degrees of freedom \(n-p-1\). Its p-value is an upper-tail probability. A significant overall F test does not identify which predictor is associated with the response; individual coefficient tests answer different questions.
Worked Example: Reading an Overall F Test from an ANOVA Table
A hypothetical simple regression of annual repair cost on equipment age uses 23 observations. Its ANOVA summary reports regression sum of squares \(SSR=18\) and residual sum of squares \(SSE=42\), both in squared thousands of dollars. Calculate the ANOVA degrees of freedom, mean squares, overall F statistic, \(R^2\), and overall test conclusion at the 5% level.
Method. There is one predictor, so \(p=1\). The regression degrees of freedom are 1, the residual degrees of freedom are \(23-1-1=21\), and the total degrees of freedom are \(23-1=22\).
The total sum of squares is \(SST=SSR+SSE=18+42=60\). Therefore \(MSR=18/1=18\), and \(MSE=42/21=2\). The F statistic is \(18/2=9\), with degrees of freedom \((1,21)\). For one predictor, the overall F test is equivalent to the two-sided slope t test, with \(F=t^2\); here \(\sqrt{9}=3\). Its p-value is therefore \(0.006822\), the same as the two-sided \(t_{21}\) p-value for a statistic of 3. Since \(0.006822<0.05\), reject the overall zero-slope null.
The coefficient of determination is \(R^2=SSR/SST=18/60=0.30\). Thus 30% of the sample variation in annual repair cost around its mean is accounted for by the fitted simple linear regression. This is a descriptive measure of fit, not a test of causation or a guarantee of predictive accuracy.
Check and mistake avoided. The residual mean square divides \(SSE\) by its residual degrees of freedom, 21—not by the number of observations, 23. The significant F result concerns the slope in this one-predictor model; for multiple regression it would only establish evidence that at least one slope is nonzero.
Common Mistake: Treating Every Output Column as the Same Evidence
A coefficient's t test, the overall F test, and \(R^2\) are related but not interchangeable. The t test concerns one coefficient under a specified null. The overall F test concerns all slopes jointly. \(R^2\) summarizes the fraction of sample response variation explained by the fitted regression; it does not itself provide a significance test.
A frequent wrong calculation is to divide a slope estimate by the residual standard deviation. For example, if \(b_j=1.20\), the residual standard deviation is \(s=2.00\), and the coefficient standard error is \(SE(b_j)=0.40\), the wrong statistic is \(1.20/2.00=0.60\). The correct statistic for testing a zero coefficient is \(1.20/0.40=3.00\). The residual standard deviation describes unexplained response variation; the coefficient standard error describes uncertainty in the estimated slope.
Another interpretive error is to say that a small overall F-test p-value proves every predictor matters. The correct conclusion is that the joint zero-slope null is rejected; at least one slope contributes evidence against that joint null. Individual coefficient output is needed for predictor-specific conclusions, and even then the interpretation is conditional on the other included predictors.
Check Your Understanding
For each problem, identify the relevant inference and check that your conclusion matches the supplied degrees of freedom and confidence level.
- A simple regression has 16 observations. A slope estimate is \(0.90\), with standard error \(0.30\). For testing a zero slope, find the residual degrees of freedom and t statistic, then state whether the two-sided test rejects at 5%. (Answer: \(df=14\), \(t=3.00\); reject at 5%.)
- A coefficient estimate is \(-2.00\), its standard error is \(0.75\), and there are 18 residual degrees of freedom. Construct a 95% interval using \(t_{0.975,18}=2.1009\), rounding endpoints to four decimal places. (Answer: \((-3.5757,-0.4243)\).)
- A regression with two predictors uses \(n=16\) observations, with \(SSR=36\) and \(SSE=84\). Find the regression and residual degrees of freedom, \(MSR\), \(MSE\), and the overall F statistic, rounding mean squares and F to four decimals. (Answer: \(df_R=2\), \(df_E=13\), \(MSR=18.0000\), \(MSE=6.4615\), \(F=2.7857\).)
- A 95% interval for a slope is \((-0.20,1.10)\). What is the corresponding decision for a two-sided zero-slope test at 5%, assuming the same model-based standard error and degrees of freedom? (Answer: Do not reject; zero is inside the interval.)
- An overall F test in a regression with four predictors has p-value 0.02 at the 5% level. What is the strongest appropriate conclusion from this result alone? (Answer: Reject the joint null that all four slopes are zero; it does not show that all four individual slopes are nonzero.)