Why P-Values Matter in Regression
A regression coefficient test turns an estimate and its uncertainty into a measure of how surprising the observed result would be if a specified null value were correct. The p-value is that measure, calculated using a reference distribution under the null hypothesis. Its size depends both on the observed test statistic and on the alternative hypothesis: a one-sided alternative uses one tail, while a two-sided alternative uses both.
This tutorial focuses on testing a slope in simple linear regression. The same basic logic applies to other coefficient tests, but the degrees of freedom and model details must match the regression being used. A small p-value can provide evidence against the null value; it is not the probability that the null hypothesis is true.
The Slope Test Statistic and Its Reference Distribution
Let \(b_1\) be the least-squares slope estimate, and let \(SE(b_1)\) be its estimated standard error. To test a specified slope value \(\beta_{1,0}\), compare the estimate with that value in standard-error units:
Here \(\nu\) is the residual degrees of freedom in simple linear regression with an estimated intercept and slope. If \(S_{xx}=\sum_{i=1}^n(x_i-\bar{x})^2\), then \(SE(b_1)=s/\sqrt{S_{xx}}\), where \(s^2=SSE/(n-2)\) and \(SSE=\sum_{i=1}^n(Y_i-\hat{Y}_i)^2\). The standard error measures estimated sampling variability; it is not the standard deviation of the observed responses.
The degrees of freedom are \(n-2\), not \(n-1\), because the fitted simple regression estimates two parameters, the intercept and slope, leaving \(n-2\) residual degrees of freedom. The statistic is centered on the hypothesized slope, not necessarily on zero. For example, testing \(\beta_1=0.30\) requires subtracting \(0.30\) from \(b_1\), even when the alternative is one-sided.
Choosing the Tail Before Calculating
The alternative hypothesis determines which outcomes count as evidence against the null. Let \(F_\nu\) denote the cumulative distribution function of a t distribution with \(\nu\) degrees of freedom. The p-value rules are:
| Alternative hypothesis | p-value | Interpretation of extremeness |
|---|---|---|
| \(H_1:\beta_1>\beta_{1,0}\) | \(P(T\ge t_{\mathrm{obs}})\) | Large positive statistics |
| \(H_1:\beta_1<\beta_{1,0}\) | \(P(T\le t_{\mathrm{obs}})\) | Large negative statistics |
| \(H_1:\beta_1\ne\beta_{1,0}\) | \(2P(T\ge |t_{\mathrm{obs}}|)\) | Large departures in either direction |
The two-sided formula uses the symmetry of the t distribution about zero. It doubles the probability in the tail beyond the absolute value of the observed statistic. The p-value is not generally obtained by doubling a tail for a one-sided alternative. In particular, if the observed statistic points opposite to a one-sided alternative, the p-value will be large, not small.
Identify the null value and whether the alternative is greater than, less than, or different from that value.
Subtract the null value from the estimated coefficient and divide by its standard error.
For a simple regression with an intercept and slope, use \(n-2\).
Use the one-tail or two-tail rule corresponding to the alternative, then report the p-value and interpret it under the null.
Why the t Reference Distribution Applies
The logic comes from standardizing the slope estimate and accounting for the fact that the error variance is unknown. Under the model, the slope estimate has a normal sampling distribution with variance \(\sigma^2/S_{xx}\). Also, the residual sum of squares estimates \(\sigma^2\). Under the null hypothesis, the standardized numerator and the residual variance estimate combine to give the t statistic.
For this normal-error regression model, these two quantities are independent. Since \(s^2=SSE/(n-2)\), the denominator of the test statistic can be written as \((\sigma/\sqrt{S_{xx}})\sqrt{[SSE/\sigma^2]/(n-2)}\). Thus the statistic is a standard normal variable divided by the square root of an independent chi-square variable divided by its degrees of freedom. That ratio has a Student t distribution with \(n-2\) degrees of freedom. This explains why estimating the variance changes the normal reference distribution to a t reference distribution.
Worked P-Value Calculations
Worked Example: Evidence for a Negative Slope
A hypothetical routing analysis relates the number of daily alerts at a service center to the average time needed to complete a particular task. For \(n=11\) centers, the estimated slope is \(b_1=-0.80\) minutes per additional alert, with \(SE(b_1)=0.40\). Test \(H_0:\beta_1=0\) against \(H_1:\beta_1<0\), and decide whether to reject at significance level \(0.05\).
Method and statistic. This is a lower-tail t test. Under the stated simple-regression model, the degrees of freedom are \(11-2=9\), and
P-value. The alternative is negative, so the relevant probability is the lower tail:
Decision and interpretation. Since \(0.0383<0.05\), reject the null at the 0.05 level. Under the null slope of zero, a t statistic of \(-2.00\) or smaller has probability about \(0.0383\). This is evidence against a zero slope in the negative direction; it is not a probability that the null is true.
Common mistake avoided: Using the upper-tail probability would point in the wrong direction. The stated alternative is \(\beta_1<0\), so the lower tail is required.
Worked Example: A Two-Sided Slope Test
A hypothetical greenhouse study estimates the relationship between additional daily light-hours and plant growth. For \(n=14\) plants, the estimated slope is \(b_1=0.90\) growth units per light-hour, and its standard error is \(0.45\). Test \(H_0:\beta_1=0\) against \(H_1:\beta_1\ne0\). Calculate the p-value and compare it with \(0.05\).
Method and statistic. The alternative allows either a positive or negative slope. The degrees of freedom are \(14-2=12\), and
P-value. Because the alternative is two-sided, use both tails. By symmetry, this is twice the upper-tail probability beyond \(2.00\):
Decision and interpretation. The p-value \(0.06866\) exceeds \(0.05\), so the test does not reject the null at that level. The observed positive estimate alone does not make this a positive-direction test: the alternative specifically includes departures in either direction.
Common mistake avoided: Reporting \(0.03433\) would use only one tail and would not match the two-sided alternative. Do not choose the tail from the sign of the estimate; choose it from \(H_1\).
Worked Example: Testing a Nonzero Slope Value
A hypothetical materials laboratory models a durability score as a function of curing time. With \(n=20\) batches, the estimated slope is \(b_1=0.72\) score units per hour, its standard error is \(0.24\), and the test is \(H_0:\beta_1=0.30\) against \(H_1:\beta_1>0.30\). Find the p-value and assess the evidence at level \(0.05\).
Method and statistic. This is an upper-tail test against the specified nonzero slope. The degrees of freedom are \(20-2=18\). Center the estimate at \(0.30\), rather than at zero:
P-value. The alternative is greater than \(0.30\), so use the upper tail:
Decision and interpretation. Since \(0.04857<0.05\), reject \(H_0\) at the 0.05 level in favor of slopes greater than \(0.30\). Under a true slope of \(0.30\), the probability of a t statistic at least \(1.75\) is approximately \(0.04857\).
Common mistake avoided: Testing against zero instead would give a different statistic and answer a different question. The null value must be subtracted exactly as stated.
Interpreting a P-Value Correctly
A p-value is conditional on the null hypothesis and the test procedure. For example, a p-value of \(0.04\) means that, if the null model is correct, the probability of getting a test statistic at least as extreme as the observed one, under the specified alternative's tail rule, is \(0.04\). It does not mean there is a 4% chance that the null hypothesis is true. The calculation does not assign probabilities to hypotheses.
A p-value also does not measure the size or practical importance of a slope. A precisely estimated small slope can produce a small p-value, while a potentially important estimate with a large standard error can produce a larger one. The p-value answers a narrower question: how compatible is the observed test statistic with the null reference distribution, using this test's definition of extremeness?
A significance level such as \(0.05\) is a decision threshold selected for the testing procedure. Comparing the p-value with that threshold gives a reject-or-do-not-reject decision. Not rejecting does not establish that the null is true; it means the data do not provide sufficient evidence to reject it at the chosen level.
Common Exam Traps
- Wrong tail: A test of \(\beta_1<\beta_{1,0}\) uses a lower tail; a test of \(\beta_1>\beta_{1,0}\) uses an upper tail. A two-sided test uses both.
- Wrong degrees of freedom: In simple linear regression with an intercept and slope, use \(n-2\), not \(n-1\).
- Wrong null value: For \(H_0:\beta_1=0.30\), calculate \(b_1-0.30\), not \(b_1\).
- Confusing the p-value with a null probability: The p-value is computed assuming the null is true. It is not \(P(H_0\mid\text{data})\).
- Confusing evidence with importance: Statistical evidence against a null value does not by itself say whether the estimated effect matters in the application.
Check Your Understanding
For each problem, identify the null-centered statistic, use the specified t distribution and tail, and interpret the result as a probability under the null.
- A hypothetical slope test has \(n=3\), \(b_1=-0.40\), \(SE(b_1)=0.40\), and hypotheses \(H_0:\beta_1=0\) versus \(H_1:\beta_1<0\). Find the statistic, degrees of freedom, and p-value. (Answer: \(t=-1.00\), \(1\) degree of freedom, \(p=0.2500\).)
- For a hypothetical two-sided test, \(n=3\), \(b_1=0.60\), \(SE(b_1)=0.60\), and \(H_0:\beta_1=0\) versus \(H_1:\beta_1\ne0\). Find the p-value. (Answer: \(t=1.00\), \(1\) degree of freedom, \(p=0.5000\).)
- A hypothetical coefficient test has \(n=3\), estimate \(b_1=1.20\), standard error \(0.60\), and \(H_0:\beta_1=0\) versus \(H_1:\beta_1>0\). Calculate the upper-tail p-value. (Answer: \(t=2.00\), \(1\) degree of freedom, \(p\approx0.1476\).)
- A simple regression has \(n=6\). Its estimated slope is \(1.10\), with standard error \(0.40\). Test \(H_0:\beta_1=0.2472\) against \(H_1:\beta_1>0.2472\). Use the upper-tail probability for \(t=2.132\) with \(4\) degrees of freedom. (Answer: \(t=2.132\), \(4\) degrees of freedom, \(p\approx0.0500\).)
- A two-sided slope test reports a p-value of \(0.04\). State what this means under the null and identify one interpretation it does not support. (Answer: Under the null, the probability of a t statistic at least as extreme in either direction as the observed statistic is \(0.04\); it does not mean the null has a 4% probability of being true.)