From Hypotheses to Contextual Conclusions
Regression output often puts several coefficients and standard errors side by side. The inferential task is not simply to read off a p-value: first identify the claim being tested, then match it to a coefficient or contrast, and finally describe what the result does and does not establish. This workshop applies that sequence to four common questions.
Each example uses hypothetical data. The calculations assume the stated regression model is appropriate. In particular, for the usual exact t tests, the conditional mean is correctly specified, errors are independent with mean zero and constant variance, and errors are normally distributed. The residual degrees of freedom are those from the fitted model. In practice, plots and knowledge of how observations were collected can reveal problems with these assumptions, but cannot prove that they hold.
The Test for a Coefficient or Contrast
A test asks whether the target \(L\) equals a hypothesized value \(L_0\). If the estimated contrast has standard error \(SE(\hat{L})\), the test statistic is
For an exact t test under the normal linear model, this statistic is compared with a t distribution using the model's residual degrees of freedom, \(df\). For a two-sided alternative, the p-value is the probability, assuming \(H_0\), of a t statistic at least as far from zero as the observed statistic:
The standard error of a contrast must account for the covariance among coefficient estimates. If \(V\) is the estimated covariance matrix of \(b\), then \(SE(\hat{L})=\sqrt{c^\mathsf{T}Vc}\). Software often supplies this standard error directly. It should not generally be found by treating the component coefficient estimates as independent.
Why the t Statistic Has This Form
The numerator measures the distance between the estimate and the null value in the units of the contrast. Dividing by its standard error expresses that distance in estimated standard-error units. Under the null hypothesis, that standardized distance has a t distribution when the normal linear model conditions hold.
To see the justification, let the standard deviation of \(\hat{L}\) be \(\sigma\sqrt{c^\mathsf{T}(X^\mathsf{T}X)^{-1}c}\) in a full-rank linear model. Under \(H_0:L=L_0\), the numerator divided by this true standard deviation is standard normal. The residual variance estimate, divided by the true error variance, is an independent chi-squared quantity divided by its degrees of freedom. Replacing the unknown \(\sigma\) with the residual standard deviation therefore gives the ratio of a standard normal variable to the square root of an independent chi-squared variable per degree of freedom. By definition, that ratio follows a t distribution. This also explains why the residual degrees of freedom determine the reference distribution.
Worked Example: Testing a Single Slope
A hypothetical sample of 30 students is used to relate weekly hours of independent study, \(x\), to quiz score, \(Y\), measured in points. The fitted simple regression slope is \(b_1=0.84\) points per study hour, with \(SE(b_1)=0.28\) points per hour and \(df=28\). The question is whether the population slope differs from zero.
Model and conditions. Write \(Y_i=\beta_0+\beta_1x_i+\varepsilon_i\). The errors are assumed independent, mean zero, constant in variance, and normal for the exact t test. For this hypothetical example, the study observations are assumed to be independent; the scatterplot shows no marked curvature, and residual-versus-fitted and normal-quantile plots show no substantial pattern or severe departure. These checks support, but do not establish, the assumptions.
Hypotheses. \(H_0:\beta_1=0\) versus \(H_a:\beta_1\ne 0\). The slope is the expected change in quiz score associated with one additional weekly study hour; the test concerns a linear association, not a causal effect.
Statistic and p-value. Under the null, \(L_0=0\), so
The two-sided p-value is \(2P(T_{28}\geq 3.00)=0.005617\). At significance level \(\alpha=0.05\), this is smaller than \(\alpha\), so the data provide evidence that the population linear slope is not zero. The estimated association is positive: the fitted mean quiz score increases by 0.84 points per additional weekly study hour. The test does not show that studying caused the increase.
Worked Example: Testing a Nonzero Coefficient Benchmark
In a separate hypothetical regression, exam score in points is related to weekly hours of practice. The estimated slope is \(b_1=2.00\) points per hour, with \(SE(b_1)=0.40\) points per hour and \(df=40\). A benchmark question asks whether the slope differs from 1.00 point per hour, not whether it differs from zero.
Model and conditions. The fitted conditional mean is assumed linear in practice hours, with independent, mean-zero, constant-variance normal errors. In this hypothetical data set, observations are independent by design, and residual and scatterplot checks show no strong curvature, changing spread, or serious normality concern.
Hypotheses. \(H_0:\beta_1=1.00\) versus \(H_a:\beta_1\ne 1.00\), with the benchmark measured in exam points per practice hour. The benchmark is a specified value for the coefficient; it is not the intercept and is not a claim that the coefficient must be positive.
Statistic and p-value.
The two-sided p-value is approximately 0.0166. Since \(0.0166<0.05\), the result is evidence that the population slope differs from 1.00 point per hour. The fitted estimate is above the benchmark, but this observational regression alone does not establish that extra practice causes higher exam scores. Testing \(H_0:\beta_1=0\) would answer a different question.
Worked Example: Testing a Difference Between Slopes
A hypothetical regression relates daily energy use to outdoor temperature and building type, with an interaction between temperature and type. The estimated temperature slope is \(-0.80\) kWh per degree for one building type and \(-0.20\) kWh per degree for the other. The estimated difference, first type minus second type, is \(-0.60\) kWh per degree, with standard error \(0.20\) kWh per degree and \(df=28\).
Model and conditions. The conditional mean is assumed linear in temperature, type, and their interaction. The errors are assumed independent, mean zero, constant in variance, and normal. For this hypothetical example, residual plots show no substantial pattern or changing spread, the temperature relationship appears adequately linear over the observed range, and no single observation is unduly influential.
Hypotheses. Let \(\beta_{\text{temp,1}}\) and \(\beta_{\text{temp,2}}\) denote the temperature slopes for the two types. The contrast is \(L=\beta_{\text{temp,1}}-\beta_{\text{temp,2}}\). Test \(H_0:L=0\) against \(H_a:L\ne 0\). This directly tests whether the slopes differ; separate tests of whether each slope is zero do not answer that question.
Statistic and p-value. The contrast estimate is also obtained from the displayed slopes: \(-0.80-(-0.20)=-0.60\) kWh per degree. Using the standard error of that difference, which accounts for covariance between the slope estimates,
The two-sided p-value is \(2P(T_{28}\geq 3.00)=0.005617\). At the 0.05 level, there is evidence that the temperature slopes differ. The estimated difference is negative, so the first type's estimated energy-use change per degree is 0.60 kWh per degree more negative than the second type's. This is a comparison of conditional associations in the fitted model.
Worked Example: Testing a Weighted Group Comparison
A hypothetical regression estimates adjusted mean outcome scores for three groups at the same specified covariate values. The estimates are 12.0 points for Group A, 10.0 for Group B, and 9.2 for Group C. A planned comparison asks whether Group A differs from the average of Groups B and C. The standard error of this contrast is 0.80 points, with \(df=28\).
Model and conditions. The group means are assumed to come from a correctly specified regression model with the covariates held at the same values for all three groups. The errors are assumed independent, mean zero, constant in variance, and normal. In this hypothetical example, the design yields independent observations, and residual checks show no marked variance pattern or serious outlier problem.
Hypotheses. The contrast is \(L=\mu_A-\tfrac{1}{2}\mu_B-\tfrac{1}{2}\mu_C\), with weights \(1,-0.5,-0.5\), which sum to zero. Test \(H_0:L=0\) against \(H_a:L\ne 0\). The comparison is specified in terms of adjusted means at common covariate values, not raw group averages.
Estimate, statistic, and p-value. The estimated weighted comparison is
Therefore,
The two-sided p-value is 0.005617. At \(\alpha=0.05\), the data provide evidence that Group A's adjusted mean differs from the average adjusted mean of Groups B and C. The estimated contrast is positive: Group A's adjusted mean is 2.40 points higher than that average. If this comparison were selected after inspecting many possible contrasts, the unadjusted p-value would not by itself account for that selection.
Common Mistakes and a Practical Checklist
A test's conclusion depends on the null value, the contrast, and its uncertainty—not just on whether an individual coefficient's software p-value is small. Use the following checks before interpreting a result:
- Using zero when the benchmark is nonzero. A test of \(\beta=0\) is not a test of \(\beta=1\). Subtract the stated benchmark in the numerator.
- Comparing separate p-values instead of testing a difference. One slope being significant and another not significant does not demonstrate that the slopes differ. Test their difference directly.
- Ignoring covariance in a contrast. The standard error for a difference or weighted comparison must correspond to the entire contrast, not to separate coefficient standard errors combined as if independent.
- Misreading a p-value. A p-value is calculated assuming the null hypothesis and model conditions; it is not the probability that the null is true, nor the probability that the observed effect is due to chance.
- Turning association into causation. A regression coefficient describes a modeled relationship. A causal interpretation needs an appropriate design and assumptions beyond a significant test.
Check Your Understanding
For each question, identify the hypothesis or calculation needed before stating a contextual conclusion.
- A fitted slope is 0.72 with standard error 0.24 and \(df=18\). Write the two-sided hypotheses for testing whether the slope is zero and calculate the t statistic.
- A coefficient estimate is 4.5 with standard error 1.0. What numerator should be used to test whether the population coefficient equals 3.0?
- Two estimated group-specific slopes are 1.1 and 0.6. Which hypothesis tests whether their difference is zero?
- For three adjusted means, define a contrast comparing Group A with the average of Groups B and C. What are its weights?
- Why is it generally incorrect to calculate a difference's standard error by assuming its coefficient estimates are independent?