Common Misreadings of Regression Intervals and Tests

Why Interpretation Matters

Regression software can produce a slope test, a confidence interval, and one or more intervals for responses in the same output. These quantities answer different questions. A small p-value does not give the probability that a null hypothesis is true, a confidence interval does not assign a probability to a fixed parameter after it has been calculated, and an interval for an individual response is not an interval for the mean response.

A useful interpretation begins by identifying the target: a population slope, a mean response at a specified predictor value, or one future individual response. It also identifies the assumptions that make the inferential procedure appropriate. The aim here is not to repeat how software fits a regression line, but to read and describe the inferential results without changing what they mean.

The Regression Setting

For a simple linear regression, write the response for observation \(i\) as \(Y_i\) and its predictor value as \(x_i\). The population mean response is assumed to follow a straight line, with random errors around that line:

Y_i=\beta_0+\beta_1x_i+\varepsilon_i.

Here, \(\beta_1\) is the population slope: the change in mean response associated with a one-unit increase in the predictor. For the usual exact \(t\)-based slope inference, the model requires a linear mean relationship, independent errors with common variance, and normally distributed errors. The observations should also come from a suitable sampling or study design, and the predictor values should provide enough variation to estimate a slope. Severe outliers or influential observations can undermine the usual interpretation.

Definition: A p-value for a test is the probability, calculated under the null hypothesis and the test's assumptions, of obtaining a test statistic at least as extreme as the one observed, in the direction specified by the alternative hypothesis. It is not the probability that the null hypothesis is true.

A Slope Test: What the P-Value Says

For a two-sided test of whether the population slope is zero, the hypotheses are \(H_0:\beta_1=0\) and \(H_a:\beta_1\ne 0\). The sample slope \(b_1\) is compared with its standard error \(SE(b_1)\). Under the null model and the regression assumptions, the test statistic follows a \(t\) distribution with \(n-2\) degrees of freedom:

t=\frac{b_1-0}{SE(b_1)},\qquad df=n-2.

For a two-sided alternative, the p-value is \(P(|T|\ge |t|)\), where \(T\) has a \(t\) distribution with \(n-2\) degrees of freedom. It describes how unusual the observed statistic, or a more extreme one, would be if the null slope were zero and the model assumptions held. It does not measure the size or practical importance of an association, and it does not tell us the chance that the null is correct.

Worked Example: Interpreting a Slope-Test P-Value

Suppose a hypothetical study samples 20 plants independently across a range of fertilizer amounts. The response is plant growth in centimetres, and the predictor is fertilizer amount in grams. The fitted slope is \(b_1=1.80\) centimetres per gram, with \(SE(b_1)=0.75\) centimetres per gram. Suppose a residual plot is consistent with a linear mean and roughly constant spread, the residuals show no serious outliers and are reasonably compatible with normality, and the sampling design supports independence. These conditions make the usual slope \(t\)-test reasonable for this example.

Test \(H_0:\beta_1=0\) against \(H_a:\beta_1\ne 0\). Since \(n=20\), the degrees of freedom are \(20-2=18\). The observed statistic is

t=\frac{1.80-0}{0.75}=2.40.

For a two-sided test with 18 degrees of freedom, the p-value is approximately \(0.02743\). In context, if the population slope were zero and the model assumptions held, the probability of obtaining a slope-test statistic at least as far from zero as \(2.40\) in either direction would be about \(0.02743\).

At significance level \(\alpha=0.05\), this p-value is below \(\alpha\), so the data provide evidence against a zero population slope. The conclusion is evidence of a nonzero linear association in the population represented by the study, subject to the assumptions and study design. It is not evidence that there is a \(2.743\%\) probability the null hypothesis is true. Nor does the test establish that increasing fertilizer causes greater growth; a causal conclusion requires an appropriate design and justification.

A Slope Confidence Interval: What Is Random?

A confidence interval for the population slope gives a range of values compatible with the sample and the specified confidence procedure. In a simple linear regression, a two-sided interval with confidence level \(1-\alpha\) is

b_1\ \pm\ t^*SE(b_1),

where \(t^*\) is the appropriate critical value from a \(t\) distribution with \(n-2\) degrees of freedom. A 95% confidence procedure has a 95% long-run coverage rate when the model assumptions hold: across repeated appropriate samples, about 95% of intervals constructed this way would contain the fixed population slope. For a particular sample, both the calculated interval and the population slope are fixed. The frequentist interpretation therefore does not say there is a 95% probability that the slope lies in this particular interval.

Worked Example: Interpreting a Slope Confidence Interval

Use the hypothetical plant study above, with \(b_1=1.80\) centimetres per gram, \(SE(b_1)=0.75\) centimetres per gram, and \(df=18\). For a 95% two-sided interval, the critical value is approximately \(t^*=2.101\). The assumptions and study conditions described for that example are taken to hold here as well.

The margin of error is

2.101(0.75)=1.576\text{ centimetres per gram}.

Thus the interval is

1.80\pm 1.576 =(0.224,\ 3.376)\text{ centimetres per gram}.

A suitable interpretation is: using this method, we are 95% confident that the population mean growth changes by between about \(0.224\) and \(3.376\) centimetres for each additional gram of fertilizer, over the range and setting represented by the study. More precisely, the 95% describes the long-run success rate of the interval procedure, not a probability assigned to the fixed slope after the interval has been calculated.

The interval excludes zero, consistent with the two-sided test at the 5% level. This is evidence against a zero slope, but it does not by itself establish practical importance or causation. For example, whether a change of \(0.224\) centimetres per gram is important depends on the setting and the range of fertilizer amounts considered.

Mean Response and Individual Prediction Are Different Targets

At a chosen predictor value \(x_0\), the fitted value \(\hat{y}_0\) estimates the population mean response \(E(Y\mid x_0)\). A confidence interval for mean response describes uncertainty about that mean. A prediction interval describes where one new individual response at \(x_0\) may fall. Individual responses vary around their mean, so a prediction interval must account for both uncertainty in estimating the mean and the variation of individual outcomes around it.

In simple linear regression, the estimated standard error for the mean response at \(x_0\) is

SE_{\text{mean}}=s\sqrt{\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}},

where \(s\) is the residual standard deviation, \(\bar{x}\) is the sample mean of the predictor, and \(S_{xx}=\sum (x_i-\bar{x})^2\). For an individual prediction at the same \(x_0\), the corresponding standard error is

SE_{\text{prediction}}=s\sqrt{1+\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}}.

Both intervals use a \(t\) critical value with \(n-2\) degrees of freedom. The extra \(1\) in the prediction standard error represents the variation of a new individual response around the mean response. It is why a prediction interval is wider than the corresponding confidence interval for the mean response.

Worked Example: Mean Response Interval Versus Prediction Interval

Suppose a hypothetical, well-checked simple linear regression uses 10 independent observations to predict a response measured in milligrams from a predictor measured in hours. The residual standard deviation is \(s=3.00\) milligrams, \(\bar{x}=5\) hours, and \(S_{xx}=40\) hours squared. At \(x_0=5\) hours, the fitted response is \(\hat{y}_0=50.0\) milligrams. Assume the linearity, constant-variance, independence, and normal-error conditions are reasonable, and that \(x_0\) is within the observed predictor range.

Here \(x_0=\bar{x}\), so the term involving \(x_0-\bar{x}\) is zero. With \(n-2=8\) degrees of freedom, the 95% critical value is approximately \(2.306\). For the mean response, the standard error is

SE_{\text{mean}} =3.00\sqrt{\frac{1}{10}+\frac{(5-5)^2}{40}} =3.00\sqrt{0.10} =0.9487\text{ milligrams}.

The margin of error is \(2.306(0.9487)=2.188\) milligrams, giving the 95% confidence interval

50.0\pm2.188=(47.81,\ 52.19)\text{ milligrams}.

This interval estimates the mean response for all individuals with a predictor value of 5 hours. It does not claim that 95% of individual responses fall between these endpoints.

For one new individual at 5 hours, the prediction standard error is

SE_{\text{prediction}} =3.00\sqrt{1+\frac{1}{10}+\frac{(5-5)^2}{40}} =3.00\sqrt{1.10} =3.146\text{ milligrams}.

The margin of error is \(2.306(3.1464)\approx7.256\) milligrams, so the 95% prediction interval is

50.0\pm7.256=(42.74,\ 57.26)\text{ milligrams}.

Under the model assumptions, this prediction interval is intended to cover one future individual response at 5 hours with the stated long-run coverage rate. Its wider endpoints reflect the additional individual-to-individual variation. It is not the uncertainty interval for the population mean.

A Short Justification for the Wider Prediction Interval

The distinction between the two response intervals follows from the sources of uncertainty. Write a new response at \(x_0\) as its population mean plus a new error:

Y_{\text{new}}(x_0)=\beta_0+\beta_1x_0+\varepsilon_{\text{new}}.

The fitted mean \(\hat{y}_0\) estimates \(\beta_0+\beta_1x_0\). The difference between the new response and that fitted mean can be written as the sum of the new error and the error in estimating the mean:

Y_{\text{new}}(x_0)-\hat{y}_0 =\varepsilon_{\text{new}}+\bigl(\beta_0+\beta_1x_0-\hat{y}_0\bigr).

Under the model, the new error has variance \(\sigma^2\) and is independent of the estimation error. Therefore the variances add. The estimated variance for the mean-estimation part is \(s^2\left(\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}\right)\), while the new individual error contributes another \(s^2\). Adding them gives

s^2+s^2\left(\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}\right) =s^2\left(1+\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}\right),

which is the squared prediction standard error. Since the added term is positive, the prediction standard error exceeds the mean-response standard error at the same \(x_0\). Using the same positive critical value consequently produces a wider prediction interval.

Common Misreadings and Corrections

Wrong: “The p-value is the probability that the null hypothesis is true.” Correction: The p-value is calculated assuming the null hypothesis and measures how extreme the observed test statistic is under that assumption. It does not give a probability for the hypothesis.
Wrong: “There is a 95% probability that the fixed population slope is inside this calculated confidence interval.” Correction: The slope and calculated endpoints are fixed. The 95% confidence level describes the long-run coverage of the method across repeated appropriate samples.
Wrong: “A confidence interval for the mean response tells us where 95% of individual outcomes will lie.” Correction: It estimates the mean response. Use a prediction interval when the target is one individual response.
Wrong: “Because the prediction interval is wider, it is a less accurate version of the mean-response interval.” Correction: They target different quantities. The prediction interval is wider because individual outcomes vary around their mean.
Wrong: “A statistically significant slope proves a useful effect or a causal effect.” Correction: Statistical evidence addresses compatibility with a specified null model. Practical importance depends on the effect size and context; causation depends on study design and assumptions beyond a small p-value.

Key takeaway: Before interpreting a regression result, name its target. A slope test evaluates evidence against a null slope; a slope confidence interval estimates the population slope; a mean-response interval estimates a conditional mean; and a prediction interval concerns an individual future response. State the assumptions, context, and units, and do not turn a confidence level or p-value into a probability statement about a fixed hypothesis or parameter.

Check Your Understanding

For each item, identify the target and give an interpretation that avoids the common misreading.

A two-sided slope test reports \(p=0.03\). What does this probability describe if the null slope is zero?
A 95% confidence interval for a slope is \((0.4, 1.2)\) minutes per kilometre. Why is “there is a 95% probability that the slope is in this interval” not the standard frequentist interpretation?
At a specified predictor value, what is the difference between the target of a confidence interval for mean response and the target of a prediction interval?
Why is a prediction interval wider than a mean-response interval at the same predictor value?
A slope test is statistically significant. Name one conclusion it supports and two conclusions it does not establish on its own.