Why Report Both?
A regression table can contain an estimate, a confidence interval, and a p-value, but a row of numbers is useful only when readers can tell what each number represents. A confidence interval shows a range of coefficient values compatible with the data under a stated method. A p-value summarizes how unusual the observed test statistic would be if a specified null hypothesis were true. Reporting both, with the confidence level and test stated, gives readers more information than marking a coefficient simply “significant” or “not significant.”
This tutorial focuses on presenting those quantities together. It does not replace reporting the model specification, analysis sample, estimate units, or reference groups. Those details provide the context needed to understand the results; the interval and p-value add uncertainty and test information.
The Model and the Two Summaries
For conventional coefficient inference in a linear regression, write the model for observation \(i\) as \(Y_i=\beta_0+\beta_1X_{i1}+\cdots+\beta_kX_{ik}+\varepsilon_i\). Here \(Y_i\) is the outcome, the \(X\)'s are predictors, and \(\beta_j\) is the population coefficient for predictor \(j\), conditional on the other predictors in the model. For exact small-sample \(t\)-inference, assume the conditional mean is correctly specified as linear in the included predictors, the errors are independent and normally distributed with mean zero and constant variance, and the model and analysis sample have been specified appropriately. The estimated coefficient is \(\hat{\beta}_j\), and its estimated standard error is \(SE(\hat{\beta}_j)\).
For the usual two-sided test of \(H_0:\beta_j=0\), the test statistic is \(t=\hat{\beta}_j/SE(\hat{\beta}_j)\). In a model with \(n\) observations and \(k\) predictors plus an intercept, the residual degrees of freedom are \(df=n-k-1\). The p-value is calculated from the \(t\) distribution with those degrees of freedom, assuming \(H_0\) and the model conditions hold.
A confidence level must be stated, such as 95%. In repeated samples under the model and method, 95% of intervals constructed this way would cover the fixed coefficient \(\beta_j\). It is not correct to say that there is a 95% probability that this particular interval contains \(\beta_j\). The p-value is also conditional on a null hypothesis; it is not the probability that \(H_0\) is true.
Worked Examples
Worked Example: An Interval That Excludes Zero
Suppose a hypothetical multiple regression models a test score using five predictors and an intercept, with \(n=42\) observations. The estimated coefficient for study time is \(2.40\) score points per additional hour, with \(SE=0.90\) score points per hour. Assume the observations are independent, the conditional mean model is correctly specified, and the errors meet the normality and constant-variance conditions for conventional \(t\)-inference. Take the reported standard error and coefficient as coming from that fitted model.
There are \(df=42-5-1=36\) residual degrees of freedom. For the two-sided test \(H_0:\beta_{\text{study}}=0\), the statistic is
Using the \(t\) distribution with 36 degrees of freedom, the two-sided p-value is approximately \(0.0114\). For a 95% confidence interval, the critical value is approximately \(t^*=2.028\). The margin of error is
Thus the interval is
A concise report could read: “The estimated study-time coefficient was 2.40 score points per additional hour (95% CI 0.57 to 4.23; two-sided \(p=0.0114\)).” The interval excludes zero, as does the corresponding two-sided test at the 5% significance level. The result describes an estimated conditional association in this model; it does not by itself establish a causal effect.
Worked Example: A Wide Interval That Includes Zero
In a second hypothetical regression, a model with five predictors and an intercept is fitted to \(n=30\) independent observations. For one predictor, suppose the estimated coefficient is \(1.20\) outcome units per 10-minute increase in the predictor, with standard error \(0.75\) outcome units. For this example, assume the linear conditional mean, independent errors, constant error variance, and normal errors are reasonable for the stated \(t\)-inference.
The residual degrees of freedom are \(30-5-1=24\). The test statistic for \(H_0:\beta_j=0\) is
The two-sided p-value with 24 degrees of freedom is approximately \(0.123\). The 95% critical value is approximately \(2.064\), giving a margin of error of
The 95% confidence interval is therefore
A suitable report is: “The estimated coefficient was 1.20 outcome units per 10-minute increase (95% CI -0.35 to 2.75; two-sided \(p=0.123\)).” The interval includes zero and the p-value exceeds 0.05, so this analysis does not reject the zero-coefficient null hypothesis at the 5% level. It does not show that the coefficient is zero: values ranging from a modest negative association to a positive association remain compatible with this interval and method.
Worked Example: Reporting a Very Small P-Value
Consider a third hypothetical regression with five predictors, an intercept, and \(n=66\) independent observations. A coefficient is estimated as \(4.80\) response units per one-unit increase in a predictor, with standard error \(1.20\) response units. Suppose the conditional mean model is appropriate and the independent, normal, constant-variance error conditions are adequate for the conventional \(t\)-procedure.
The residual degrees of freedom are \(66-5-1=60\), and the statistic for the two-sided test of \(H_0:\beta_j=0\) is
The two-sided p-value from a \(t\) distribution with 60 degrees of freedom is less than \(0.001\). It should not be reported as \(p=0.000\), because a continuous test distribution does not give this statistic a p-value of exactly zero. For a 95% interval, \(t^*\) is approximately \(2.000\), so the margin of error is \(2.000(1.20)=2.400\). The interval is
Report, for example: “The estimated coefficient was 4.80 response units per one-unit increase (95% CI 2.40 to 7.20; two-sided \(p<0.001\)).” The interval gives a range on the coefficient scale, while the p-value communicates the extremeness of the test statistic under the null. A threshold such as \(p<0.001\) is a concise format for a very small value, not a complete description of the result.
Why the Interval and Test Usually Agree
When a confidence interval and p-value use the same conventional standard error, \(t\) distribution, degrees of freedom, coefficient, and two-sided test, their conclusions at matching levels must agree. For example, a 95% interval corresponds to a two-sided test at \(\alpha=0.05\). The link follows directly from their formulas.
The interval is \(\hat{\beta}_j\mathbin{\pm}t^*SE(\hat{\beta}_j)\). It includes zero exactly when \(\hat{\beta}_j-t^*SE(\hat{\beta}_j)\leq 0\leq\hat{\beta}_j+t^*SE(\hat{\beta}_j)\).
Dividing by the positive standard error gives \(-t^*\leq\hat{\beta}_j/SE(\hat{\beta}_j)\leq t^*\), or equivalently \(|t|\leq t^*\).
For a two-sided test, \(p<0.05\) exactly when \(|t|>t^*\), where \(t^*\) is the 97.5th percentile of the same \(t\) distribution. Therefore, the 95% interval excludes zero exactly when \(p<0.05\), apart from rounding in displayed values.
This equivalence depends on matching the inference procedures. A 90% interval should not be compared with a 5% two-sided test as though they had the same cutoff. Nor should a conventional interval be expected to align exactly with a p-value computed using a different standard error or method. State the method when it matters, and do not conceal a discrepancy by changing displayed digits.
Presenting the Numbers Clearly
A useful report places the estimate, interval, confidence level, p-value, and test direction together. Include units for the estimate and interval endpoints, and identify the coefficient's predictor and model context. Use a consistent number of meaningful digits: typically two or three decimal places for estimates and interval limits when appropriate, and three significant figures for ordinary p-values. Precision should reflect the analysis, not imply more certainty than the data support.
- Write the confidence level explicitly: “95% CI,” not simply “CI.”
- State whether the p-value is from a one-sided or two-sided test when that is not already clear from the analysis.
- Use \(p<0.001\) for very small p-values rather than \(p=0.000\).
- Keep units and sign in both endpoints; a negative lower endpoint is meaningful when zero is included.
- Report the estimate and interval even when the p-value is above a chosen threshold.
A p-value is not the probability that the null hypothesis is true, and a 95% confidence interval is not a statement that the parameter has a 95% probability of lying in the particular observed interval. Also avoid using “statistically significant” as a substitute for describing the estimate and its uncertainty. Evidence against a null value is not automatically evidence of a large, important, or causal association.
Common Mistake: Letting a Threshold Replace the Result
A common mistake is to write “the predictor had no effect because \(p>0.05\).” That reasoning treats a failure to reject the null as proof that the coefficient equals zero. The correction is to report the estimate, interval, and p-value, then describe what the interval permits. In the second example, the interval from -0.35 to 2.75 outcome units shows that the estimate is uncertain and includes both negative and positive values. It does not establish a zero association.
The opposite mistake is to report only “\(p<0.05\)” and imply that the result is important. A small p-value does not state the coefficient's magnitude or precision. The confidence interval and estimate supply those missing details, while the p-value describes compatibility with a specified null under the model and test.
Check Your Understanding
Answer these questions using the definitions and examples above.
- What information does a 95% confidence interval communicate that a p-value alone does not?
- For a two-sided test of a zero coefficient, why should a 95% interval and a 5% test agree when they use the same model-based inference?
- A regression report gives \(p=0.000\). What is wrong with this report, and how should a very small p-value be presented?
- A 95% interval includes zero and the p-value is 0.18. Does this prove the coefficient is zero? Explain what the interval adds.
- Why is “95% probability that the coefficient is in the interval” not the standard frequentist interpretation of a 95% confidence interval?