Reporting Effect Magnitude Alongside Uncertainty

Introduction

A regression result can be statistically precise yet small in practical terms, or uncertain enough to include effects that would matter. A report that says only “the coefficient was significant” hides both the size of the estimated association and the uncertainty around it. A useful summary gives readers the information needed to judge those features: the coefficient, its outcome units, a confidence interval, a p-value when relevant, and a practical benchmark expressed on the same scale.

This tutorial focuses on how to report and interpret those quantities together. The aim is not to replace the numerical result with a label such as “significant” or “not significant,” but to state what the fitted model estimates and how that estimate compares with a meaningful amount of change.

The Regression Result to Be Reported

For clarity, consider a simple linear regression model for a quantitative outcome \(Y\) and predictor \(x\):

Here \(\beta_0\) is the intercept, \(\beta_1\) is the population slope, and \(\epsilon_i\) is the error for observation \(i\). The slope describes the change in the model’s mean outcome for a one-unit increase in \(x\). Its units are outcome units per predictor unit. The fitted slope is written \(\hat{\beta}_1\); an interval estimates plausible values for \(\beta_1\).

The confidence interval and p-value in the examples below use the usual t-based regression inference. In this model, the standard assumptions for exact small-sample t inference are that the errors are independent, have mean zero at each predictor value, have constant variance \(\sigma^2\), and are normally distributed. Equivalently, conditional on the predictor values, \(\epsilon_i\) are independent and follow \(N(0,\sigma^2)\). The examples state the relevant checks or treat the reported summaries as coming from analyses in which these assumptions have been assessed. These assumptions concern the model and its errors; they do not by themselves show that a predictor causes an outcome.

A confidence interval describes uncertainty about the coefficient: under repeated sampling and the stated model assumptions, the interval procedure captures the true coefficient at its stated confidence rate. A p-value for a two-sided test of \(H_0:\beta_1=0\) describes how unusual a test statistic at least as far from zero as the observed one would be if that null hypothesis and the model assumptions held. It is not the probability that the null hypothesis is true, nor is it a measure of practical importance.

A Compact Reporting Structure

A concise regression report normally answers four questions in a sentence or two:

1. What was estimated? Name the predictor change and the outcome, with the coefficient in outcome units per predictor unit.
2. How uncertain is the estimate? Give the confidence level and interval, in the same units and with enough precision to be informative.
3. What does the p-value address? If relevant to the question, state the test and report the p-value rather than using “significant” as a substitute for the estimate.
4. How does the result compare with a practical benchmark? State the benchmark in context and explain what the estimate and interval indicate relative to it.

The benchmark should be chosen for the subject matter, not selected after seeing which value makes the result sound important. It might be a difference considered worthwhile by stakeholders, a change that would affect a decision, or a threshold established by subject-matter knowledge. A benchmark is not automatically a hypothesis-testing cutoff.

Why the Interval and Benchmark Belong in the Same Report

A coefficient is the best-fitting point estimate, not a complete account of what the data support. Its confidence interval shows a range of values compatible with the analysis at the stated confidence level. A benchmark gives context to both: readers can see whether the point estimate is small or large relative to a meaningful amount, and whether the interval also covers values on either side of that benchmark.

The comparison is most useful when the benchmark is stated in the effect’s own units. A threshold of 2 points is difficult to interpret alongside a coefficient reported per 1,000 units of a predictor unless the reader also knows what a realistic predictor change means. When a different predictor contrast is more meaningful, the estimate and interval can be rescaled together. For a positive multiplier \(c\), multiplying each endpoint by \(c\) gives the interval for \(c\beta_1\), because every value in the original interval is multiplied by the same positive number. For a negative multiplier, the endpoints must be reversed after multiplication so they remain in increasing order.

When a P-Value Adds Information

A p-value belongs in a report when a hypothesis test is relevant to the question or reporting standard. It is especially important to identify the null hypothesis it addresses. In the examples above, the test asks whether the population slope is zero; it does not test whether the coefficient exceeds a practical benchmark. If the scientific question is whether an effect is at least a certain size, a test of zero alone does not answer it.

The p-value should generally be reported numerically to a sensible precision, or as a bound when it is very small. Avoid reporting \(p=0.000\): a rounded value of zero is not the computed probability. Also avoid giving only a threshold statement such as “\(p<0.05\),” when a more informative value is available. The coefficient and interval remain central whether the p-value is small or large.

When no hypothesis test is relevant, a p-value need not be forced into the summary. The estimate, confidence interval, units, and benchmark can still make the magnitude and uncertainty clear. Whatever is reported, the test’s null and the interval’s confidence level should be identifiable.

Common Reporting Mistakes

• Writing only “significant.” This gives neither the size nor the units of the estimated association. Report the coefficient and interval; add a p-value if relevant.
• Leaving out the units or predictor contrast. A coefficient such as 0.42 is uninterpretable without saying what outcome change it represents for what predictor increase.
• Calling a confidence interval a probability statement about the parameter. A 95% confidence interval does not mean there is a 95% probability that this fixed parameter lies inside this particular interval. Describe the repeated-sampling confidence procedure and give the interval in context.
• Comparing unlike scales. A benchmark in outcome units cannot be meaningfully compared with a coefficient on an incompatible scale. Convert the estimate and interval to the same predictor contrast and outcome units first.
• Turning association into causation. A regression coefficient from observational data describes an adjusted or unadjusted association according to the fitted model. It does not by itself show that changing the predictor would change the outcome.

The practical benchmark and the statistical summary answer different questions. The coefficient describes the fitted effect magnitude, the interval describes uncertainty, the p-value addresses a specified null hypothesis, and the benchmark supplies context for practical interpretation. Keeping these roles distinct makes even a short result more informative than a significance label.