Confidence Intervals as Inverted Coefficient Tests

One Interval, Many Related Tests

A confidence interval for a regression coefficient does more than report a range of plausible values. At a specified confidence level, it also summarizes the results of every corresponding two-sided test of that coefficient: a null value inside the interval is not rejected, while a null value outside it is rejected. This connection is called inverting a test.

The confidence level and test significance level must match. A two-sided test at significance level \(\alpha\) corresponds to a confidence interval with confidence level \(1-\alpha\). For example, a 5% two-sided test corresponds to a 95% confidence interval. The null value need not be zero: the same comparison works when a study asks whether a coefficient equals a specified nonzero benchmark.

Model, Test, and Matching Interval

Consider a regression model with a coefficient \(\beta_j\). Let \(b_j\) be its estimate, \(SE(b_j)\) its estimated standard error, and \(df\) the residual degrees of freedom used for conventional coefficient inference. To use the usual t test and interval, the model and its error assumptions must support that inference: the conditional mean is correctly represented by the regression model, errors are independent with mean zero and constant variance, and errors are normally distributed for exact t-based inference. The coefficient must also be estimable. In applications, diagnostics and study design help assess whether these assumptions are reasonable; they cannot be guaranteed just by calculating a statistic.

For a specified null value \(\beta_{j,0}\), the two-sided hypotheses are \(H_0:\beta_j=\beta_{j,0}\) and \(H_a:\beta_j\ne\beta_{j,0}\). Under the stated conditions, the test statistic under \(H_0\) is compared with a t distribution having \(df\) degrees of freedom. Write \(t_{1-\alpha/2,df}\) for the positive t quantile that leaves probability \(\alpha/2\) in its upper tail.

Definition: The conventional \(100(1-\alpha)\%\) confidence interval for \(\beta_j\) is \[ b_j \pm t_{1-\alpha/2,df}SE(b_j). \] It contains the coefficient values that are not rejected by the matching two-sided t test at significance level \(\alpha\), subject to the boundary convention described below.

For a particular null value, the test statistic and rejection rule are

T=\frac{b_j-\beta_{j,0}}{SE(b_j)}, \qquad \text{reject } H_0 \text{ if } |T|>t_{1-\alpha/2,df}.

The interval has lower and upper endpoints \(L=b_j-t_{1-\alpha/2,df}SE(b_j)\) and \(U=b_j+t_{1-\alpha/2,df}SE(b_j)\). The test and interval use the same estimate, standard error, degrees of freedom, and t critical value. Changing any of these, or using a different confidence level, breaks the direct comparison.

Why the Test and Interval Give the Same Decision

The equivalence follows by rearranging the test’s rejection condition. Since a standard error is positive, multiplying an inequality by it does not reverse the inequality.

Start with the two-sided rejection rule.
Reject when \(\left|(b_j-\beta_{j,0})/SE(b_j)\right|>t_{1-\alpha/2,df}\).

Multiply by the standard error.
This gives \(|b_j-\beta_{j,0}|>t_{1-\alpha/2,df}SE(b_j)\).

Write the absolute-value inequality as two cases.
Either \(b_j-\beta_{j,0}>t_{1-\alpha/2,df}SE(b_j)\), or \(b_j-\beta_{j,0}<-t_{1-\alpha/2,df}SE(b_j)\).

Rearrange to compare the null value with the endpoints.
The first case means \(\beta_{j,0}<L\); the second means \(\beta_{j,0}>U\). Thus rejection occurs exactly when the null value is outside the interval.

Equivalently, the null is not rejected when \(L<\beta_{j,0}<U\). If equality holds at an endpoint, then \(|T|=t_{1-\alpha/2,df}\), so the two-sided p-value equals \(\alpha\). Here the rejection rule is written with a strict inequality: reject when the p-value is less than \(\alpha\), and do not reject at the exact boundary. Some conventions reject when the p-value is less than or equal to \(\alpha\); at the boundary, that convention treats an endpoint null value as rejected. In ordinary applications, rounded output can obscure such an exact boundary case, so avoid deciding from rounded values when they appear to coincide.

Worked Examples

Worked Example: Testing Whether a Coefficient Is Zero

Suppose a hypothetical regression estimates the association between weekly exercise time and a fatigue score, adjusting for other specified predictors. The estimated exercise coefficient is \(b_j=1.80\) fatigue-score points per additional exercise hour per week, with \(SE(b_j)=0.75\) and \(df=18\). Assume the model diagnostics and study design make conventional t inference reasonable. Test \(H_0:\beta_j=0\) against a two-sided alternative at \(\alpha=0.05\), and compare it with the matching interval.

For \(df=18\), \(t_{0.975,18}\approx2.101\). The test statistic is

T=\frac{1.80-0}{0.75}=2.400.

Because \(|2.400|>2.101\), reject \(H_0\) at the 5% level. The matching 95% interval is

1.80\pm2.101(0.75) =1.80\pm1.576 =[0.224,\ 3.376].

Zero is outside this interval, so the interval gives the same decision. Under this fitted model, the estimated association is positive; the interval describes coefficient values not rejected by the corresponding two-sided procedure. The result alone does not establish that exercise causes a change in fatigue score.

Worked Example: Testing Against a Nonzero Benchmark

In a second hypothetical regression, a coefficient describes the difference in an outcome, measured in points, associated with a one-unit increase in a predictor while the other model predictors are held fixed. The estimate is \(b_j=5.40\) points per predictor unit, with \(SE(b_j)=1.20\) and \(df=24\). Suppose the assumptions for conventional t inference are reasonable, and the research question is whether the coefficient equals 4 points per unit. Test \(H_0:\beta_j=4\) against \(H_a:\beta_j\ne4\) at \(\alpha=0.05\).

For \(df=24\), the 95% interval critical value is \(t_{0.975,24}\approx2.064\). Relative to the nonzero null value, the test statistic is

T=\frac{5.40-4}{1.20}=\frac{1.40}{1.20}=1.167.

Since \(|1.167|<2.064\), do not reject the null at the 5% level. The corresponding 95% interval is

5.40\pm2.064(1.20) =5.40\pm2.477 =[2.923,\ 7.877].

The benchmark value 4 is inside the interval, agreeing with the test decision. This does not show that the coefficient is exactly 4; it means the data do not provide sufficient evidence against that value at this significance level. The interval also shows that zero is not among the values it contains, but zero is not the null value specified in this particular test.

Worked Example: Matching the Significance Level to the Interval

Suppose a hypothetical regression estimates an outcome change of \(b_j=3.10\) units per one-unit increase in a predictor, with \(SE(b_j)=0.80\) and \(df=15\). Assume conventional t inference is appropriate. Consider the two-sided test of \(H_0:\beta_j=1.50\). The decision depends on the chosen significance level, so compare the 5% test with its 95% interval and the 10% test with its 90% interval.

The test statistic is the same at either level:

T=\frac{3.10-1.50}{0.80} =\frac{1.60}{0.80}=2.000.

At \(\alpha=0.05\), \(t_{0.975,15}\approx2.131\). Since \(2.000<2.131\), do not reject. The matching 95% interval is

3.10\pm2.131(0.80) =3.10\pm1.705 =[1.395,\ 4.805],

which contains 1.50. At \(\alpha=0.10\), \(t_{0.95,15}\approx1.753\). Now \(2.000>1.753\), so reject at the 10% level. The matching 90% interval is

3.10\pm1.753(0.80) =3.10\pm1.402 =[1.698,\ 4.502],

which excludes 1.50. The differing decisions are not a contradiction: the 10% test uses a less stringent rejection threshold and a narrower, 90% interval. Comparing the 5% test with the 90% interval would not be a matched comparison.

What the Equivalence Does—and Does Not—Say

For a fixed model and coefficient, a \(100(1-\alpha)\%\) interval provides a compact way to read off two-sided tests at level \(\alpha\) for any candidate null value. A value just outside the interval is rejected at that level; a value inside is not. This is especially useful when the scientific benchmark is meaningful but nonzero, such as a target change or a minimum expected difference.

The connection does not turn an interval into a probability statement about the fixed coefficient. A 95% confidence procedure has 95% coverage over repeated samples when its assumptions hold; it does not mean there is a 95% probability that the coefficient lies in this particular calculated interval. Nor does an interval’s exclusion of zero establish practical importance: the size and units of the estimated effect matter, as do the context and uncertainty.

Common Mistakes

Testing zero when the question names another benchmark. The test statistic must subtract the null value actually specified: \(T=(b_j-\beta_{j,0})/SE(b_j)\), not automatically \(b_j/SE(b_j)\).
Pairing the wrong confidence level and significance level. A two-sided test at \(\alpha=0.05\) matches a 95% interval, not a 90% or 99% interval.
Using different calculations for the test and interval. The equivalence requires the same coefficient estimate, standard error, degrees of freedom, and conventional t method in both.
Calling non-rejection proof of equality. If the interval contains the null value, the test did not detect a difference from that value at the chosen level; it did not prove the coefficient equals it.

Key takeaway: For a conventional two-sided regression coefficient t test, compare the null value with the confidence interval using confidence level \(1-\alpha\). A value outside the interval is rejected at level \(\alpha\); a value inside is not rejected, apart from the exact-boundary convention.

Check Your Understanding

Use the matching confidence level whenever you compare an interval with a two-sided coefficient test.

A coefficient estimate is 2.4 with standard error 0.9. Write the test statistic for testing a null value of 1.0. Why would \(2.4/0.9\) answer a different question?
What confidence level matches a two-sided test at significance level 0.01?
A matching interval is [0.8, 3.6]. What is the test decision for a null value of 0 at the corresponding significance level? What about a null value of 2?
Why does a 95% interval containing a benchmark not establish that the coefficient equals that benchmark?
In your own words, explain why a 95% interval and a 10% two-sided test do not give the matching decision rule.