Constructing a Confidence Interval for a Coefficient

From a Coefficient Estimate to an Interval

A regression coefficient is estimated from a sample, so the reported estimate is not usually the only value compatible with the data and model. Its standard error describes the sampling variability of the estimate. A confidence interval combines these two quantities to show a range of plausible values for the population coefficient, while accounting for the fact that the standard error is estimated.

For a two-sided interval, start with the coefficient estimate and extend a margin in both directions. That margin is the coefficient's standard error multiplied by a critical value from a \(t\) distribution. The critical value depends on the desired confidence level and the model's residual degrees of freedom. In the examples below, the appropriate critical value is supplied so that the focus remains on constructing the interval.

Here, \(\hat{\beta}_j\) is the sample estimate of the population coefficient \(\beta_j\), and \(SE(\hat{\beta}_j)\) is the estimated standard deviation of that estimate over repeated samples under the model. The standard error is not the same as the residual standard deviation: the former measures uncertainty in a coefficient estimate, while the latter measures the spread of the response around the fitted regression model.

Why the Critical Value Multiplies the Standard Error

The formula is based on a standardized coefficient estimate. For the usual regression \(t\) interval, suppose the response follows a linear regression model with independent errors having mean zero, a common variance \(\sigma^2\), and a normal distribution conditional on the predictors. The model must be correctly specified for the coefficient being estimated, and the predictor values must allow the coefficients to be estimated. Under these conditions, the estimate minus its population coefficient, divided by its estimated standard error, has a \(t\) distribution with the model's residual degrees of freedom.

In simple linear regression, the fitted model has an intercept and a slope, so the residual degrees of freedom are \(n-2\). In a multiple regression with \(q\) estimated coefficients in total, including the intercept, they are \(n-q\). The appropriate \(t^*\) is the value that leaves equal tail areas of \(\alpha/2\) in a two-sided interval with confidence level \(1-\alpha\).

To see how this gives the interval, take the central \(1-\alpha\) part of this \(t\) distribution. By the definition of the critical value, the standardized estimate lies between \(-t^*\) and \(t^*\) with probability \(1-\alpha\). Multiplying all parts of that inequality by the positive standard error and then isolating \(\beta_j\) gives the interval formula.

The \(t\) distribution accounts for estimating the error variance rather than knowing it. The resulting interval is centered on the coefficient estimate, and its margin of error is \(t^*SE(\hat{\beta}_j)\). A larger standard error or critical value produces a wider interval.

Reading the Interval in Context

An interval should be reported with the coefficient's context and units. A slope's units are response units per predictor unit. In a multiple regression, a coefficient for one predictor describes its association with the response while the other included predictors are held fixed. That adjusted interpretation does not, by itself, establish a causal effect.

The interval's endpoints represent the range of coefficient values produced by the calculation. In repeated use of this procedure under its assumptions, intervals constructed this way have the stated confidence level. The confidence level does not mean there is a probability of 95% that a fixed population coefficient is inside this particular calculated interval.

A useful reading is therefore: “Using this model and procedure, the data give a 95% confidence interval from [lower endpoint] to [upper endpoint] for the population coefficient, in [units and context].” Do not omit the coefficient's meaning, especially when a regression output contains several coefficients.

Choosing the Inputs and Checking the Arithmetic

Before calculating, identify which coefficient the question asks about and use that coefficient's own standard error. Then identify the confidence level and the model's residual degrees of freedom. The critical value must match both. The next step is arithmetic: multiply the standard error by \(t^*\), subtract the margin from the estimate for the lower endpoint, and add it for the upper endpoint.

The critical value is positive, so a correctly calculated two-sided interval is centered on the estimate and has the same margin of error on each side. This gives a quick check: the estimate should be the midpoint of the two endpoints, apart from rounding. Also check that both endpoints have the coefficient's units. A slope interval should not be reported only in response units; it is measured in response units per predictor unit.

Common Mistake: Using the Wrong Margin

A frequent error is to multiply the standard error by \(1.96\) automatically. That value is a familiar standard normal critical value, but the conventional regression coefficient interval uses a \(t\) critical value because the error variance is estimated. In particular, the \(t\) critical value depends on the residual degrees of freedom and is generally larger than the corresponding normal critical value for finite degrees of freedom.

For the greenhouse example, using \(1.96\) would give a margin of \(1.96(0.62)=1.2152\), rather than the correct \(1.38136\) from the supplied \(t^*=2.228\). It would therefore produce a narrower interval than the specified \(t\) procedure. A different mistake is to report \(1.80\pm0.62\), using one standard error as though it were the confidence margin. The correction is to use the matching critical value times the standard error, and to show the resulting endpoints.