Why Coefficient Intervals Have Conditions
A conventional regression confidence interval combines an estimated coefficient with a standard error and a critical value. The calculation is straightforward, but its usual interpretation depends on how the observations vary around the regression model. In particular, the familiar small-sample interval relies on conditions about the errors: their independence, their variance, and their distribution.
These are conditions for the statistical calculation, not guarantees that the chosen predictors capture the science of a problem. A model can meet the error conditions and still omit an important variable or fail to answer a research question. Conversely, a scientifically motivated model may have errors whose dependence or spread makes a conventional interval unreliable.
The Model and the Conditions
Suppose there are \(n\) observations and a regression model with \(p\) coefficients, including the intercept. Write the response for observation \(i\) as \(Y_i\), and let its predictor values be collected in \(x_i\). The specified mean model is linear in its coefficients:
Here \(\beta\) is the vector of population coefficients and \(\varepsilon_i\) is the error for observation \(i\). The error represents the difference between the response and the model's conditional mean, not merely a recorded or fitted residual. For the conventional coefficient interval, the standard conditions are stated conditional on the predictor values.
The first condition concerns the mean structure: on average, the model's fitted mean agrees with the conditional mean of the response. Independence concerns how errors from different observations vary together. Constant variance, also called homoscedasticity, says that the error spread is the same at all predictor patterns. The normal-error condition is stronger: it specifies the shape of the conditional error distribution, not only its mean and variance.
These statements do not require the predictors themselves to be independent of one another. Predictors can be strongly related, provided the model matrix still has full column rank. Nor does independence mean that the responses must have the same mean: their means can differ according to their predictor values. The condition is about the errors after accounting for the specified mean model.
What the Conditions Support
Let \(\hat{\beta}_j\) be the estimated value of coefficient \(\beta_j\), and let \(SE(\hat{\beta}_j)\) be its conventional standard error. Under the stated conditions, a conventional \(100(1-\alpha)\%\) interval is
The full-rank condition makes the coefficients estimable, while \(n>p\) leaves positive residual degrees of freedom. Independence and constant variance support the conventional variance calculation. Normal errors allow an exact \(t\) reference distribution in finite samples. If the normality condition is not plausible, the exact small-sample justification no longer applies; whether an approximation is adequate depends on the data and setting.
A short derivation shows how these conditions enter. In matrix notation, the model is \(Y=X\beta+\varepsilon\). With full column rank, the least-squares estimator satisfies
If the errors are independent with mean zero and common variance \(\sigma^2\), then, conditional on \(X\), their covariance matrix is \(\sigma^2 I\). Therefore,
This is the variance structure used by the conventional standard errors. Under normal errors, \(\hat{\beta}\) is conditionally normal. Also, the fitted component and residual component are projections of the normal error vector onto perpendicular subspaces, so they are independent. The residual sum of squares divided by \(\sigma^2\) has a chi-square distribution with \(n-p\) degrees of freedom. Thus the coefficient's standardized error, divided by the estimated residual standard deviation, has a \(t\) distribution with \(n-p\) degrees of freedom. Replacing the unknown \(\sigma\) with its residual estimate gives the stated interval.
This derivation clarifies the roles of the conditions. Without independence or equal variance, the covariance calculation above does not generally hold. Without normal errors, the exact \(t\) argument does not generally hold, even if the variance formula remains appropriate.
Worked Example: Interpreting an Interval Under the Conditions
Suppose a hypothetical regression estimates the association between weekly practice time and a performance score. The estimated coefficient for practice time is \(2.40\) score points per hour, its conventional standard error is \(0.80\) points per hour, and the residual degrees of freedom are 18. Assume the design is full rank and the independence, constant-variance, and normal-error conditions are reasonable.
For a 95% interval, the \(t\) critical value with 18 degrees of freedom is approximately \(2.101\). The margin of error is
The interval is
Rounded suitably, the interval is about \(0.719\) to \(4.08\) score points per hour. Under the model and its conditions, the interval procedure has 95% coverage in repeated sampling. The calculation does not establish that practice time causes performance to increase: that is a scientific interpretation requiring evidence beyond these error conditions.
Independence: How Observations Were Generated Matters
Independence is often primarily a design question. Errors may be dependent when observations are repeated measurements on the same individual, collected close together in time, or grouped within a clinic, classroom, or household. Such data can share unmeasured influences even after the included predictors are accounted for. A residual plot may reveal some dependence, but it cannot replace understanding how the observations were collected.
Worked Example: Patients Grouped Within Clinics
Imagine a hypothetical study recording an outcome and several predictors for 48 patients: 12 patients at each of four clinics. A conventional regression analysis treats the 48 errors as independent. But patients at the same clinic may share staff, procedures, or local conditions. Those shared influences could make their errors more alike than errors from patients at different clinics.
The count of 48 observations does not by itself justify independence. The relevant question is whether, conditional on the predictors and specified model, the errors for patients within a clinic can reasonably be treated as unrelated. If clinic-level similarities remain in the errors, the independence condition is doubtful, and the conventional standard error and interval may not have their advertised performance. The data structure should therefore be considered before interpreting the interval.
This concern is about dependence among errors, not about whether the clinic predictor values are independent. Nor can a narrow interval make clustered observations independent.
Constant Variance: Look for Changing Error Spread
Constant variance means that the vertical spread of responses around the conditional mean is the same across predictor values or fitted means. It does not mean that the response itself has the same variance at every predictor value when its mean changes. A residual-versus-fitted plot is commonly used to look for changing spread: a widening or narrowing band can suggest nonconstant variance.
Residuals are estimates of errors, so a plot is a diagnostic rather than a direct observation of the error distribution. A pattern can raise concern, while an apparently even band cannot prove the condition. It is also important to look for a changing mean pattern separately: curvature in the residuals suggests that the specified mean model may not capture the relationship.
Worked Example: Comparing Residual Spread Across Fitted Values
In a hypothetical dataset, divide observations into two groups by fitted score: lower fitted scores and higher fitted scores. Suppose each group has five residuals. In the lower group, the residuals are \(-2, 0, 2, -1, 1\); in the higher group, they are \(-6, 0, 6, -3, 3\), all in score points. Each group has residuals summing to zero.
The sum of squared residuals in the lower group is
In the higher group it is
The average squared residuals are \(10/5=2\) and \(90/5=18\) score-points-squared, respectively. The higher-fitted group has much greater residual spread in this example, which would raise concern about constant variance. These calculations describe the hypothetical residuals; they are not a formal proof that the population error variances differ. In practice, residual plots and subject-matter context help assess whether the pattern is systematic.
Normal Errors: The Exact Small-Sample Condition
Normality is a condition on the errors conditional on the predictors. It does not say that the predictors must be normally distributed, and it does not require the observed responses to follow a single normal distribution when their means vary with predictors. Residuals are used as imperfect evidence about errors, with plots such as a normal quantile plot helping identify marked skewness or unusually extreme residuals.
Normality matters especially for the exact finite-sample \(t\) interval. When the sample is small, a strong departure from normal errors can make that reference distribution a poor description of coefficient uncertainty. With larger samples, approximate methods may sometimes work despite moderate nonnormality, but sample size alone does not guarantee a good approximation: influential observations or severe departures can still matter.
Worked Example: A Small-Sample Normality Concern
Consider a hypothetical regression with eight observations and an intercept. Its residuals, in outcome units, are \(-1,-1,-1,0,0,0,0,3\). They sum to zero, as residuals from a least-squares model with an intercept do. Seven residuals are between \(-1\) and \(0\), while one is \(3\), so the pattern has a pronounced positive tail rather than a roughly balanced shape.
With only eight observations, this single large residual is an important warning when relying on an exact small-sample \(t\) interval. It does not, by itself, prove that the errors are nonnormal: residuals are estimated, and eight values provide limited information about a distribution. The appropriate conclusion is that the normal-error condition deserves scrutiny, not that a particular alternative distribution has been established.
Conditions Are Not Claims of Scientific Correctness
A regression interval is conditional on a specified model and its assumptions. Independence, constant variance, and normal errors do not prove that the model includes the right predictors, captures the right functional relationship, avoids important omitted factors, or represents a causal effect. Those are separate questions about model construction and research design.
For example, an interval can be calculated correctly for a linear association between two measured variables even if the relationship is curved in the population. Or a model may satisfy the error conditions while an unmeasured factor influences both a predictor and the outcome. In either case, the interval describes uncertainty for the coefficient in the specified model; it does not certify a complete scientific explanation.
A useful assessment combines knowledge of how the observations were collected with examination of residuals and the stated mean model. Diagnostics can identify evidence against assumptions, but passing a diagnostic does not prove an assumption. Conclusions should be conditional and measured: “the conventional interval is reasonable if these conditions are adequate,” rather than “the model is correct because the interval was computed.”
Common Mistake
Wrong reasoning: “The regression software produced a coefficient interval, so the assumptions must have been satisfied.” Software can perform the calculation without knowing whether observations are dependent, whether residual spread changes, or whether the error distribution is plausible.
Correction: The interval is a numerical result under specified conditions, not a certificate that those conditions hold. Consider the data-generation process, the residual patterns, the sample size, and the scientific question before deciding how to interpret it.
Check Your Understanding
Try these questions using the definitions and examples above.
- In a regression with several predictors, does the independence condition require the predictors themselves to be independent? Explain what it concerns instead.
- What feature of a residual-versus-fitted plot might raise concern about constant variance?
- Why does normality support an exact small-sample \(t\) interval, and does it require the predictors to be normally distributed?
- A study records repeated measurements from each person. What feature of the data collection makes the independence condition worth examining?
- Explain why a conventional coefficient interval does not, by itself, establish that a predictor causes a change in the response.