Confidence Intervals With Robust Standard Errors

Why Use a Robust Confidence Interval?

A regression coefficient is an estimate of an unknown feature of a population relationship. A confidence interval pairs that estimate with a measure of uncertainty, giving a range of coefficient values compatible with the data and the interval procedure. When the variability of the regression errors is not constant across predictor values, a conventional standard error can give a misleading measure of uncertainty. A heteroskedasticity-consistent standard error provides an alternative for constructing coefficient intervals.

The key point is that the robust standard error changes the estimated uncertainty, not the least-squares coefficient itself. Its use does not make an interval exact, nor does it guarantee good coverage in every sample. The usual justification is a large-sample approximation under stated conditions.

Model and Conditions

Consider a multiple linear regression with \(n\) observations and \(p\) coefficients, including the intercept. Let \(Y_i\) be the response for observation \(i\), and let \(x_i\) be the vector of that observation’s predictor values, including a 1 for the intercept. The model for the conditional mean is

Here \(\beta\) is the vector of population regression coefficients, \(X\) is the design matrix containing the predictor values, and \(\varepsilon_i\) is the error. The condition \(E(\varepsilon_i\mid X)=0\) says that the specified linear mean model is correct, conditional on the predictors. Unequal error variance is allowed: \(\operatorname{Var}(\varepsilon_i\mid X)=\sigma_i^2\) may differ across observations.

These conditions do not require normally distributed errors or equal error variances. They do require more than merely selecting a robust standard error: dependence between observations, a misspecified conditional mean, or influential observations can undermine the justification. In particular, “robust” here means robust to unequal error variances under the stated conditions, not robust to every possible problem with the data or model.

Constructing the Interval

Let \(\hat\beta_j\) be the least-squares estimate of coefficient \(\beta_j\). Choose a heteroskedasticity-consistent covariance estimator, such as one of the HC estimators, and take the square root of its \(j\)th diagonal entry to obtain the robust standard error \(SE_{\mathrm{robust}}(\hat\beta_j)\). For a two-sided confidence level \(100(1-\alpha)\%\), the large-sample interval is

where \(z_{1-\alpha/2}\) is the \(1-\alpha/2\) quantile of the standard normal distribution. For example, \(z_{0.975}\approx1.96\) gives a 95% interval, and \(z_{0.95}\approx1.645\) gives a 90% interval. The same construction can be written as a lower endpoint and an upper endpoint:

The interval is centered at the fitted coefficient. Its margin of error is the normal critical value multiplied by the robust standard error. A different HC estimator can produce a different standard error and therefore a different interval, even though the fitted coefficient stays the same.

Why the Normal Reference Distribution Is Approximate

The rationale for the normal critical value comes from the sampling behavior of the least-squares estimator. In matrix notation, the estimator is \(\hat\beta=(X^\top X)^{-1}X^\top Y\). Substituting \(Y=X\beta+\varepsilon\) gives an exact algebraic expression for its estimation error:

Conditional on the predictors, if the errors are independent with variances collected in the diagonal matrix \(\Omega\), the covariance of this expression is

When the error variances differ, this covariance is generally not the equal-variance covariance formula. Heteroskedasticity-consistent estimators estimate the relevant covariance without requiring all the \(\sigma_i^2\) to be equal. Under the regularity conditions above, the estimator, after scaling by its standard error, is approximately standard normal for large samples. The normal quantiles then give the interval’s approximate repeated-sampling coverage.

This argument explains both the value and the limit of the method. The covariance estimate corrects the uncertainty calculation for unequal variances, but the normal approximation is a large-sample result. It is not an exact finite-sample statement. A small or unusual sample may have coverage noticeably different from the advertised level. This tutorial uses the large-sample normal reference explicitly; using a \(t\) critical value in some software conventions does not, by itself, make robust inference exact under heteroskedasticity.

Reading an Interval Carefully

A coefficient interval should always be interpreted with its predictor, response, units, and conditioning context. For a slope in a multiple regression, the coefficient describes the change in the conditional mean response associated with a one-unit increase in that predictor, with the other included predictors held fixed. The interval gives uncertainty about that coefficient. It does not describe the range of individual responses, and it does not guarantee that the fitted relationship is an adequate description of the data.

Whether an interval includes zero can help summarize which coefficient values are compatible with the data under the same model and inference procedure. It is not a probability statement about the coefficient or a direct measure of practical importance. A narrow interval around a small effect may be statistically informative but practically unimportant; a wide interval can leave substantial uncertainty even when its point estimate is sizable.

Common Mistakes and Limits

• Using the conventional standard error after detecting unequal variance. A robust interval uses the robust standard error in the margin of error. Substituting the conventional standard error changes the procedure.
• Thinking robust standard errors change the coefficient estimate. In ordinary least squares, the fitted coefficient remains \(\hat\beta_j\); the robust covariance estimate changes the estimated uncertainty around it.
• Calling the interval exact. Under heteroskedasticity, the normal-based robust interval is generally justified by a large-sample approximation, not by an exact finite-sample distribution.
• Giving the interval a probability interpretation. In frequentist terms, the coefficient is fixed and the endpoints vary from sample to sample. The confidence level describes the long-run coverage of the procedure under its conditions.
• Treating a robust interval as a remedy for every model problem. It addresses uncertainty from unequal variances under suitable conditions. It does not, on its own, validate the conditional mean model or resolve dependence between observations.

When reporting a robust interval, identify the coefficient and its units, the confidence level, the HC standard-error version, and the reference distribution used. State that coverage is approximate when using the large-sample normal method. Such details let readers distinguish the estimated relationship from the uncertainty calculation and understand what the interval does, and does not, support.