Robust t Intervals for Single Coefficients

Why Use a Robust Interval for One Coefficient?

A regression coefficient is an estimate, not a known quantity. Its standard error describes sampling uncertainty, and a confidence interval uses that uncertainty to report a range of coefficient values compatible with the data and an inferential procedure. In applications where the conditional variance of the regression error may differ across observations, a conventional homoskedastic standard error can be unreliable. An HC (heteroskedasticity-consistent) standard error provides an alternative that does not require constant conditional error variance.

This tutorial concerns inference about one coefficient at a time. The basic calculation resembles a familiar coefficient-over-standard-error calculation, but the standard error is robust and the reference distribution is large-sample normal. The word “t” is often used for the ratio itself; it does not mean that the ratio has an exact finite-sample Student t distribution under heteroskedasticity.

Model and Robust Standard Error

Let $Y_i$ be an outcome and $X_i$ a $k$-component vector of regressors, including an intercept if the model has one. Write the linear regression model as

Y_i = X_i'\beta + u_i,\qquad i=1,\ldots,n.

Here $\beta$ is the coefficient vector and $u_i$ is the regression error. The OLS estimate is $\hat{\beta}$. For the large-sample results below, suppose observations are independent and identically distributed, the regressors have finite second moments and a nonsingular population second-moment matrix, the conditional mean is correctly specified so that $E[u_i\mid X_i]=0$, and suitable finite-moment conditions hold. These conditions allow the OLS estimate to be consistent and asymptotically normal. They do not require $Var(u_i\mid X_i)$ to be constant. In particular, heteroskedasticity is allowed.

The HC0 sandwich covariance estimator uses the OLS residuals $\hat{u}_i=Y_i-X_i'\hat{\beta}$:

\widehat{Var}_{HC0}(\hat{\beta}) = (X'X)^{-1} \left(\sum_{i=1}^n X_iX_i'\hat{u}_i^2\right) (X'X)^{-1}.

For coefficient $j$, its HC0 standard error is the square root of the $j$th diagonal element of this estimated covariance matrix. It estimates the sampling uncertainty of $\hat{\beta}_j$ while allowing the conditional variance of $u_i$ to vary with $X_i$. Other HC estimators modify the covariance estimate; whatever HC estimator is used, its matching diagonal element must be used for the standard error. The interval and test calculations below then proceed in the same way.

Definition: A large-sample robust $100(1-\alpha)\%$ confidence interval for coefficient $\beta_j$ is $$ \hat{\beta}_j \ \pm\ z_{1-\alpha/2}SE_{HC}(\hat{\beta}_j), $$ where $SE_{HC}(\hat{\beta}_j)$ is the selected HC standard error and $z_{1-\alpha/2}$ is the $1-\alpha/2$ quantile of the standard normal distribution. “Large-sample” describes the approximation used to calibrate the interval; it is not a claim that the interval has exact finite-sample coverage.

Why the Normal Reference Distribution Appears

Under the stated regularity conditions and a consistent HC covariance estimator, the estimated standard deviation puts the coefficient’s estimation error on a scale that is asymptotically standard normal. For a specified value $b_0$, the corresponding studentized statistic is

Z_j = \frac{\hat{\beta}_j-b_0}{SE_{HC}(\hat{\beta}_j)} \ \overset{d}{\longrightarrow}\ N(0,1) \quad\text{under }H_0:\beta_j=b_0.

The arrow means that the statistic’s distribution approaches the standard normal distribution as the sample size grows. It does not say that the statistic is exactly normal for a particular finite sample. Nor does the HC standard error make finite-sample distributional assumptions disappear: the justification is asymptotic.

A one-coefficient robust test is the one-dimensional case of a Wald test. Its statistic is $Z_j^2$, which converges to a chi-squared distribution with one degree of freedom under the null. Equivalently, using the signed statistic $Z_j$, a two-sided test compares $|Z_j|$ with a standard normal critical value. This equivalence gives a direct way to construct the corresponding confidence interval.

Worked Example: A 95% Robust Interval for a Slope

Suppose a hypothetical regression of hourly earnings on years of schooling and other controls uses 240 independent observations. The estimated schooling coefficient is $0.42$ dollars per hour for each additional year of schooling, and its HC standard error is $0.16$ dollars per hour per year. Assume the regression and large-sample conditions stated above are appropriate. Construct a 95% robust interval.

For a 95% interval, $\alpha=0.05$, and the standard normal critical value is $z_{0.975}=1.96$. Thus

0.42 \pm 1.96(0.16) = 0.42 \pm 0.3136 = (0.1064,\ 0.7336).

The endpoints are in dollars per hour per additional year of schooling. In this hypothetical regression, the large-sample robust 95% interval runs from about $0.106$ to $0.734$ dollars per hour per year. Under repeated sampling and the stated conditions, the procedure has approximately 95% coverage. This is not a statement that there is a 95% probability that the fixed coefficient lies in this particular calculated interval.

Two-Sided Tests for a Single Coefficient

To test $H_0:\beta_j=b_0$ against the two-sided alternative $H_a:\beta_j\ne b_0$, calculate

Z_j=\frac{\hat{\beta}_j-b_0}{SE_{HC}(\hat{\beta}_j)}.

At significance level $\alpha$, reject $H_0$ when $|Z_j|>z_{1-\alpha/2}$. The large-sample two-sided p-value is

p=2\left[1-\Phi(|Z_j|)\right],

where $\Phi$ is the standard normal cumulative distribution function. This p-value measures how unusual a statistic at least as far from zero as the observed one would be under the null and the reference distribution. It is not the probability that the null hypothesis is true.

Worked Example: Testing a Nonzero Slope Value

In a hypothetical regression, an estimated treatment-intensity coefficient is $0.31$ outcome points per intensity unit, with HC standard error $0.12$ points per unit. Consider the two-sided hypotheses $H_0:\beta_j=0.20$ and $H_a:\beta_j\ne0.20$, at $\alpha=0.05$. Assume the model, independence, and large-sample conditions are suitable.

The robust test statistic is

Z_j = \frac{0.31-0.20}{0.12} = \frac{0.11}{0.12} \approx 0.9167.

The 5% two-sided critical value is $1.96$, and $|0.9167|<1.96$, so this test does not reject $H_0$. The normal-reference p-value is approximately

p=2[1-\Phi(0.9167)]\approx 0.359.

The data do not provide strong evidence against a coefficient of $0.20$ outcome points per intensity unit at the 5% level. This result does not establish that the coefficient equals $0.20$; it describes the evidence from this test.

Intervals and Tests Are Two Views of the Same Calculation

For a two-sided test of $H_0:\beta_j=b_0$, non-rejection at level $\alpha$ means

\left| \frac{\hat{\beta}_j-b_0}{SE_{HC}(\hat{\beta}_j)} \right| \le z_{1-\alpha/2}.

Because the standard error is positive, multiplying through by it and rearranging gives

\hat{\beta}_j-z_{1-\alpha/2}SE_{HC}(\hat{\beta}_j) \le b_0 \le \hat{\beta}_j+z_{1-\alpha/2}SE_{HC}(\hat{\beta}_j).

The values of $b_0$ that are not rejected are therefore exactly the values inside the corresponding large-sample robust confidence interval. This inversion explains why the interval and the two-sided test must agree when they use the same HC standard error, normal reference distribution, and significance level.

Worked Example: Checking a Test Against Its Interval

Suppose a hypothetical model estimates that one additional weekly training session is associated with $0.55$ more performance points, with an HC standard error of $0.24$ points per session. Consider a two-sided test of $H_0:\beta_j=0$ at the 5% level. Assume the regression and large-sample conditions are appropriate.

The test statistic is

Z_j=\frac{0.55-0}{0.24}\approx2.292.

Since $2.292>1.96$, the test rejects zero at the 5% level. The matching 95% interval is

0.55\pm1.96(0.24) = 0.55\pm0.4704 = (0.0796,\ 1.0204).

Zero is outside this interval, confirming the test decision. The interval describes the estimated association in performance points per additional weekly session; by itself, it does not show that increasing training sessions causes a performance change.

Why Not Use an Exact Student t Distribution?

In some classical regression settings, a coefficient divided by its estimated standard error has an exact Student t distribution in finite samples. That exact result relies on stronger distributional and variance assumptions than the robust large-sample procedure requires. Under heteroskedasticity, the HC standard error estimates a more general sampling variance, and the resulting studentized coefficient is generally not exactly Student t in a finite sample. Its justification is its asymptotic standard normal distribution.

For that reason, the ratio is sometimes informally called a “robust t-statistic,” even when software reports a normal-reference p-value. The label for the ratio does not determine its reference distribution. The researcher should check which critical values and p-value calculation are being used, rather than assume that a Student t distribution with some degrees of freedom is exact for robust inference.

Worked Example: The Effect of Choosing a Reference Distribution

A hypothetical regression with 64 independent observations estimates a coefficient of $0.55$ outcome points per unit, with HC standard error $0.24$. Suppose the model and large-sample conditions are suitable. For the usual large-sample robust 95% interval, use $z_{0.975}=1.96$:

0.55\pm1.96(0.24) = (0.0796,\ 1.0204).

For comparison only, a Student t critical value with 60 degrees of freedom is about $2.000$. Substituting it gives

0.55\pm2.000(0.24) = 0.55\pm0.480 = (0.0700,\ 1.030).

The endpoints differ slightly because the critical values differ. The second calculation is not automatically preferable, and the 60 degrees of freedom do not make the HC statistic exactly Student t. The large-sample robust procedure uses the normal critical value because its result is asymptotic normality. A Student t reference would require a separate justification, not merely a finite sample size or a software label.

Common Pitfalls and Practical Interpretation

Using the conventional standard error by habit. A robust interval requires an HC standard error, not the default homoskedastic standard error. The coefficient estimate may stay the same while its estimated uncertainty changes.
Calling the robust statistic exactly t-distributed. Dividing by an estimated standard error creates a studentized statistic, but does not establish an exact Student t distribution. For the HC procedure here, use the large-sample normal reference unless another finite-sample result is justified.
Interpreting non-rejection as proof of the null. A large p-value means the data do not provide sufficient evidence against the specified null under the test procedure. It does not show that the null is true.
Ignoring the coefficient’s units or interpretation. An interval for a slope is expressed in outcome units per unit change in the regressor, conditional on the other included regressors. Robust standard errors address variance estimation; they do not turn an association into a causal effect.

These intervals are useful when the inferential target is one particular coefficient—for example, a slope or an indicator-variable coefficient—and the observations meet the independence and regularity conditions. They report uncertainty without requiring a constant conditional error variance. They do not, by themselves, answer a question about several coefficients jointly; that requires a joint test rather than separate one-coefficient intervals.

Key takeaway: For one coefficient, pair its estimate with the matching HC standard error, use the standard normal critical value for a large-sample interval or two-sided test, and interpret the result as asymptotic rather than an exact finite-sample Student t procedure.

Check Your Understanding

Work through these questions using the definitions and results above.

A coefficient estimate is $1.2$ units per regressor unit, and its HC standard error is $0.4$. Calculate the large-sample 95% interval and state its units.
For $H_0:\beta_j=0$, an estimate is $0.18$ and the HC standard error is $0.10$. Calculate the statistic and decide whether to reject at the 5% two-sided level.
Explain why an HC coefficient-over-standard-error statistic is not generally exactly Student t distributed in a finite sample.
What relationship should hold between a two-sided 5% test of $H_0:\beta_j=b_0$ and the corresponding 95% robust interval?
A robust test has a p-value of $0.28$. What does this say about evidence against the null, and what does it not establish?