Consistency of HC0 Under Conditional Heteroskedasticity

Why HC0 Consistency Matters

The OLS sandwich covariance replaces the constant-variance formula with a term that allows the error variance to depend on the regressors. But its central ingredient, the “meat” of the sandwich, uses squared OLS residuals rather than the unobserved squared errors. Consistency therefore requires two facts: the sample average of the error-based terms must converge to a population quantity, and substituting residuals for errors must not change that limit.

This tutorial establishes those facts for independent and identically distributed observations with a correctly specified conditional mean. Conditional heteroskedasticity is allowed: the conditional variance of the error may vary with the regressors. The result concerns large-sample covariance estimation; it does not assert that HC0 gives an unbiased covariance estimate in every finite sample.

Model, Assumptions, and the HC0 Estimator

For observation $i$, let $y_i$ be the outcome and let $x_i$ be a $k$-component column vector of regressors, including an intercept if the model calls for one. Write the linear conditional-mean model as

y_i=x_i'\beta+u_i,

where $\beta$ is a fixed $k$-component parameter vector and $u_i$ is the error. The observations $(y_i,x_i)$ are iid, $k$ is fixed as the sample size $n$ grows, and the regressors are exogenous in the conditional-mean sense:

E[u_i\mid x_i]=0.

No constant conditional variance is assumed. Define $Q=E[x_i x_i']$ and $\Omega=E[u_i^2x_i x_i']$. Assume $Q$ is finite and positive definite, and assume the moments $E[\|x_i\|^4]<\infty$ and $E[u_i^2\|x_i\|^2]<\infty$. These conditions provide finite moments for the sample averages used below and for the OLS limiting distribution. The positive-definiteness condition rules out a singular population design matrix.

Definition: Let $\hat{\beta}$ be the OLS estimator and let $\hat{u}_i=y_i-x_i'\hat{\beta}$ be its residual. The HC0 covariance estimator is $$ \widehat{V}_{HC0}=(X'X)^{-1}\left(\sum_{i=1}^n x_i x_i'\hat{u}_i^2\right)(X'X)^{-1}, $$ where $X$ is the matrix with rows $x_i'$. Its target is the covariance of $\sqrt{n}(\hat{\beta}-\beta)$, namely $Q^{-1}\Omega Q^{-1}$. Equivalently, $n\widehat{V}_{HC0}$ estimates that limiting covariance, while $\widehat{V}_{HC0}$ is on the scale of the covariance of $\hat{\beta}$.

The population matrix $\Omega$ averages the error variance contribution across regressor values. In particular, since $E[u_i\mid x_i]=0$, it can be written as

\Omega=E\!\left[E[u_i^2\mid x_i]\,x_i x_i'\right].

Thus, if the conditional variance changes with $x_i$, that variation remains represented in $\Omega$. The sandwich does not require the conditional variance to be constant or to follow a particular parametric form.

Worked Example: A Changing Conditional Variance

Suppose $x_i=(1,z_i)'$, where $z_i$ takes the values $-1$ and $1$ with equal probability. Let $y_i=1+2z_i+u_i$, with $E[u_i\mid z_i]=0$ and $\operatorname{Var}(u_i\mid z_i)=2+z_i$. The conditional error variance is 1 when $z_i=-1$ and 3 when $z_i=1$, so it is not constant. The regressors and errors have finite moments, and both values of $z_i$ occur with positive probability, giving a nonsingular design limit.

First calculate $Q$. Since $E[z_i]=0$ and $z_i^2=1$,

Q=E[x_i x_i'] = \begin{pmatrix} 1 & E[z_i]\\ E[z_i] & E[z_i^2] \end{pmatrix} = \begin{pmatrix} 1&0\\ 0&1 \end{pmatrix}.

Next, use the conditional variance to calculate $\Omega$:

\Omega =E[(2+z_i)x_i x_i'] = \begin{pmatrix} E[2+z_i] & E[(2+z_i)z_i]\\ E[(2+z_i)z_i] & E[(2+z_i)z_i^2] \end{pmatrix} = \begin{pmatrix} 2&1\\ 1&2 \end{pmatrix}.

Because $Q$ is the identity matrix, $Q^{-1}\Omega Q^{-1}=\Omega$. Therefore $n\widehat{V}_{HC0}$ converges to the displayed matrix. Its diagonal entries are the limiting variances for the scaled intercept and slope estimators, and the off-diagonal entry is their limiting covariance. The variance difference across regressor values is included rather than replaced by a single common error variance.

Why the Residual-Based Meat Has the Right Limit

The OLS sandwich derivation gives the relevant asymptotic covariance in terms of $Q$ and $\Omega$. To establish HC0 consistency, it remains to show that its sample ingredients converge to these population matrices. The law of large numbers applies to the design average and to the error-based meat:

\frac{1}{n}X'X=\frac{1}{n}\sum_{i=1}^n x_i x_i'\ \xrightarrow{p}\ Q, \qquad \frac{1}{n}\sum_{i=1}^n x_i x_i'u_i^2\ \xrightarrow{p}\ \Omega.

The moment assumptions make the entries of these averages integrable. The remaining issue is that HC0 uses $\hat{u}_i$, not $u_i$. Let $d=\hat{\beta}-\beta$. Since $\hat{u}_i=u_i-x_i'd$, the difference in the two meat matrices is

\frac{1}{n}\sum_{i=1}^n x_i x_i'(\hat{u}_i^2-u_i^2) = \frac{1}{n}\sum_{i=1}^n x_i x_i'\left[-2u_i(x_i'd)+(x_i'd)^2\right].

For any matrix norm compatible with the vector norm, the norm of this difference is bounded above by

2\|d\|\left(\frac{1}{n}\sum_{i=1}^n\|x_i\|^3|u_i|\right) + \|d\|^2\left(\frac{1}{n}\sum_{i=1}^n\|x_i\|^4\right).

The moment assumptions ensure that both sample averages in parentheses are $O_p(1)$. For the first, the Cauchy–Schwarz inequality gives $E[\|x_i\|^3|u_i|]\leq \{E[\|x_i\|^4]E[\|x_i\|^2u_i^2]\}^{1/2}<\infty$; the second has finite expectation directly. OLS consistency gives $\|d\|=o_p(1)$, so the entire bound is $o_p(1)$. Thus replacing the unobserved errors by residuals changes the normalized meat by a term that vanishes in probability.

Result: Under the stated iid, exogeneity, fixed-dimension, moment, and nonsingularity conditions, $$ \frac{1}{n}\sum_{i=1}^n x_i x_i'\hat{u}_i^2\xrightarrow{p}\Omega \qquad\text{and}\qquad n\widehat{V}_{HC0}\xrightarrow{p}Q^{-1}\Omega Q^{-1}. $$

To see the final step, write the estimator as

n\widehat{V}_{HC0} = \left(\frac{X'X}{n}\right)^{-1} \left(\frac{1}{n}\sum_{i=1}^n x_i x_i'\hat{u}_i^2\right) \left(\frac{X'X}{n}\right)^{-1}.

The first and last factors converge to $Q^{-1}$, because $X'X/n$ converges to the positive definite matrix $Q$. The middle factor converges to $\Omega$, as just shown. Continuity of matrix multiplication and inversion at these limits gives the result. In the usual asymptotic-normality formulation, $Q^{-1}\Omega Q^{-1}$ is the covariance matrix of the limiting distribution of $\sqrt{n}(\hat{\beta}-\beta)$.

Worked Example: Calculating HC0 From a Small Hypothetical Sample

Consider a through-the-origin regression with scalar regressor $x_i$, so $y_i=x_i\beta+u_i$. Suppose iid observations have exogenous errors and a conditional variance that depends on $x_i$; for example, $E[u_i\mid x_i]=0$ and $\operatorname{Var}(u_i\mid x_i)=x_i^2$, with $x_i$ taking values 1, 2, and 3 with positive probability. These conditions permit heteroskedasticity and a nonsingular scalar design limit. For the following hypothetical sample, the observed pairs are $(1,2),(2,5),(3,7)$. The sample is used to illustrate the HC0 calculation, not to assess whether the assumptions hold in a particular application.

For a through-the-origin regression, the OLS estimate is

\hat{\beta}=\frac{\sum_i x_i y_i}{\sum_i x_i^2} =\frac{1(2)+2(5)+3(7)}{1^2+2^2+3^2} =\frac{33}{14}\approx 2.357.

The residuals are $\hat{u}_1=2-33/14=-5/14$, $\hat{u}_2=5-66/14=2/7$, and $\hat{u}_3=7-99/14=-1/14$. The scalar HC0 estimate is

\widehat{V}_{HC0} = \frac{\sum_i x_i^2\hat{u}_i^2}{(\sum_i x_i^2)^2} = \frac{1(25/196)+4(16/196)+9(1/196)}{14^2} = \frac{98/196}{196} = \frac{1}{392} \approx 0.002551.

This number is the sample HC0 covariance estimate for the slope, in squared slope units. The calculation shows how the squared residuals enter the meat; it does not claim that this three-observation estimate is close to its population limit. The consistency result concerns the behavior of this estimator as $n$ grows under the stated conditions.

Worked Example: A Two-Group Regressor Design

Let $x_i=(1,z_i)'$, where $z_i$ is 0 or 1 with equal probability. Suppose $E[u_i\mid z_i]=0$, the conditional error variances are 1 at $z_i=0$ and 4 at $z_i=1$, and $y_i=x_i'\beta+u_i$. This is an iid, finite-moment setup with both regressor values represented in the population, so the design limit is nonsingular. The variance changes across groups.

The design and meat limits are

Q= \frac{1}{2} \begin{pmatrix}1&0\\0&0\end{pmatrix} + \frac{1}{2} \begin{pmatrix}1&1\\1&1\end{pmatrix} = \begin{pmatrix}1&1/2\\1/2&1/2\end{pmatrix}, \qquad \Omega= \frac{1}{2} \begin{pmatrix}1&0\\0&0\end{pmatrix} + \frac{4}{2} \begin{pmatrix}1&1\\1&1\end{pmatrix} = \begin{pmatrix}5/2&2\\2&2\end{pmatrix}.

The determinant of $Q$ is $1/2-1/4=1/4>0$, and its inverse is

Q^{-1}= \begin{pmatrix}2&-2\\-2&4\end{pmatrix}.

Multiplying gives the limiting covariance:

Q^{-1}\Omega Q^{-1} = \begin{pmatrix}2&-2\\-2&10\end{pmatrix}.

Accordingly, $n\widehat{V}_{HC0}$ converges to this matrix. The intercept and slope components refer to the conditional mean at $z=0$ and the change in conditional mean from $z=0$ to $z=1$, respectively. The off-diagonal term can be negative even though the matrix is a valid covariance matrix; its determinant is $20-4=16>0$.

Common Mistakes and Practical Meaning

Assuming HC0 needs homoskedasticity. It does not. Its meat converges to $E[u_i^2x_ix_i']$, which includes variation in the conditional error variance.
Treating residuals as the errors. They are not equal in a finite sample. Consistency follows because the residual-error difference in the normalized meat vanishes as the OLS coefficient error goes to zero under the moment conditions.
Calling the finite-sample covariance exactly correct. The result is an asymptotic consistency statement. It does not say that HC0 is unbiased or exact in a small sample.
Ignoring exogeneity or a singular design. If $E[u_i\mid x_i]\neq 0$, the OLS target and error-based covariance argument may change. If $Q$ is singular, the inverse in the limiting sandwich does not exist.

The practical value of the result is that analysts need not correctly specify the conditional variance function to obtain a consistent large-sample covariance estimate for OLS under these conditions. They do still need a suitable conditional mean, independent sampling, adequate moments, and enough population variation in the regressors. Robustness to heteroskedasticity is not robustness to every form of model failure.

Check Your Understanding

Use the definitions and conditions above to answer the following questions.

Why does the HC0 consistency result allow $\operatorname{Var}(u_i\mid x_i)$ to vary across regressor values?
What population matrix is estimated by the normalized residual-based meat, $n^{-1}\sum_i x_i x_i'\hat{u}_i^2$?
In the residual-substitution argument, why must the average of $\|x_i\|^3|u_i|$ be bounded in probability?
Explain the difference between the limit of $n\widehat{V}_{HC0}$ and the limit of $\widehat{V}_{HC0}$.
Which condition ensures that the inverse of the sample design matrix converges to a well-defined population inverse?

Consistency of HC0 Under Conditional Heteroskedasticity

What You'll Learn