HC2 and HC3 Leverage Corrections

Why Adjust Residual Contributions for Leverage?

HC0 robust covariance estimates use squared least-squares residuals to estimate the error variation associated with each observation. But a residual is not the same as that observation’s error: fitting the regression has already used the observation’s response. This matters especially for observations with high leverage, whose responses can have a substantial effect on their own fitted values. HC2 and HC3 modify squared residual contributions to account for this leverage.

The focus here is the adjustment to each squared residual and its role in the covariance estimate. These corrections do not change the least-squares coefficients, and they do not make an incorrect mean model correct. We will derive two motivations for the adjustments: the residual-variance benchmark behind HC2 and the deleted-observation residual behind HC3.

The diagonal leverages satisfy \(0 \le h_{ii} \le 1\), and their sum is \(r\), the number of fitted coefficients. Leverage depends on the predictors and the design matrix, not on the observed response. It describes how much an observation’s fitted value is tied to its own response. When \(h_{ii}\) is large, the least-squares fit can track that observation more closely, leaving a smaller residual than its error might suggest.

The HC2 and HC3 Adjustments

For a row of the design matrix, write \(x_i'\) for the row vector of predictors used to fit observation \(i\), including the intercept column if present. In the sandwich covariance estimate, the squared residual for that row is the scalar contribution that is adjusted. HC2 and HC3 replace \(e_i^2\) with different leverage-adjusted quantities.

Here \(\hat{\beta}\) is the vector of least-squares coefficient estimates. When \(j=2\), the expression is HC2; when \(j=3\), it is HC3. The denominator is specific to each observation. If \(h_{ii}=0\), the observation receives no leverage adjustment; as \(h_{ii}\) approaches 1, both adjustments grow, with HC3 growing faster.

Why HC2 Divides by \(1-h_{ii}\)

To understand HC2, consider the usual linear mean model conditional on the design: \(y=X\beta+\varepsilon\), where the errors have mean zero, are mutually independent, and have finite variances \(\sigma_i^2\), which may differ across observations. The fitted residual vector is \(e=(I-H)y=(I-H)\varepsilon\), because \((I-H)X\beta=0\). Thus each residual combines errors from the whole data set.

Under the reference case of equal error variances, the expected squared residual has a simple relationship to leverage. Let \(M=I-H\). The hat matrix is symmetric and idempotent, so \(M\) is also symmetric and idempotent. If all errors have variance \(\sigma^2\), then the conditional variance of \(e_i\) is \(\sigma^2\) times the sum of the squared entries in row \(i\) of \(M\). Idempotence makes that sum \(M_{ii}\):

This gives the HC2 motivation: dividing by \(1-h_{ii}\) corrects the downward scaling of a squared residual under the equal-variance reference case. With unequal variances, however, the expected squared residual also depends on the variances of other observations. Therefore, this calculation does not show that HC2 recovers each \(\sigma_i^2\) exactly under arbitrary heteroskedasticity. It provides a leverage-based correction, not a guarantee of exactness.

Why HC3 Uses the Squared Leverage Adjustment

HC3 has a different motivation: its adjusted residual is the residual observation \(i\) would have if the regression were fit without observation \(i\). This interpretation requires that removing observation \(i\) leaves enough independent design information to fit the same coefficients; equivalently, \(h_{ii}<1\).

Let \(\hat{\beta}_{(-i)}\) be the least-squares coefficient vector fitted after deleting observation \(i\), and write \(A=X'X\). The full-sample and deleted-sample normal equations imply \((A-x_i x_i')(\hat{\beta}-\hat{\beta}_{(-i)})=x_i e_i\). Also, \(A^{-1}x_i\) is mapped by \(A-x_i x_i'\) to \(x_i(1-h_{ii})\), since \(h_{ii}=x_i'A^{-1}x_i\). Therefore:

The deleted-fit prediction at the omitted observation is \(x_i'\hat{\beta}_{(-i)}\). Subtracting it from the observed response gives the deleted-observation residual:

Squaring this deleted-observation residual gives \(e_i^2/(1-h_{ii})^2\), exactly the HC3 adjustment. This explains why HC3 can be interpreted as using a residual less directly influenced by the observation’s own response. It does not mean the deleted residual is a known error variance; it is a leverage-adjusted residual contribution used in the robust covariance estimate.

Comparing the Adjustments

HC0 uses \(e_i^2\) without a leverage correction. HC1 applies a common sample-size adjustment to the HC0 covariance estimate, whereas HC2 and HC3 use observation-specific leverage factors. For the same residual, HC3 is at least as large as HC2 whenever \(0 \le h_{ii}<1\), because \((1-h_{ii})^{-2}\ge(1-h_{ii})^{-1}\). Both equal the unadjusted squared residual when leverage is zero.

Assumptions, Limits, and a Common Mistake

The leverage derivations use the ordinary least-squares design with full column rank and a correctly specified linear conditional mean. For the residual-variance calculation, errors are conditionally independent with mean zero and finite variances; equality of those variances is used only for the simple benchmark that motivates HC2. Heteroskedasticity is allowed in the robust covariance setting. The deleted-observation interpretation of HC3 additionally requires that deleting the observation leaves the model estimable.

If \(h_{ii}=1\), the HC2 and HC3 denominators are zero, and the leave-one-out fit cannot be formed with the same full-rank design. If leverage is merely close to 1, the corrections can be very large. This is a signal to inspect the design and observation rather than to treat a large adjusted contribution as proof of an error or of a particular substantive explanation.

• Common mistake: “HC3 is just HC2 with a different sample-size multiplier.” This is wrong because HC2 and HC3 adjust each observation by different powers of its own \(1-h_{ii}\); neither is a common multiplier applied to all observations.
• Correction: Compare the formulas observation by observation. Their ratio is \(1/(1-h_{ii})\), so their difference depends on leverage and becomes more pronounced as leverage increases.
• Common mistake: “The HC2 adjustment makes \(e_i^2/(1-h_{ii})\) an exact estimate of \(\sigma_i^2\) under any unequal variances.” The benchmark derivation assumed equal variances. Under unequal variances, \(e_i^2\) reflects error variation from other observations too.