HC4 and HC5 Under Extreme Leverage

Why HC4 and HC5 Treat High Leverage Differently

Heteroskedasticity-robust covariance estimators use squared OLS residuals to estimate the observation-level variation entering the sandwich covariance. But a high-leverage observation can have an unusually small residual partly because it helped determine its own fitted value. HC2 addresses this by dividing the squared residual by \(1-h_{ii}\). HC4 and HC5 extend that idea: they use observation-specific powers of \(1-h_{ii}\), with larger corrections for observations whose leverage is unusually high relative to the average.

These estimators are adjustments to the residual contributions in a covariance estimate, not changes to the OLS coefficients. Their purpose is to reduce the risk that a few high-leverage observations make an estimated covariance too small. More aggressive adjustment is not automatically better: if leverage is close to one, the correction can become enormous and the resulting covariance estimate can be highly sensitive to individual observations.

Setup and notation: Consider the OLS model \(y=X\beta+u\), with \(n\) observations and a design matrix \(X\) of full column rank \(k\). Let \(\hat{\beta}=(X'X)^{-1}X'y\), let \(e_i\) be the \(i\)th OLS residual, and let \(h_{ii}\) be the \(i\)th diagonal element of the hat matrix \(H=X(X'X)^{-1}X'\). The average leverage is \(\bar h=k/n\). The definitions below require \(h_{ii}<1\) for every observation whose residual adjustment is used; full column rank alone does not ensure this condition.

Write \(A=(X'X)^{-1}\). A heteroskedasticity-consistent covariance estimator has the sandwich form \(A(\sum_i x_i x_i'\hat{\omega}_i)A\), where \(x_i'\) is row \(i\) of \(X\) and \(\hat{\omega}_i\) is an adjusted squared residual. HC4 and HC5 differ in how they choose that adjustment.

Definition: The HC4 adjustment uses \(\delta_i^{(4)}=\min\{4,h_{ii}/\bar h\}\) and \(\hat{\omega}_i^{(4)}=e_i^2/(1-h_{ii})^{\delta_i^{(4)}}\). The HC5 adjustment, using the conventional tuning constant \(\kappa=0.7\), uses \(\delta_i^{(5)}=\min\{h_{ii}/\bar h,\max(4,\kappa h_{\max}/\bar h)\}\), where \(h_{\max}=\max_i h_{ii}\), and \(\hat{\omega}_i^{(5)}=e_i^2/(1-h_{ii})^{\delta_i^{(5)}/2}\). In each case, the covariance estimate is \(A(\sum_i x_i x_i'\hat{\omega}_i)A\).

The leverage ratio \(h_{ii}/\bar h\) compares an observation’s leverage with the design’s average leverage. HC4 caps its exponent at 4. HC5 also uses a cap, but sets it in relation to the sample’s maximum leverage and applies half of its selected exponent in the denominator. Thus, an observation with ordinary leverage receives a relatively mild adjustment, while one with extreme leverage may receive a much larger one. These formulas specify the convention used here; implementations can expose tuning choices, so comparisons should identify the version used.

Why Residual Corrections Depend on Leverage

The reason for correcting residuals can be seen under the simpler homoskedastic model, where \(E(u\mid X)=0\) and \(\operatorname{Var}(u\mid X)=\sigma^2 I\). In matrix notation, \(e=(I-H)u\). Since \(H\) is a symmetric idempotent projection matrix, the variance of residual \(e_i\) is \(\sigma^2(1-h_{ii})\). In particular, larger leverage means less residual variance under this benchmark. HC2 divides by \(1-h_{ii}\), counteracting that shrinkage. HC4 and HC5 make the correction more aggressive for unusually high leverage; under heteroskedasticity their adjustments are not exact unbiased corrections for each observation’s error variance.

\begin{aligned} \operatorname{Var}(e_i\mid X) &=\sigma^2\sum_{j=1}^n(\mathbf{1}\{i=j\}-h_{ij})^2\\ &=\sigma^2\bigl((1-h_{ii})^2+\sum_{j\ne i}h_{ij}^2\bigr)\\ &=\sigma^2(1-h_{ii}). \end{aligned}

For the final equality, idempotence gives \(\sum_j h_{ij}^2=h_{ii}\), so \(\sum_{j\ne i}h_{ij}^2=h_{ii}-h_{ii}^2\). Substituting this into the preceding line yields \(\sigma^2((1-h_{ii})^2+h_{ii}-h_{ii}^2)=\sigma^2(1-h_{ii})\). This explains the basic leverage correction. It does not show that HC4 or HC5 is exact under general conditional heteroskedasticity: their larger powers are finite-sample adjustments intended to protect against leverage-related residual shrinkage.

Worked Example: Calculating HC4 and HC5 Exponents

Consider a hypothetical regression with \(n=20\) observations and \(k=2\) regressors, including any intercept. Suppose three observations have leverages \(0.80\), \(0.30\), and \(0.05\); the other 17 each have leverage \(0.05\). These leverages sum to \(0.80+0.30+18(0.05)=2\), as required for a rank-2 hat matrix. The average leverage is \(\bar h=2/20=0.10\), and the maximum is \(h_{\max}=0.80\). All leverages are less than one, so the stated adjustments are defined.

For the first observation, \(h_{ii}/\bar h=0.80/0.10=8\). HC4 gives \(\delta_i^{(4)}=\min(4,8)=4\). For HC5, \(\max(4,0.7(0.80/0.10))=\max(4,5.6)=5.6\), so \(\delta_i^{(5)}=\min(8,5.6)=5.6\). The corresponding multipliers on \(e_i^2\) are \(0.20^{-4}=625\) for HC4 and \(0.20^{-2.8}\approx90.59746\) for HC5.

For the observation with leverage \(0.30\), the ratio is \(3\). Both methods select an exponent of \(3\): HC4 gives \(\min(4,3)=3\), and HC5 gives \(\min(3,5.6)=3\). Their multipliers are nevertheless different because HC5 uses half the selected exponent: \(0.70^{-3}\approx2.915\) for HC4 and \(0.70^{-1.5}\approx1.707\) for HC5. For an observation with leverage \(0.05\), the ratio is \(0.5\), giving multipliers \(0.95^{-0.5}\approx1.026\) and \(0.95^{-0.25}\approx1.013\), respectively. The difference between the methods is therefore especially pronounced for the most influential leverage in this example.

From Adjusted Residuals to a Covariance Estimate

For a coefficient or a linear contrast, the covariance estimate is built by adding each observation’s contribution. For a contrast \(c'\hat{\beta}\), define \(q_i=c'Ax_i\). Then its estimated variance under either method is \(\sum_i q_i^2\hat{\omega}_i\). This scalar form makes clear why a large residual multiplier can matter most when the observation also contributes substantially in the contrast direction. It also allows a direct comparison without changing the OLS coefficient estimate.

Worked Example: Comparing HC4 and HC5 Contributions

Continue with the hypothetical design in the preceding example. Suppose a fitted regression reports the following values for three observations: the observation with leverage \(0.80\) has residual \(e_i=0.20\) and contrast weight \(q_i=0.40\); the observation with leverage \(0.30\) has \(e_i=0.10\) and \(q_i=0.30\); and one observation with leverage \(0.05\) has \(e_i=0.05\) and \(q_i=0.10\). These are illustrative fitted-regression quantities, and the calculation below reports only these three observations’ contributions, not the full contrast variance.

For each row, the contribution is \(q_i^2e_i^2\) times the method’s leverage multiplier. For the leverage-\(0.80\) observation, \(q_i^2e_i^2=(0.40)^2(0.20)^2=0.0064\). Its contribution is \(0.0064(625)=4.000\) under HC4 and \(0.0064(90.59746)\approx0.5798237\) under HC5.

For leverage \(0.30\), the unadjusted quantity is \((0.30)^2(0.10)^2=0.0009\). The contributions are \(0.0009(2.91545)\approx0.002624\) under HC4 and \(0.0009(1.70747)\approx0.0015367\) under HC5. For leverage \(0.05\), it is \((0.10)^2(0.05)^2=0.000025\), giving \(0.000025(1.02598)\approx0.00002565\) under HC4 and \(0.000025(1.01291)\approx0.00002532\) under HC5.

Adding these three contributions gives \(4.00265\) for HC4 and \(0.58139\) for HC5. These are partial variance contributions in the squared units of the contrast, not complete variance estimates or standard errors. The large difference comes mainly from the very high-leverage observation; the remaining observations in a full analysis would also contribute to the sum.

Extreme Leverage and Instability

The same mechanism that protects against residual shrinkage can make HC4 and HC5 unstable. As \(h_{ii}\) approaches one, \(1-h_{ii}\) approaches zero, and raising its reciprocal to a positive power produces a large multiplier. The resulting estimate may depend heavily on a single residual and on small changes in the design. HC5’s half-exponent adjustment can be less extreme than HC4’s in some designs, but it is not a general safeguard against near-unit leverage.

Worked Example: A Near-Unit Leverage Observation

Suppose a hypothetical regression has \(n=20\), \(k=2\), and one observation with leverage \(0.98\), while all leverages are below one. The average leverage is \(0.10\), so the observation’s ratio is \(0.98/0.10=9.8\), and \(h_{\max}/\bar h=9.8\). HC4 selects \(\delta_i^{(4)}=4\), giving multiplier \(0.02^{-4}=6{,}250{,}000\). HC5 uses \(\max(4,0.7(9.8))=6.86\), selects \(\delta_i^{(5)}=\min(9.8,6.86)=6.86\), and gives multiplier \(0.02^{-3.43}\approx6.72\times10^5\).

If this observation’s residual were \(0.01\), its adjusted squared-residual terms would be \(0.01^2(6{,}250{,}000)=625\) for HC4 and approximately \(0.01^2(6.72\times10^5)=67.2\) for HC5. These figures are only the observation-level terms before multiplication by the design vectors and the outer factors \(A\). They nevertheless show how a small residual can receive a very large correction when leverage is near one. If leverage equals one, the denominator is zero; the formulas are not defined, and those observations cannot be handled by these adjustments as written.

Large corrections should prompt scrutiny of the design and of the observations involved, rather than automatic trust in whichever estimator produces the smaller standard error. HC4 and HC5 do not remove high-leverage observations, alter the fitted regression, or establish that the conditional mean is correctly specified. Their covariance estimates are inputs to robust inference, but the adjustments alone do not guarantee accurate finite-sample coverage or test size. In particular, a robust standard error is not evidence that a coefficient is a causal effect.

Choosing and Reporting the Adjustment

HC4 and HC5 are most relevant when some observations have leverage substantially above \(k/n\), especially in a sample where a few design points may dominate the fitted relationship. HC4 imposes a fixed maximum exponent of 4. HC5 uses the sample’s maximum leverage to set its cap and then applies half that selected exponent. In a design without unusually high leverage, their differences from less aggressive corrections may be modest. In a design with extreme leverage, the choice can materially affect estimated standard errors and subsequent robust tests or intervals.

A careful report names the covariance estimator, identifies any tuning convention used, and describes the leverage pattern when it is consequential. It should not describe HC4 or HC5 as a test for heteroskedasticity. These are covariance estimators designed to remain useful under heteroskedasticity, with distinct finite-sample leverage adjustments; they do not diagnose the form of the variance.

Common Mistakes

Assuming full rank rules out unit leverage. It does not. The formulas divide by a power of \(1-h_{ii}\), so check that each adjusted observation has \(h_{ii}<1\).
Applying HC5’s full exponent. HC5 divides by \((1-h_{ii})^{\delta_i^{(5)}/2}\), not by \((1-h_{ii})^{\delta_i^{(5)}}\).
Using the wrong HC5 cap. Under the stated convention, the cap is \(\max(4,0.7h_{\max}/\bar h)\), and the selected exponent is the smaller of that cap and \(h_{ii}/\bar h\).
Calling the multiplier an estimated variance by itself. It adjusts \(e_i^2\); the resulting term must still be combined with \(x_i\) and the sandwich factors.
Treating a larger correction as automatically more reliable. A large multiplier can make the covariance estimate fragile, particularly when leverage is near one.

Key takeaway: HC4 and HC5 adapt squared-residual corrections to leverage relative to its average. HC4 caps its exponent at 4; conventional HC5 uses a cap informed by maximum leverage and applies half its selected exponent. Both require leverage below one for adjusted observations, and neither eliminates the instability that can arise under extreme leverage.

Check Your Understanding

Try these questions using the definitions and calculations above.

For \(n=40\) and \(k=4\), what is the average leverage? What leverage ratio corresponds to an observation with \(h_{ii}=0.30\)?
For an observation with \(h_{ii}/\bar h=6\) and \(h_{\max}/\bar h=7\), calculate the HC4 and conventional HC5 exponents.
Why does an HC5 adjustment use \(\delta_i^{(5)}/2\) in its denominator rather than the full exponent?
What additional condition is needed for the HC4 and HC5 adjustments to be defined, beyond full column rank of \(X\)?
Give one reason that a very large HC4 or HC5 multiplier may make an estimated covariance unstable.