Observation-Level Contributions to HC Covariance

Why Inspect Observation-Level Contributions?

A heteroskedasticity-consistent standard error can be valid even when the error variance changes across observations. But a reported standard error is a single summary: it does not show whether its estimated variance is supported broadly across the sample or is driven mainly by a few observations. That distinction matters when a small change in the data or in the HC adjustment could substantially change the result.

The HC covariance estimator has an additive structure. Each observation supplies a piece to its middle matrix, often called the meat, and therefore a corresponding piece to the final covariance matrix. For a particular coefficient or linear contrast, these pieces reduce to nonnegative contributions to the estimated variance. Examining them makes it possible to describe concentration directly, while keeping clear that such a description is a diagnostic—not a replacement for the estimator or a formal test.

The Model and the HC Decomposition

Let \(y_i\) be the outcome and \(x_i\) a \(k\)-component column vector of regressors for observation \(i\), including a constant if the model has an intercept. Write \(X\) for the \(n\) by \(k\) design matrix, and suppose \(X\) has full column rank. The linear conditional-mean model is \(y_i=x_i'\beta+u_i\), where \(E(u_i\mid X)=0\). For heteroskedasticity-robust inference, observations are assumed independent conditional on \(X\), with finite error variances that need not be equal. The HC covariance estimator is motivated by this setting; its usual large-sample justification also requires appropriate regularity conditions.

Let \(\hat\beta=(X'X)^{-1}X'y\) be the ordinary least-squares estimator, let \(\hat u_i=y_i-x_i'\hat\beta\) be its residual, and put \(A=(X'X)^{-1}\). An HC estimator uses residual squares with an observation-specific adjustment. Write that adjustment as \(\lambda_i\). Then the estimator is

The middle matrix is the meat: it is the sum of the observation-level matrices \(\lambda_i\hat u_i^{\,2}x_i x_i'\). The matching contribution to the final covariance matrix is

This follows by distributing \(A\) over the sum and using the symmetry of \(A\). Each \(C_i\) is positive semidefinite: for any vector \(a\), \(a'C_i a=\lambda_i\hat u_i^{\,2}(a'Ax_i)^2\geq 0\), provided \(\lambda_i\geq0\). The off-diagonal entries of \(C_i\), however, can be negative. So the matrices add up observation by observation, but their individual entries are not all nonnegative.

Common choices of \(\lambda_i\) include \(1\) for HC0, \(n/(n-k)\) for HC1, \(1/(1-h_{ii})\) for HC2, and \(1/(1-h_{ii})^2\) for HC3, where \(h_{ii}=x_i'Ax_i\) is the observation’s leverage. HC1 applies the same multiplier to every observation; HC2 and HC3 adjust each observation according to its leverage. The decomposition works for each choice, but the contributions—and their relative sizes—can change with the choice.

Contributions for a Coefficient or Contrast

Let \(c\) be a \(k\)-component vector defining a target \(c'\beta\). For example, \(c\) can select one coefficient, or it can define a difference between two coefficients. The estimated variance of \(c'\hat\beta\) is \(c'\widehat{\operatorname{Var}}_{\mathrm{HC}}(\hat\beta)c\). Its observation-level contribution is

The square in this expression makes every \(q_i(c)\) nonnegative when the HC adjustment is nonnegative. If the total variance is positive, the share \(q_i(c)/\sum_j q_j(c)\) gives the fraction of that estimated variance supplied by observation \(i\). These shares are specific to the target: an observation can contribute substantially to one coefficient’s variance and little to another’s.

Finding Which Observations Drive a Target Variance

For the slope in the example, take \(c=(0,1)'\). The slope component of \(Ax_i\) is \(-0.1+0.2x_i\). Under HC0, the contributions are the squared residual multiplied by the square of this component:

How Leverage Adjustments Change the Decomposition

The observation-level display is also useful for seeing how an HC adjustment changes the estimated variance. With HC2, each residual square is divided by \(1-h_{ii}\). In the example, the leverages are \(h_{ii}=x_i'Ax_i\), which evaluate to \(0.7,0.3,0.3,0.7\). They are all below one, so the HC2 adjustment is defined. For the slope, the unadjusted squared pieces \(0.09,0.04,0.01,0\) are divided by \(0.3,0.7,0.7,0.3\), respectively.

Using Contribution Summaries Carefully

A concentrated contribution profile is a reason to investigate the sensitivity of the reported standard error, not a reason to discard observations automatically. A large contribution can reflect a large residual, a regressor vector that strongly projects onto the target after multiplication by \(A\), a large HC adjustment, or a combination of these. In leverage-adjusted HC estimators, a factor such as \(1/(1-h_{ii})\) can amplify a residual-based piece when leverage is high.

It is useful to report the target being examined, the HC estimator used, the largest contribution shares, and—if helpful—a concentration summary. Comparing contribution profiles across HC variants can reveal whether a large standard error is broadly supported or depends heavily on a few adjusted pieces. For an overall covariance matrix, inspect target variances or specified contrasts rather than trying to rank observations by individual matrix entries: off-diagonal contributions may be negative, and “largest matrix contribution” has no single natural ranking without a chosen measure.

These calculations do not measure how much an observation would change the regression estimate if removed. They decompose the covariance estimator at the fitted model and for a stated target. Nor do they establish that an observation is erroneous, influential in every sense, or responsible for a causal effect. Such conclusions require additional analysis and substantive context.

Common Pitfall

• Wrong reasoning: “One observation contributes 65% of the slope variance, so the robust standard error is invalid and that observation should be removed.”
• Correction: The share identifies concentration in this estimator for this target. It is a diagnostic prompt to check the data, design, residual, HC choice, and sensitivity of the result. It is not itself a validity test, and deleting a genuine observation just to reduce its share can introduce bias.
• Wrong reasoning: “A negative off-diagonal entry means an observation has a negative covariance contribution.”
• Correction: Individual covariance matrices can have negative off-diagonal entries while remaining positive semidefinite. For a specified contrast, calculate \(q_i(c)=c'C_i c\); that scalar contribution is nonnegative under a nonnegative HC adjustment.