COVRATIO for Case Effects on Coefficient Uncertainty

Why Compare Coefficient Uncertainty?

A case can affect more than the fitted coefficients or fitted values. Removing it can also change how precisely the regression estimates the coefficients. COVRATIO summarizes that change by comparing the estimated coefficient covariance matrix from the full data with the matrix obtained after deleting one observation.

The comparison is joint: it uses the determinant of the covariance matrix, not a separate ratio for each coefficient. A case may have a noticeable effect on the estimated precision of the coefficient vector even when no single coefficient tells the whole story. COVRATIO is therefore a diagnostic, not a test that an observation is erroneous or should automatically be removed.

Definition: Consider a linear regression with $n$ observations and $p$ fitted coefficients, including the intercept if the model has one. Let $\widehat{\operatorname{Cov}}(\hat{\beta})$ be the estimated covariance matrix of the coefficient estimates using all observations, and let $\widehat{\operatorname{Cov}}_{(i)}(\hat{\beta})$ be the corresponding estimate after deleting observation $i$ and refitting. Define $$ \operatorname{COVRATIO}_i = \frac{\left|\widehat{\operatorname{Cov}}_{(i)}(\hat{\beta})\right|} {\left|\widehat{\operatorname{Cov}}(\hat{\beta})\right|}. $$ A value above one means the estimated covariance determinant is larger after deletion; a value below one means it is smaller. The determinant is a measure of joint uncertainty, not the variance of a particular coefficient.

Model and Calculation Formula

Use the ordinary least squares model $Y=X\beta+\varepsilon$, with $X$ an $n \times p$ design matrix of full column rank. The usual covariance estimate is based on the homoskedastic error model: errors have mean zero, common variance $\sigma^2$, and are uncorrelated. Normality is not needed for the algebraic COVRATIO calculation, although it is an assumption used for exact conventional small-sample $t$- and $F$-inference. Deleting observation $i$ must leave the design matrix with full column rank, and there must be positive residual degrees of freedom in both fits.

Write $e_i$ for the full-model residual, $h_{ii}$ for the $i$th diagonal element of the hat matrix, and $\mathrm{SSE}$ for the full-model residual sum of squares. The full fit has $n-p$ residual degrees of freedom; the deleted fit has $n-p-1$. The deleted residual sum of squares is

\mathrm{SSE}_{(i)} = \mathrm{SSE}-\frac{e_i^2}{1-h_{ii}}.

Thus, deleting an observation decreases the residual sum of squares by $e_i^2/(1-h_{ii})$. The deleted mean squared error is $s^2_{(i)}=\mathrm{SSE}_{(i)}/(n-p-1)$, while the full-model estimate is $s^2=\mathrm{SSE}/(n-p)$. Under the stated model, the covariance estimates are $s^2(X^\mathsf{T}X)^{-1}$ and $s^2_{(i)}(X_{(i)}^\mathsf{T}X_{(i)})^{-1}$, respectively. Combining these gives the working formula

\operatorname{COVRATIO}_i = \left(\frac{s^2_{(i)}}{s^2}\right)^p \frac{1}{1-h_{ii}} = \left[ \frac{\mathrm{SSE}_{(i)}/(n-p-1)} {\mathrm{SSE}/(n-p)} \right]^p \frac{1}{1-h_{ii}}.

Here $p$ is the number of fitted coefficients, not just the number of predictors. The exponent matters because multiplying a $p \times p$ covariance matrix by a scalar multiplies its determinant by that scalar raised to the $p$th power.

Why the Formula Holds

The formula has two components. One tracks the change in residual variance; the other tracks how deleting a design point changes the coefficient geometry. For the second component, let $x_i^\mathsf{T}$ be row $i$ of $X$, and set $A=X^\mathsf{T}X$. Deleting that row changes the cross-product matrix to $A-x_ix_i^\mathsf{T}$. The determinant lemma gives

\frac{|X_{(i)}^\mathsf{T}X_{(i)}|}{|X^\mathsf{T}X|} = \frac{|A-x_ix_i^\mathsf{T}|}{|A|} = 1-x_i^\mathsf{T}A^{-1}x_i = 1-h_{ii}.

Therefore the determinant of the inverse cross-product matrix increases by the factor $1/(1-h_{ii})$ after deletion. Multiplying the deleted covariance matrix by $s^2_{(i)}$, rather than $s^2$, contributes the additional determinant factor $(s^2_{(i)}/s^2)^p$.

The deleted-SSE identity can also be understood through the leave-one-out residual. The residual for observation $i$, calculated from the fit that excludes it, is $e_i/(1-h_{ii})$. The least-squares sum-of-squares identity then gives the change in SSE as

\mathrm{SSE}-\mathrm{SSE}_{(i)} = \frac{e_i^2}{1-h_{ii}}.

This is a decrease in SSE on deletion. It does not imply that the deleted mean squared error must decrease: its denominator also falls by one degree of freedom. Both changes must be included before interpreting COVRATIO.

Worked Calculations

Worked Example: A Case That Increases the Covariance Determinant When Deleted

A hypothetical regression has $n=30$ observations and $p=2$ fitted coefficients. Its full-model $\mathrm{SSE}$ is 280. For observation $i$, $h_{ii}=0.20$ and $e_i^2=22.4$. Calculate COVRATIO and interpret it. Use the usual homoskedastic linear model and round the final ratio to four decimal places.

Method. First calculate the deleted SSE and the two mean squared errors, then substitute their ratio and the leverage into the COVRATIO formula.

The full residual degrees of freedom are $30-2=28$, so

s^2=\frac{280}{28}=10.

The decrease in SSE on deletion is

\frac{e_i^2}{1-h_{ii}} = \frac{22.4}{1-0.20} = \frac{22.4}{0.80} = 28.

Thus the deleted SSE is $280-28=252$, and the deleted residual degrees of freedom are $30-2-1=27$. Therefore

s^2_{(i)}=\frac{252}{27}=\frac{28}{3}, \qquad \frac{s^2_{(i)}}{s^2} = \frac{28/3}{10} = \frac{14}{15}.

Since $p=2$,

\operatorname{COVRATIO}_i = \left(\frac{14}{15}\right)^2\frac{1}{0.80} = \frac{196}{225}\cdot\frac{5}{4} = \frac{49}{45} \approx 1.0889.

Deleting the case increases the estimated covariance determinant by about 8.89% relative to the full fit, so this case contributes to the estimated joint precision. The result does not, on its own, establish that the case is problematic. Common mistake avoided: the deleted SSE is $280-28$, not $280+28$; deletion reduces SSE by the stated amount.

Worked Example: Accounting for the Lost Degree of Freedom

A hypothetical regression has $n=29$, $p=2$, and full-model $\mathrm{SSE}=280$. For a particular case, $h_{ii}=0.20$ and $e_i^2=16$. Find COVRATIO, rounding only the final answer to four decimal places.

The full residual degrees of freedom are $29-2=27$, giving $s^2=280/27$. The decrease in SSE on deletion is

\frac{16}{1-0.20}=\frac{16}{0.80}=20.

Hence $\mathrm{SSE}_{(i)}=280-20=260$. The deleted fit has $29-2-1=26$ residual degrees of freedom, so $s^2_{(i)}=260/26=10$. The mean-squared-error ratio is

\frac{s^2_{(i)}}{s^2} = \frac{10}{280/27} = \frac{270}{280} = \frac{27}{28}.

Substitution gives

\operatorname{COVRATIO}_i = \left(\frac{27}{28}\right)^2\frac{1}{0.80} = \frac{729}{784}\cdot\frac{5}{4} = \frac{3645}{3136} \approx 1.1623.

The deleted fit has an estimated covariance determinant about 16.23% larger than the full fit. Common mistake avoided: comparing SSE values alone misses the degree-of-freedom adjustment. Here the deleted MSE is 10, while the full MSE is $280/27$; the ratio is $27/28$, not $260/280$.

Worked Example: A Ratio Below One

A hypothetical regression has $n=24$, $p=2$, and full-model $\mathrm{SSE}=220$. A case has $h_{ii}=0.10$ and $e_i^2=19.8$. Calculate COVRATIO and explain what a value below one means. Round the final ratio to four decimal places.

The full residual degrees of freedom are $24-2=22$, giving $s^2=220/22=10$. The decrease in SSE on deletion is

\frac{19.8}{1-0.10} = \frac{19.8}{0.90} = 22.

Therefore $\mathrm{SSE}_{(i)}=220-22=198$, and the deleted degrees of freedom are $24-2-1=21$. The deleted MSE and its ratio to the full MSE are

s^2_{(i)}=\frac{198}{21}=\frac{66}{7}, \qquad \frac{s^2_{(i)}}{s^2} = \frac{66/7}{10} = \frac{33}{35}.

Applying the formula,

\operatorname{COVRATIO}_i = \left(\frac{33}{35}\right)^2\frac{1}{0.90} = \frac{1089}{1225}\cdot\frac{10}{9} = \frac{242}{245} \approx 0.9878.

The determinant is slightly smaller after deletion, indicating a slight increase in joint estimated precision in the deleted fit. The ratio is close to one, so the change is small. Common mistake avoided: a value below one does not mean that deleting the case has reduced precision; it means the deleted covariance determinant is smaller.

Interpreting COVRATIO Carefully

COVRATIO combines the effects of residual variance and leverage. The factor $(s^2_{(i)}/s^2)^p$ reflects the change in estimated error variance, magnified by the number of coefficients. The factor $1/(1-h_{ii})$ reflects the change in the design information matrix. These components can work in opposite directions, so neither leverage nor the residual alone determines the final ratio.

A commonly used screening rule flags a case when

\left|\operatorname{COVRATIO}_i-1\right|>\frac{3p}{n}.

This is a rule of thumb, not a formal significance test or a universal boundary. Its value depends on both the sample size and the number of fitted coefficients. Report the ratio and the context rather than treating a cutoff as an automatic instruction to delete a case.

COVRATIO also has limits. A determinant summarizes the joint covariance matrix, and a ratio near one does not guarantee that every coefficient’s estimated variance or covariance is nearly unchanged. Conversely, a large departure from one identifies a possible case effect on joint precision but does not explain why it occurs. Check the data, model specification, and relevant diagnostics before drawing a substantive conclusion.

Common Exam Traps

Reversing the ratio. This definition puts the deleted covariance determinant in the numerator and the full-data determinant in the denominator. Reversing them reverses the interpretation.
Adding the SSE change. The correct identity is $\mathrm{SSE}_{(i)}=\mathrm{SSE}-e_i^2/(1-h_{ii})$. The deleted SSE is smaller, not larger.
Using the wrong degrees of freedom. The full MSE denominator is $n-p$; after deleting one observation it is $n-p-1$. Do not compare the two SSE values without adjusting these denominators.
Using the wrong exponent. The exponent is $p$, the total number of coefficients estimated, including the intercept.
Calling the ratio a coefficient-specific effect. It compares determinants of full covariance matrices and does not identify which coefficient changed most.
Deleting a case solely because of a cutoff. A diagnostic flag calls for investigation; it is not proof of data error or a sufficient reason for removal.

Key takeaway: Calculate the deleted SSE with the correct decrease, use the correct residual degrees of freedom for each fit, and then compare covariance determinants in the deleted-to-full order. Interpret COVRATIO as a joint precision diagnostic, not as an automatic deletion rule.

Check Your Understanding

For each problem, calculate or interpret the stated COVRATIO using the deleted-to-full convention.

A hypothetical regression has $n=18$, $p=2$, $\mathrm{SSE}=160$, $h_{ii}=0.25$, and $e_i^2=12$. Find the deleted SSE, both MSEs, and COVRATIO, rounded to four decimals. (Answer: deleted SSE $=144$; full MSE $=10$; deleted MSE $=9.6$; COVRATIO $=1.2288$.)
A hypothetical model has $n=22$, $p=3$, $\mathrm{SSE}=190$, $h_{ii}=0.20$, and $e_i^2=12.8$. Calculate COVRATIO to four decimals. (Answer: $1.1291$.)
A hypothetical model has $n=40$, $p=2$, $\mathrm{SSE}=380$, $h_{ii}=0.10$, and $e_i^2=18$. Calculate COVRATIO to four decimals and assess it using the $3p/n$ screening rule. (Answer: COVRATIO $=1.0519$; the cutoff is $0.1500$, so it is not flagged.)
A case in a hypothetical model with $n=25$ and $p=2$ has COVRATIO $=1.30$. Apply the $3p/n$ rule and state the diagnostic conclusion. (Answer: cutoff $=0.24$; $|1.30-1|=0.30>0.24$, so it is flagged for investigation, not automatic deletion.)
A hypothetical regression has $n=29$, $p=2$, $\mathrm{SSE}=280$, $h_{ii}=0.20$, and $e_i^2=16$. Find the deleted MSE and COVRATIO to four decimals. (Answer: deleted MSE $=10$; COVRATIO $=1.1623$.)

What You'll Learn