Average Leverage and Leverage Cutoffs

Why Average Leverage Is a Useful Reference

Leverage describes how unusual an observation’s predictor values are relative to the predictor values used to fit a regression model. It is a property of the design matrix, not of the observed response. In a regression diagnostic, an individual leverage value is easier to interpret when compared with the average leverage for that model.

The average is determined by the model’s fitted dimension and the sample size. This gives a useful baseline, but it does not make any particular observation problematic. Common rules flag values that are two or three times the average for further examination. Those rules are screening conventions, not formal hypothesis tests or automatic deletion rules.

Deriving the Average Leverage

Suppose the model has a fixed design matrix \(X\) with \(n\) rows and \(k\) linearly independent columns, including a column of ones for the intercept. Ordinary least squares then uses the hat matrix \(H=X(X^\mathsf{T}X)^{-1}X^\mathsf{T}\) to map the observed response vector to its fitted values. The leverage \(h_{ii}\) is the \(i\)th diagonal entry of \(H\).

The key fact is that \(H\) is a projection matrix onto the column space of \(X\). Its trace, the sum of its diagonal entries, equals its rank. Since \(X\) has full column rank \(k\), the rank of \(H\) is \(k\). Therefore the total leverage is \(k\), and dividing by the \(n\) diagonal entries gives the average.

This result assumes ordinary least squares, a full-column-rank design matrix, and the stated model specification. If there is no intercept, or if the fitted design has a different number of linearly independent columns, use the rank of the design matrix in place of \(p+1\). If predictors are exactly collinear, the ordinary inverse in the displayed formula does not exist; the effective fitted dimension must be determined using the appropriate rank or fitting procedure.

Because leverage is a diagonal element of a projection matrix, each \(h_{ii}\) lies between 0 and 1. A value near 1 means that the fitted value at that design point depends strongly on that observation’s response. That fact alone does not establish that the response is unusual or that the observation changes a conclusion.

Common Screening Cutoffs

A frequently used screening approach compares each \(h_{ii}\) with two reference levels: twice the average leverage and three times the average leverage. For a regression with an intercept, those are \(2(p+1)/n\) and \(3(p+1)/n\). Some sources write the rule using \(p\) for the number of estimated coefficients rather than the number of predictors. Checking the author’s convention prevents an off-by-one error.

The cutoffs depend on both the number of fitted coefficients and the sample size. Adding predictors increases the average leverage because the model fits in a higher-dimensional space. With a fixed number of coefficients, increasing the sample size reduces the average. A cutoff greater than 1 cannot be exceeded, since no diagonal leverage is greater than 1. In a small sample, this can make a nominal screening rule uninformative.

The two-times and three-times rules are conventions rather than universal laws. They do not account for every feature of the predictor distribution, and a value below a cutoff is not proof that the design point is ordinary. A value above a cutoff is a reason to inspect the observation and the model specification. It is not, by itself, grounds to remove a data point.

Leverage in a Simple Regression

For a simple linear regression with an intercept and one quantitative predictor \(x\), the diagonal leverage for observation \(i\) can also be written in terms of the predictor’s distance from its sample mean:

The first term, \(1/n\), is shared by all observations. The second term is larger when \(x_i\) is farther from the predictor mean relative to the overall spread of the predictor values. Thus, an unusually extreme predictor value can have high leverage even though leverage says nothing by itself about whether the observed response is unusual.

Why a Cutoff Is an Alert, Not a Verdict

A cutoff compresses a complicated predictor-space comparison into a convenient number. That makes it useful for screening many observations, but it cannot replace context. A design point can be unusual because it represents a legitimate but rare combination of predictor values. Conversely, a point can fall below a screening threshold while still being worth checking if the data-entry process, model form, or study design raises concerns.

Keep three questions separate:

• How unusual are the predictors? Leverage addresses the observation’s position in the fitted predictor space.
• How unusual is the response relative to the fitted value? That requires examining residual-based diagnostics, not leverage alone.
• Does the observation materially affect the fit? A leverage cutoff by itself does not answer this question.

This distinction matters in actuarial work. A hypothetical policy with rare but valid characteristics could have high leverage simply because few similar policies were observed. That may identify a region where the fitted relationship is supported by limited data. It does not establish a data error, an abnormal claim outcome, or a reason to remove the policy. The analyst should first verify the observation and the model specification, then assess the relevant diagnostics and business context.