Why VIFs Are Useful
A multiple regression coefficient describes the association between a predictor and the response while holding the other predictors fixed. When predictors contain overlapping information, the data may provide limited information about those separate conditional effects. The variance inflation factor, or VIF, summarizes how much linear predictability of a predictor from the other predictors is associated with greater uncertainty in its estimated coefficient.
A VIF is a diagnostic for a particular predictor in a specified regression design. It is not a test of whether the overall regression is useful, a measure of predictive accuracy, or a universal pass-or-fail rule. Its value depends on which predictors are in the model and on how they are represented.
The \(R_j^2\) in this definition is not the \(R^2\) from the main regression of the response on the predictors. It measures how well the other predictors, together, linearly explain \(x_j\). The auxiliary regression includes an intercept when the main model includes one. Use the same rows and the same predictor coding as in the main model; changing either can change the result.
The usual variance interpretation applies to an estimable ordinary least squares model with a full-rank design matrix and errors that have mean zero, constant variance \(\sigma^2\), and zero covariance across observations. Normal errors are additionally needed for exact finite-sample normal and \(t\) inference, but not for the algebraic VIF calculation. If exact collinearity makes a coefficient unidentifiable, its VIF is infinite rather than a finite uncertainty multiplier.
From Auxiliary Regression to VIF
To calculate a VIF, first find the auxiliary regression’s \(R_j^2\), or its tolerance, and then apply the definition. An equivalent calculation uses the auxiliary regression’s centered total sum of squares and residual sum of squares:
Here \(\operatorname{SST}_j\) is the sum of squared deviations of \(x_j\) from its mean, and \(\operatorname{SSE}_j\) is the sum of squared residuals from predicting \(x_j\) using the other predictors. These sums of squares must refer to the same auxiliary regression. The second expression is not a formula using the main regression’s response residual sum of squares.
Worked Example: Calculating VIFs From Auxiliary R-squared Values
A hypothetical insurer fits a linear regression with an intercept to model annual adjustment expense using policy age and insured value. The auxiliary regression of policy age on insured value has \(R_{\text{age}}^2=0.75\). The auxiliary regression of insured value on policy age has \(R_{\text{value}}^2=0.75\). Calculate both VIFs and interpret them. Assume the main-model errors have mean zero, constant variance \(\sigma^2\), and zero covariance across observations, and that the predictors are not exactly collinear.
Method. Apply the definition separately to each predictor. The auxiliary \(R^2\) values here describe prediction of the named predictor, not prediction of annual adjustment expense.
For policy age, the tolerance is \(1-0.75=0.25\). Therefore,
For insured value, the calculation is the same: its tolerance is \(1-0.75=0.25\), so \(\operatorname{VIF}_{\text{value}}=1/0.25=4\).
Under the stated regression assumptions, each coefficient’s variance is four times the corresponding variance it would have with the same centered predictor sum of squares and residual variance but with that predictor orthogonal to the other predictors. The associated standard-error factor is \(\sqrt{4}=2\). This comparison does not mean either coefficient’s actual standard error is 2; a baseline standard error is needed to calculate that.
Check. Both VIFs exceed 1, as they must when the predictors are at least partly explained by the other predictor. The calculation avoids confusing the auxiliary \(R^2\) with the response-model \(R^2\).
Why the Variance Ratio Appears
The variance interpretation follows from how much of a predictor remains after its linear relationship with the other predictors is removed. For a model with an intercept, let \(S_{jj}=\sum_i(x_{ij}-\bar{x}_j)^2\). In the auxiliary regression, the residual sum of squares is \(\operatorname{SSE}_j=(1-R_j^2)S_{jj}\). In multiple regression, the variance of the least-squares coefficient for \(x_j\), conditional on the design matrix, is
If \(x_j\) were orthogonal to the other predictors while retaining the same centered sum of squares \(S_{jj}\), its coefficient variance would be \(\sigma^2/S_{jj}\). Dividing the actual variance by that reference variance gives \(1/(1-R_j^2)\), which is the VIF. This explains why a VIF is a variance multiplier, while the corresponding standard-error multiplier is its square root.
For example, a VIF of 9 corresponds to nine times the reference variance but three times the reference standard error, not nine times the standard error. This comparison isolates the contribution of linear relationships among predictors under the stated model assumptions; it does not account for every possible source of uncertainty, such as a small sample or high response noise.
Worked Example: Converting Tolerance to VIF and an Uncertainty Factor
In a hypothetical regression of monthly repair cost on several building characteristics, the tolerance for building age is reported as \(0.08\). Find its VIF and the associated standard-error factor relative to the orthogonal-predictor reference. State what can and cannot be concluded. Assume the fitted model satisfies the ordinary least squares conditions stated above and the coefficient is estimable.
Method. Use the reciprocal relationship between tolerance and VIF:
The standard-error factor is the square root of the VIF:
Thus, relative to the stated reference design, this predictor’s coefficient variance is multiplied by 12.5 and its standard error by about 3.5355. The result signals substantial linear predictability of building age from the other predictors. It does not establish that the coefficient is biased, that the regression predictions are poor, or that the predictor must be removed.
Check. Tolerance is between 0 and 1, and its reciprocal is greater than 1. The calculation would be inconsistent if it produced a VIF below 1.
Using Sums of Squares and Reading Results in Context
Some regression output gives auxiliary-regression sums of squares rather than \(R_j^2\). In that case, calculate \(R_j^2\) from the auxiliary residual and total sums of squares, then take the reciprocal of the unexplained proportion. Keep the response and predictor roles straight: the response in this calculation is the predictor being diagnosed.
Worked Example: Finding a VIF From Auxiliary Sums of Squares
A hypothetical insurer models the annual labor cost of inspections using property size and distance to a service center. In the auxiliary regression of property size on distance, the centered total sum of squares for property size is 80, and the residual sum of squares is 24. For distance, the corresponding auxiliary regression has total sum of squares 50 and residual sum of squares 30. Calculate each \(R^2\), tolerance, and VIF, then identify which predictor is more linearly predictable from the other. Assume the main model is estimable and satisfies the ordinary least squares variance assumptions.
Method. For each auxiliary regression, compute the explained proportion as \(1-\operatorname{SSE}/\operatorname{SST}\). Then calculate tolerance and VIF.
For property size, the auxiliary \(R^2\) is
Its tolerance is \(1-0.70=0.30\), and its VIF is \(1/0.30=3.3333\ldots\), or \(3.3333\) to four decimal places.
For distance, the auxiliary \(R^2\) is
Its tolerance is \(1-0.40=0.60\), and its VIF is \(1/0.60=1.6666\ldots\), or \(1.6667\) to four decimal places. Property size is more linearly predictable from the other predictor because its auxiliary \(R^2\) is larger and its tolerance is smaller. Its VIF is also larger.
Check. For each predictor, \(\operatorname{SSE}\leq\operatorname{SST}\), so the auxiliary \(R^2\) lies between 0 and 1. The property-size residual proportion \(24/80=0.30\) matches its tolerance, and \(1/0.30=3.3333\ldots\).
Limits, Cutoffs, and Common Traps
If a predictor is orthogonal to all the other predictors in the fitted design, its auxiliary \(R^2\) is 0 and its VIF is 1. If it is increasingly well predicted by the other predictors, \(R_j^2\) approaches 1, tolerance approaches 0, and VIF grows without bound. At exact linear dependence, \(R_j^2=1\), the coefficient cannot be separately estimated in the usual full-rank regression, and the VIF is infinite.
For two predictors with an intercept, the auxiliary \(R^2\) for either predictor is the squared correlation between them. Thus the VIF is \(1/(1-r^2)\), where \(r\) is their correlation. The sign of the correlation does not matter: a correlation of \(-0.8\) and one of \(0.8\) yield the same VIF, since both have \(r^2=0.64\).
Rules of thumb sometimes flag VIFs above 5 or 10 for closer review. These are prompts for investigation, not universal boundaries between acceptable and unacceptable models. A VIF’s practical importance depends on the purpose of the analysis, the precision needed for individual coefficients, the sample information, and the structure of the predictors. In a predictive task, a high VIF does not by itself prove predictions are unreliable. For inference about separate effects, a high VIF may make estimates less precise, so the researcher should examine the model and the scientific question rather than apply a mechanical deletion rule.
VIFs also depend on model specification. Adding, removing, or recoding predictors can change an auxiliary regression. Multiplying a predictor by a nonzero constant does not change its auxiliary \(R^2\), but replacing it with a nonlinear transformation generally can. For a categorical predictor represented by multiple indicator columns, individual column VIFs can depend on the chosen coding; interpret them with that representation in mind.
Common mistake: taking the reciprocal of the auxiliary \(R^2\). Suppose \(R_j^2=0.75\). The incorrect calculation \(1/0.75=1.3333\) uses the explained proportion. The correct calculation uses the unexplained proportion:
The error reverses the meaning of the diagnostic: it can even suggest a value below 1 when \(R_j^2\) is low, although a standard VIF cannot be below 1. Another frequent error is reporting \(\sqrt{\operatorname{VIF}}\) as the variance factor. The VIF is the variance ratio; its square root is the standard-error ratio.
Check Your Understanding
For each problem, use the auxiliary-regression definition and distinguish variance factors from standard-error factors.
- An auxiliary regression for a claim-cost predictor has \(R_j^2=0.36\). Find its tolerance and VIF. (Answer: tolerance \(=0.64\); VIF \(=1.5625\).)
- A predictor has tolerance \(0.20\). Find its VIF and its standard-error factor relative to the orthogonal-predictor reference, rounded to four decimal places. (Answer: VIF \(=5\); standard-error factor \(=\sqrt{5}\approx2.2361\).)
- For an auxiliary regression, \(\operatorname{SST}_j=80\) and \(\operatorname{SSE}_j=20\). Calculate \(R_j^2\), tolerance, and VIF. (Answer: \(R_j^2=0.75\); tolerance \(=0.25\); VIF \(=4\).)
- In a two-predictor regression with an intercept, the predictors have correlation \(-0.60\). Find the VIF for either predictor. (Answer: \(1/(1-(-0.60)^2)=1/0.64=1.5625\).)
- Two predictors have auxiliary \(R^2\) values of \(0.90\) and \(0.50\), respectively. Find their VIFs and state which has the larger standard-error factor relative to the orthogonal reference. (Answer: VIFs \(=10\) and \(2\); the first predictor has the larger factor, \(\sqrt{10}\approx3.1623\) versus \(\sqrt{2}\approx1.4142\).)