Tutorials › Regression Analysis (Undergraduate) › Computing Variance Inflation Factors

Home  /  Tutorials  /  Collinearity and Estimate Stability

Collinearity and Estimate Stability · Tutorial 798 of 991

Computing Variance Inflation Factors

Calculate VIFs from auxiliary-regression \(R^2\) values, with attention to the correct predictors, sample, arithmetic, and reporting.

Intermediate 9 min read

What You'll Learn

  • Define the auxiliary regression used to calculate a predictor’s VIF
  • Apply the formula \(1/(1-R_j^2)\) to one or several predictors
  • Calculate and round VIFs from hypothetical regression outputs
  • Check that auxiliary regressions use the same observations and predictor set
  • Recognize what an \(R_j^2\) of zero or one means for the calculation
  • Avoid confusing an auxiliary regression’s \(R^2\) with the outcome model’s \(R^2\)

From Auxiliary Regression Output to VIF

A variance inflation factor (VIF) is calculated separately for each predictor in a multiple regression. The calculation uses how well that predictor can be explained by the other predictors in the model. In practice, an auxiliary regression supplies the needed \(R^2\), and one formula converts that value to a VIF.

The essential bookkeeping is to identify which variable is the response in each auxiliary regression. To calculate the VIF for \(X_j\), make \(X_j\) the response and use all the other predictors in the fitted model as explanatory variables. Do not use the original outcome as the auxiliary response.

Definition: For a model with predictors \(X_1,\ldots,X_p\), the auxiliary regression for \(X_j\) is the regression of \(X_j\) on the other predictors \(X_k\), \(k\ne j\), using the same observations and including an intercept. Let \(R_j^2\) be its coefficient of determination. The variance inflation factor for \(X_j\) is $$ \operatorname{VIF}_j=\frac{1}{1-R_j^2}. $$

Here, \(R_j^2\) is a proportion from 0 to 1, not a percentage. For example, an auxiliary \(R^2\) of 0.64 is entered as 0.64, not as 64. The formula applies predictor by predictor: a model with four predictors generally requires four auxiliary regressions and produces four VIFs.

Set Up the Calculation Correctly

Suppose the original regression includes a quantitative outcome \(Y\) and predictors \(X_1,\ldots,X_p\). For observation \(i\), the auxiliary regression for predictor \(X_j\) can be written as

$$ x_{ij}=\gamma_{0j}+\sum_{k\ne j}\gamma_{kj}x_{ik}+e_{ij}. $$

The response values in this regression are the observed values of predictor \(X_j\). The other predictors in the original model appear on the right-hand side. The auxiliary regression is not a new model for the outcome \(Y\); it is a calculation describing the linear predictability of one predictor from the others.

The usual VIF calculation assumes the auxiliary regression includes an intercept and uses the same rows and the same modeled predictor set as the original regression. If the original model uses transformed predictors or interaction terms, those modeled terms belong in the auxiliary regression just as they appear in the original model. A different sample or an omitted term can produce an \(R_j^2\) that does not correspond to the model whose VIF is being reported.

VIF is a property of the predictor design for a specified model and sample; the observed values of \(Y\) do not enter the formula. Under the standard linear-model conditions used to relate VIF to coefficient variance, the VIF is the variance multiplier associated with predictor redundancy. Here the practical task is to calculate that multiplier from the auxiliary \(R^2\), not to assess a particular cutoff or explain its consequences.

Worked Example: Calculating One VIF

In a hypothetical regression, an auxiliary regression predicts a participant’s weekly study hours from the model’s other predictors. Its reported \(R^2\) is 0.75. Calculate the VIF for weekly study hours.

The auxiliary regression has the correct target: weekly study hours is its response, and the remaining original predictors are its explanatory variables. Substitute \(R_j^2=0.75\):

$$ \operatorname{VIF}_j =\frac{1}{1-R_j^2} =\frac{1}{1-0.75} =\frac{1}{0.25} =4. $$

The VIF for weekly study hours is 4. This is the numerical result of the formula for this predictor and this model specification. It is not the \(R^2\) of the regression predicting the outcome.

Calculate a VIF for Every Predictor

When a model has several predictors, do not calculate one shared VIF by combining their auxiliary \(R^2\) values. Each predictor has its own auxiliary regression and its own value \(R_j^2\). The calculations are repetitive, so a table is a useful way to keep the predictor, its auxiliary \(R^2\), and the resulting VIF together.

1
List the modeled predictors.
Record every predictor included in the regression whose VIFs are needed.
2
Match each predictor to its auxiliary regression.
For a given predictor, use it as the response and all the other modeled predictors as explanatory variables. Use the same observations and include an intercept.
3
Record the auxiliary \(R^2\).
Use the coefficient of determination from that auxiliary regression, expressed as a decimal.
4
Apply the VIF formula.
Calculate \(1/(1-R_j^2)\) separately for each predictor, then report the result to a suitable number of significant figures.

Worked Example: VIFs for Four Quantitative Predictors

Consider a hypothetical model predicting a continuous outcome from four quantitative predictors: age in years, work experience in years, training hours, and weekly workload in hours. Suppose separate auxiliary regressions, each using the same observations and the other three predictors as explanatory variables, produce the following \(R^2\) values. Calculate all four VIFs.

Auxiliary responseAuxiliary \(R_j^2\)CalculationVIF
Age (years)0.64\(1/(1-0.64)=1/0.36\)2.778
Work experience (years)0.59\(1/(1-0.59)=1/0.41\)2.439
Training hours0.21\(1/(1-0.21)=1/0.79\)1.266
Weekly workload (hours)0.08\(1/(1-0.08)=1/0.92\)1.087

For example, age has \(R_j^2=0.64\), so its VIF is \(1/0.36=2.777\ldots\), or 2.778 to four significant figures. Work experience has \(R_j^2=0.59\), giving \(1/0.41=2.439\ldots\), or 2.439. The other two calculations are \(1/0.79=1.2658\ldots\) and \(1/0.92=1.0869\ldots\).

Each row describes a different auxiliary regression. For instance, age is predicted by work experience, training hours, and weekly workload in its auxiliary regression; work experience is predicted by age, training hours, and weekly workload in its own. The four \(R^2\) values therefore need not be equal, even though each regression uses the same overall predictor set.

The units shown for the predictors help identify them, but they are not inserted into the VIF formula. The calculation uses only the auxiliary \(R^2\). Report the VIF alongside its predictor so readers can tell which auxiliary regression produced it.

Why the Denominator Is \(1-R_j^2\)

The coefficient of determination in an auxiliary regression is the fraction of the variation in its response explained by its explanatory variables. With an intercept, the unexplained fraction is \(1-R_j^2\). The VIF is the reciprocal of that fraction:

$$ R_j^2=\frac{\text{explained variation in }X_j}{\text{total variation in }X_j} \quad\Longrightarrow\quad 1-R_j^2=\frac{\text{unexplained variation in }X_j}{\text{total variation in }X_j}, \qquad \operatorname{VIF}_j=\frac{1}{1-R_j^2}. $$

This identity also gives a quick check on the arithmetic. Since \(0\le R_j^2<1\) for a non-exact auxiliary fit, \(1-R_j^2\) is positive and at most 1, so its reciprocal is at least 1. As \(R_j^2\) increases, the denominator shrinks and the calculated VIF increases. These checks can catch a reversed fraction or a percentage entered incorrectly.

Worked Example: Checking a High Auxiliary \(R^2\)

In another hypothetical model, the auxiliary regression for monthly sales calls has \(R_j^2=0.92\). Calculate the VIF and check whether the result is consistent with the formula.

Substitution gives

$$ \operatorname{VIF}_j =\frac{1}{1-0.92} =\frac{1}{0.08} =12.5. $$

The denominator, 0.08, is positive and less than 1, so the reciprocal is greater than 1, as expected. A calculation such as \(1/0.92\) would be incorrect because it leaves out the \(1-R_j^2\) denominator. The value 12.5 is the computed VIF for monthly sales calls in this hypothetical model.

Boundary Cases and Reporting Checks

Two boundary cases are useful when checking a calculation. If \(R_j^2=0\), the other predictors explain none of the variation in \(X_j\) under the auxiliary regression, and

$$ \operatorname{VIF}_j=\frac{1}{1-0}=1. $$

At the other boundary, \(R_j^2=1\) makes the denominator zero. There is then no finite VIF: the auxiliary predictors reproduce \(X_j\) exactly, corresponding to exact collinearity in the original predictor set. The coefficient for that redundant predictor is not separately identifiable in the usual full model.

In software output, an \(R^2\) rounded to 1.000 may represent an exact value of 1 or a value just below 1. If the reported precision is insufficient to distinguish these cases, do not claim a precise finite VIF from the rounded number. Obtain more precise auxiliary output or investigate whether the predictor is an exact linear combination of the others.

For routine reporting, give each predictor’s name, auxiliary \(R_j^2\), and VIF, with a consistent rounding convention. VIF itself has no measurement units. Keeping the \(R^2\) beside the VIF makes it easier to audit the calculation and spot a mismatched auxiliary regression.

Common Mistakes

  • Using the outcome regression’s \(R^2\). The wrong reasoning is that the original model’s fit must determine every VIF. The correction is to use the auxiliary \(R_j^2\) for each predictor, with that predictor as the auxiliary response.
  • Forgetting to subtract from 1. The wrong calculation is \(1/R_j^2\). The correct formula is \(1/(1-R_j^2)\).
  • Entering an \(R^2\) percentage as a whole number. If a report gives 64%, enter 0.64 in the formula, not 64.
  • Running only one auxiliary regression for the whole model. Each predictor has a separate response and generally a separate \(R_j^2\); calculate one VIF per predictor.
  • Changing the sample or predictor set between regressions. Use the same observations and the other predictors from the specified original model in every auxiliary regression.
Key takeaway: For each predictor \(X_j\), obtain \(R_j^2\) by regressing it on all the other predictors in the same model, using the same observations and an intercept. Then calculate \(\operatorname{VIF}_j=1/(1-R_j^2)\) and report the result with the predictor it belongs to.

Check Your Understanding

Use the formula and setup from this tutorial to answer the questions.

  1. An auxiliary regression for \(X_1\) has \(R_1^2=0.40\). What is its VIF?
  2. For a model with predictors \(X_1,X_2,X_3\), which variables belong on the right-hand side of the auxiliary regression for \(X_2\)?
  3. An auxiliary regression reports \(R_j^2=0.88\). Write the VIF calculation and give the result to three significant figures.
  4. Why is the original outcome-model \(R^2\) not the value to substitute when calculating a predictor’s VIF?
  5. What does it mean for the VIF formula if an auxiliary regression has \(R_j^2=1\)?

Previous

Next

Back to Tutorials