The Kaiser Rule for Correlation-Based PCA

Why the Kaiser Rule Uses One as Its Cutoff

Principal components analysis (PCA) produces components with variances summarized by eigenvalues. To decide how many components to retain, the Kaiser Rule uses a simple threshold: retain components whose eigenvalues are greater than one. That cutoff is meaningful when PCA is performed on the correlation matrix, which corresponds to standardizing each variable to have variance one.

The rule compares each component’s variance with the variance of one standardized input variable. It does not test whether a component is statistically significant, and it does not establish whether a retained component will improve predictions. It is a retention heuristic: a useful benchmark to examine, not a complete decision procedure.

Definition: For PCA based on the correlation matrix of \(p\) standardized variables, the Kaiser Rule retains component \(j\) if its eigenvalue satisfies \(\lambda_j > 1\). An eigenvalue equal to one does not pass the strict threshold. Each eigenvalue is the variance of its corresponding principal-component score; the total variance across all \(p\) components is \(p\).

The Unit-Variance Benchmark

Suppose the \(p\) input variables have been centered and scaled so each has sample variance one. Their sample correlation matrix \(R\) is then the covariance matrix of the standardized variables, and every diagonal entry of \(R\) is one. The trace of a matrix is the sum of its diagonal entries, so \(\operatorname{tr}(R)=p\).

The eigenvalues of \(R\), written in descending order as \(\lambda_1 \ge \lambda_2 \ge \cdots \ge \lambda_p\), are nonnegative and sum to the trace. Therefore, their sum is \(p\), and their average is one. This is the source of the Kaiser Rule’s cutoff: an eigenvalue above one means that component has more variance than the average variance of a single standardized input variable.

\sum_{j=1}^{p}\lambda_j=\operatorname{tr}(R)=p \qquad\Longrightarrow\qquad \frac{1}{p}\sum_{j=1}^{p}\lambda_j=1

The component-variance interpretation follows directly from the definition of PCA. Let \(Z\) be the matrix of centered, standardized observations, and let \(a_j\) be a unit-length eigenvector of \(R=Z^\mathsf{T}Z/(n-1)\), where \(n\) is the number of observations. A row of \(Z\) is an observation \(z\), and its score on component \(j\) is \(t_j=z^\mathsf{T}a_j\). Since \(a_j\) is a unit vector and \(R a_j=\lambda_j a_j\), the variance of that score is:

\operatorname{Var}(t_j) = a_j^\mathsf{T}R a_j = a_j^\mathsf{T}(\lambda_j a_j) = \lambda_j a_j^\mathsf{T}a_j = \lambda_j

The derivation depends on using standardized variables and the correlation matrix. In that setting, each original variable has variance one, and the component score has variance \(\lambda_j\). The comparison is consequently on a common, dimensionless scale.

Interpretation: An eigenvalue above one indicates that a component accounts for more variance than the average single standardized input variable. It does not mean that the component explains more variance than every individual input variable, nor does it establish that the component is useful for a particular prediction task.

Applying the Rule

The calculation is short, but the interpretation requires care. Confirm that the PCA is based on standardized variables, order the eigenvalues, apply the strict \(>1\) threshold, and then, if useful, calculate the fraction of total variance represented by the retained components. That fraction is the sum of retained eigenvalues divided by \(p\).

Check the PCA scale.
Confirm that the input variables were standardized and the eigenvalues are from their correlation matrix.

Order and compare.
List eigenvalues from largest to smallest and retain those strictly greater than one. An eigenvalue equal to one does not qualify.

Check the variance accounting.
Verify that all eigenvalues sum to \(p\), then divide the retained eigenvalue sum by \(p\) to find the retained variance fraction.

Worked Example: A Three-Variable Correlation Pattern

A hypothetical risk team standardizes three numeric service measures and performs PCA on their correlation matrix. Suppose every pair of measures has correlation \(0.5\). Apply the Kaiser Rule and calculate the variance fraction retained. Name the method and a common mistake the calculation avoids.

Method. Use the eigenvalues of the correlation matrix, because all three variables have been standardized. The matrix is:

R= \begin{pmatrix} 1 & 0.5 & 0.5\\ 0.5 & 1 & 0.5\\ 0.5 & 0.5 & 1 \end{pmatrix}

Step 1: Find the eigenvalues. The vector \((1,1,1)^\mathsf{T}\) is an eigenvector: each row sum is \(1+0.5+0.5=2\), so its eigenvalue is \(2\). For any vector whose entries sum to zero, multiplication by \(R\) gives \(0.5\) times that vector. There are two independent directions with entries summing to zero, so the other two eigenvalues are \(0.5\) and \(0.5\).

Step 2: Apply the threshold. The ordered eigenvalues are \(2, 0.5, 0.5\). Only \(2>1\), so the rule retains one component.

Step 3: Calculate the retained fraction. The total variance is \(p=3\), and the retained eigenvalue sum is \(2\). Thus the retained fraction is \(2/3=0.6667\), or \(66.67\%\) to two decimal places.

Check. The eigenvalues sum to \(2+0.5+0.5=3\), matching the matrix trace. The common mistake avoided is treating every correlation-based PCA as though it must retain several components: here the strict rule retains only one.

Worked Example: Two Separate Pairs of Measures

In a hypothetical portfolio-monitoring analysis, four standardized measures form two pairs. The correlation within the first pair is \(0.8\), the correlation within the second pair is \(0.2\), and correlations between pairs are zero. Use the Kaiser Rule to determine the number of components retained and their total variance fraction.

Method. The correlation matrix has two \(2 \times 2\) blocks. For a pair with correlation \(r\), the vectors \((1,1)^\mathsf{T}\) and \((1,-1)^\mathsf{T}\) have eigenvalues \(1+r\) and \(1-r\), respectively.

R= \begin{pmatrix} 1 & 0.8 & 0 & 0\\ 0.8 & 1 & 0 & 0\\ 0 & 0 & 1 & 0.2\\ 0 & 0 & 0.2 & 1 \end{pmatrix}

Step 1: Calculate each pair’s eigenvalues. For the first pair, \(1+0.8=1.8\) and \(1-0.8=0.2\). For the second pair, \(1+0.2=1.2\) and \(1-0.2=0.8\). In descending order, the four eigenvalues are \(1.8, 1.2, 0.8, 0.2\).

Step 2: Apply the threshold. The first two eigenvalues are greater than one, so the rule retains two components.

Step 3: Calculate the retained fraction. The retained sum is \(1.8+1.2=3.0\). There are four standardized variables, so total variance is four. The retained fraction is \(3.0/4=0.75\), or \(75\%\).

Check. All four eigenvalues sum to \(1.8+1.2+0.8+0.2=4\). The common mistake avoided is comparing the retained eigenvalue sum with the number of retained components; the denominator for the total variance fraction is the number of original standardized variables, \(p=4\).

Why the Rule Can Mislead When Used Alone

The value one is an average-variance reference, not a universal boundary between important and unimportant structure. A component only slightly above one passes the rule, even if its advantage over the benchmark is small. Conversely, a component just below one fails, even though its variance may be close to that of a component just above the cutoff. The rule treats those near-threshold values differently despite the small numerical difference.

The next example shows how the rule can retain components for a weak correlation pattern. It does not show that retaining those components is necessarily wrong. Rather, it shows why reporting the rule’s result without considering the size and context of the eigenvalues can give an incomplete account.

Worked Example: Weak Pairwise Correlations Still Pass

A hypothetical team standardizes four indicators. The indicators form two separate pairs, with correlation \(0.1\) within each pair and zero correlation between pairs. Apply the Kaiser Rule, find the retained variance fraction, and explain why the result should not be treated as a complete assessment of component importance.

Method. As in the preceding example, each pair has eigenvalues \(1+r\) and \(1-r\). Here \(r=0.1\).

Step 1: Calculate the eigenvalues. Each pair gives \(1+0.1=1.1\) and \(1-0.1=0.9\). Across the two pairs, the ordered eigenvalues are \(1.1, 1.1, 0.9, 0.9\).

Step 2: Apply the threshold. Both \(1.1\) eigenvalues exceed one. The rule therefore retains two components.

Step 3: Calculate the retained fraction. The retained sum is \(1.1+1.1=2.2\). Total variance is \(p=4\), so the retained fraction is \(2.2/4=0.55\), or \(55\%\).

Interpretation and check. The rule retains two components even though the pairwise correlations are only \(0.1\). The eigenvalues sum to \(1.1+1.1+0.9+0.9=4\), as required. The common mistake avoided is treating “greater than one” as equivalent to “strong” or “important”: the calculation establishes only that each retained component exceeds the average single-variable variance benchmark.

Conditions and Common Exam Traps

The Kaiser Rule’s interpretation depends on its scale. Standardization gives each input variable variance one; consequently, the average input-variable variance is one. With PCA based on an unstandardized covariance matrix, the variables may have different variances and the eigenvalue-one threshold no longer has that interpretation.

For example, suppose two uncorrelated variables have variances \(25\) and \(4\). Their covariance matrix is diagonal, with eigenvalues \(25\) and \(4\). Applying the Kaiser Rule directly would retain both because both eigenvalues exceed one. But those numbers reflect the variables’ measurement scales, not evidence that both components exceed a meaningful standardized-variable benchmark. After standardizing, the correlation matrix is the identity matrix and has eigenvalues \(1,1\); neither eigenvalue is strictly greater than one.

This example also illustrates a useful boundary case. If all standardized variables are uncorrelated, every eigenvalue is exactly one. The strict Kaiser Rule retains no components. That outcome follows from the threshold and does not mean that the original variables have no variance; they each have variance one.

Common trap: Applying the cutoff to covariance-matrix eigenvalues because their numerical values exceed one. The correction is to use the Kaiser Rule only for correlation-based PCA, or otherwise define a justified benchmark for the variables’ original scales. With correlation-based PCA, retain only eigenvalues strictly greater than one.

The rule also has no response variable: it evaluates variance structure in the predictors, not predictive performance. Its retained components may or may not be useful for a later model. Nor does the threshold alone account for uncertainty in estimated correlations; sample eigenvalues can vary with the observed data. These limitations matter particularly when a decision depends heavily on components close to one.

A careful report should therefore identify that PCA used standardized variables, give the ordered eigenvalues, state the strict retention threshold, and report the retained variance fraction. It should describe the result as the Kaiser Rule’s recommendation, not as proof that the selected number of components is uniquely correct. This interpretation is aligned with the SOA Exam PA (Predictive Analytics) syllabus.

Check Your Understanding

For each problem, assume the stated eigenvalues are valid and ordered unless the question gives a correlation structure. Apply the strict Kaiser threshold where applicable.

A correlation-based PCA has three eigenvalues \(1.6, 0.9, 0.5\). How many components pass the Kaiser Rule, and what percentage of total variance do they explain? (Answer: 1 component; \(1.6/3=53.33\%\).)
A standardized four-variable PCA has eigenvalues \(1.7, 1.0, 0.8, 0.5\). How many components pass, and why does the eigenvalue equal to one not pass? (Answer: 1 component; the rule requires an eigenvalue strictly greater than one.)
Two uncorrelated variables have variances \(25\) and \(4\). What are the eigenvalues after standardizing the variables, and how many pass the Kaiser Rule? (Answer: \(1,1\); neither passes the strict threshold.)
A correlation-based PCA of five variables has eigenvalues \(2.1, 1.2, 0.9, 0.5, 0.3\). How many components pass, and what percentage of total variance do they explain? (Answer: 2 components; \((2.1+1.2)/5=66\%\).)
Four standardized variables form two independent pairs, each with within-pair correlation \(0.3\). Find the eigenvalues, the number retained, and the retained variance fraction. (Answer: Eigenvalues \(1.3, 1.3, 0.7, 0.7\); 2 components; \((1.3+1.3)/4=65\%\).)

What You'll Learn