Why Component Retention Is a Decision
PCA orders components by the amount of variation they account for, but it does not identify a universally correct number to keep. A scree plot can suggest where additional components contribute less; cumulative explained variance can show how much information a selected set summarizes; and parallel analysis can indicate whether observed eigenvalues exceed values expected under a reference null model. These are pieces of evidence, not interchangeable rules.
A defensible decision states the intended use first. A compact descriptive summary may impose a target for retained variance, while a conservative noise-screening summary may give more weight to parallel-analysis evidence. If the criteria conflict, the analyst should make that conflict explicit instead of presenting one threshold as proof that the others are irrelevant. This decision-making approach is aligned with the SOA Exam PA (Predictive Analytics) syllabus.
A Single PCA Output to Evaluate
Use the following hypothetical PCA output throughout. Six numeric variables were centered and scaled to unit variance before PCA, so this is PCA of a correlation matrix. The six observed eigenvalues sum to 6. The last column gives hypothetical 95th-percentile parallel-analysis reference eigenvalues, obtained by simulating data with the same sample size and number of variables under a null model of no common component structure.
| Component | Observed eigenvalue | Variance explained | Cumulative variance | Parallel-analysis reference |
|---|---|---|---|---|
| 1 | 2.70 | 45.00% | 45.00% | 1.24 |
| 2 | 1.55 | 25.83% | 70.83% | 1.13 |
| 3 | 0.82 | 13.67% | 84.50% | 1.05 |
| 4 | 0.48 | 8.00% | 92.50% | 0.98 |
| 5 | 0.28 | 4.67% | 97.17% | 0.91 |
| 6 | 0.17 | 2.83% | 100.00% | 0.84 |
The percentages are rounded to two decimal places in the table. For example, \(2.70/6=0.45\), or 45.00%, and \((2.70+1.55)/6=4.25/6\), or approximately 70.83%. The calculations below use the unrounded eigenvalues and totals.
What Each Retention Signal Says
The scree plot graphs the ordered eigenvalues against component number. Its shape can reveal a bend, or elbow, after which the remaining eigenvalues decline more gradually. An elbow is a visual judgment: a gradual curve may not have a single unambiguous bend, and the plot does not itself specify how much total variance is adequate.
Cumulative explained variance answers a different question: how much of the total variance is represented by the first \(m\) components? An analyst may set a target such as 80%, but the target is a decision criterion, not a guarantee that every retained component represents meaningful structure. The acceptable target depends on the use and information-loss tolerance.
Parallel analysis compares each observed eigenvalue with a reference eigenvalue for the same component position. Under the convention used here, retain a component when its observed eigenvalue exceeds the simulated 95th-percentile reference. This helps assess whether its size is larger than would commonly occur under the specified null model. It does not establish that a component has a useful interpretation or will serve every later purpose.
Why the Explained-Variance Formula Works
The formula for a component's variance proportion follows from how PCA partitions total variance. For centered data, PCA expresses the total variation across the original variables as the sum of the component variances. Those component variances are the eigenvalues. Therefore, the share attributed to component \(j\) is its variance divided by the sum of all component variances:
For correlation-based PCA with \(p\) unit-variance variables, the total variance is \(p\), because the diagonal of the correlation matrix contains \(p\) ones and its trace is \(p\). Here \(p=6\), so the denominator is 6. For PCA of a covariance matrix, the denominator is instead the sum of the original-variable variances, which need not equal the number of variables. The calculation must use the total variance for the PCA actually performed.
Worked Decisions for Different Uses
Worked Example: Meet a Variance Target While Reviewing Parallel Analysis
Suppose an analyst needs a compact descriptive summary of the six standardized variables and has set a minimum of 80% cumulative variance. Evaluate the eigenvalues and parallel-analysis references above. Choose a component count and state the evidence for and against it.
Method. Calculate the cumulative variance in order, then compare each observed eigenvalue with its parallel-analysis reference. The stated 80% target is the required criterion for this use; parallel analysis is additional evidence to discuss.
After two components, the cumulative amount is \(2.70+1.55=4.25\), so the proportion is \(4.25/6=0.708333\ldots\), or 70.83%. This is below 80%. After three components, the sum is \(4.25+0.82=5.07\), giving \(5.07/6=0.845\), or 84.50%. Thus three is the smallest count meeting the target.
For parallel analysis, component 1 has \(2.70>1.24\), and component 2 has \(1.55>1.13\). Component 3 has \(0.82<1.05\). The first two exceed their references, but the third does not. The scree eigenvalue drops are \(2.70-1.55=1.15\), \(1.55-0.82=0.73\), \(0.82-0.48=0.34\), \(0.48-0.28=0.20\), and \(0.28-0.17=0.11\). The drop becomes much smaller after the second component, though the curve does not make a uniquely decisive cutoff certain.
Decision and check. Retain three components for the stated descriptive summary because three is the minimum that reaches 80%. Report that the third component brings the total from 70.83% to 84.50%, but its eigenvalue is below the parallel-analysis reference. The decision meets the specified variance requirement; it should not be described as having unanimous support from all criteria. The common mistake avoided is claiming that a variance target alone proves every included component exceeds a null reference.
Worked Example: Choose a Conservative Parallel-Analysis Cutoff
A second analyst wants a conservative summary and has specified that only components exceeding their parallel-analysis references will be retained. What count results, and how much variance does it represent?
Method. Compare the observed and reference eigenvalues component by component. Retain the leading components for which the observed value is greater than the corresponding reference, and then calculate their cumulative variance.
For component 1, \(2.70>1.24\). For component 2, \(1.55>1.13\). For component 3, \(0.82<1.05\), so the stated rule stops at two components. The remaining comparisons also fail: \(0.48<0.98\), \(0.28<0.91\), and \(0.17<0.84\).
The retained eigenvalues sum to \(2.70+1.55=4.25\). Dividing by the total variance of 6 gives \(4.25/6=0.708333\ldots\), or 70.83%. The omitted variance is \(6-4.25=1.75\), equivalent to \(1.75/6=0.291666\ldots\), or 29.17%.
Decision and check. Retain two components under the analyst's explicitly conservative rule. They exceed the parallel-analysis references, and together account for 70.83% of total variance. This does not meet an 80% target, but no such target was stipulated for this use. The common mistake avoided is silently adding a cumulative-variance requirement that would change the question being answered.
Worked Example: Resolve a Hard Minimum That Conflicts With Parallel Analysis
A reporting team must represent at least 75% of total variance in a brief, fixed-format summary. It also wants to know whether every retained component exceeds the parallel-analysis reference. Determine the minimum count that meets the reporting requirement and describe the trade-off.
Method. First find the smallest count satisfying the hard variance minimum; then assess the parallel-analysis comparison for that selected count. Do not treat a desirable supporting criterion as if it were the stated minimum.
Two components explain \((2.70+1.55)/6=4.25/6=70.83\%\), which is less than 75%. Three explain \((2.70+1.55+0.82)/6=5.07/6=84.50\%\), which is at least 75%. Thus the minimum count meeting the reporting constraint is three.
The first two eigenvalues exceed their references, but the third does not: \(0.82<1.05\). The third component is necessary to meet the hard variance minimum, even though parallel analysis does not support retaining it under the stated 95th-percentile comparison. The report should state both facts and, if possible, explain that the selected count is driven by the reporting constraint.
Decision and check. Retain three for this fixed-format requirement, with an explicit caveat about the third component. The result is not a claim that the criteria agree; it is a constrained choice. The common mistake avoided is either ignoring the 75% requirement in favor of a preferred rule or claiming that meeting 75% makes the parallel-analysis result disappear.
Making the Decision Reproducible
A clear retention justification lets another analyst see how the conclusion follows from the output and the stated use. A useful sequence is to identify the PCA basis, calculate the variance totals, examine the scree pattern, compare parallel-analysis references, and then state the purpose-specific choice.
State whether PCA used a covariance or correlation matrix and identify the total variance. Here the six variables were standardized, so the correlation-matrix eigenvalues sum to 6.
Use the eigenvalues and the correct total to find the share represented by each possible leading set. Record any required target and the smallest count that meets it.
Describe the scree shape and compare observed eigenvalues with parallel-analysis references using the stated convention. Note where these signals agree or conflict with the variance target.
Select the count that fits the specified use, then state the trade-off. Do not imply that one criterion proves a component is meaningful for all purposes.
The decision depends on the PCA being evaluated. Parallel-analysis references are meaningful only in relation to the simulation design and comparison convention used to obtain them; the values should not be treated as universal constants. Likewise, an elbow is not a precise statistical cutoff. State the relevant setup and avoid reporting more certainty than the output supports.
Component retention is a dimension-reduction choice, not by itself a conclusion about what the components mean. The retained count should be accompanied by enough context to make the choice interpretable: the total variance represented, the evidence from the scree plot, the comparison with reference eigenvalues, and the objective that determined how conflicts were resolved.
Check Your Understanding
Use the hypothetical six-variable PCA output in this tutorial unless a question gives different values.
- A summary requires at least 90% cumulative variance. Find the smallest component count that meets the target and state whether each retained eigenvalue exceeds its parallel-analysis reference. (Answer: Four components; cumulative variance is 92.50%, and only components 1 and 2 exceed their references.)
- A conservative rule retains only components whose observed eigenvalues exceed the corresponding references. How many are retained, and what proportion of total variance do they explain? (Answer: Two components; \(4.25/6=70.83\%\).)
- A team specifies a 70% minimum and wants the smallest count meeting it. Calculate the selected count's cumulative variance and identify the parallel-analysis caveat. (Answer: Two components; 70.83%; both exceed their references, while component 3 is below its reference and is not needed for the minimum.)
- For the first three components, calculate the successive eigenvalue drops and identify where the largest drop occurs. What does that tell you, and what does it not establish? (Answer: The drops are 1.15, 0.73, and 0.34; the largest is from component 1 to 2. This describes the observed decline but does not establish a unique retention cutoff.)
- A note says, “Three components are statistically supported because they explain 84.50%.” Identify the error and give a more accurate statement. (Answer: The variance share does not establish parallel-analysis support; three explain 84.50%, but the third observed eigenvalue, 0.82, is below its 1.05 reference.)