Scree Plots and Eigenvalue Elbows

What a Scree Plot Shows

Principal component analysis (PCA) produces an ordered sequence of eigenvalues, one for each principal component. A scree plot graphs these eigenvalues against their component numbers: component number on the horizontal axis and eigenvalue on the vertical axis. Because the components are ordered from greatest to least eigenvalue, the plotted sequence cannot increase.

An eigenvalue measures the variance accounted for by its corresponding component, given the data matrix used for PCA. Thus, a steep early decline means that successive components initially account for smaller amounts of variance. A bend after which the points descend more gradually is a possible elbow. The elbow can help an analyst identify a useful candidate number of components, but it is not a formal test or a universal rule.

Definition: A scree plot is a plot of the PCA eigenvalues \(\lambda_1,\lambda_2,\ldots,\lambda_p\), arranged in nonincreasing order, against component numbers \(1,2,\ldots,p\). An elbow is a visually apparent change from a relatively steep decline to a more gradual decline. Its location is a judgment, not a mathematically defined cutoff.

Why the Eigenvalues Belong on the Vertical Axis

The eigenvalue has a direct variance interpretation. To see why, let \(Z\) be an \(n \times p\) data matrix after the centering and any scaling chosen for the PCA, and let \(S=Z^\mathsf{T}Z/(n-1)\) be its sample covariance matrix. Let \(v_j\) be a unit eigenvector of \(S\) with eigenvalue \(\lambda_j\). The score for observation \(i\) on component \(j\) is the projection of that observation onto \(v_j\).

The sample variance of those scores is the squared projected data, summed over observations and divided by \(n-1\). Since \(v_j\) is an eigenvector, that variance equals \(\lambda_j\):

\begin{aligned} \operatorname{Var}(\text{scores on component }j) &= \frac{1}{n-1}\sum_{i=1}^{n}(z_i^\mathsf{T}v_j)^2\\ &= v_j^\mathsf{T}\left(\frac{Z^\mathsf{T}Z}{n-1}\right)v_j\\ &= v_j^\mathsf{T}Sv_j\\ &= v_j^\mathsf{T}(\lambda_j v_j)\\ &= \lambda_j v_j^\mathsf{T}v_j\\ &= \lambda_j. \end{aligned}

The last equality uses the unit-length convention \(v_j^\mathsf{T}v_j=1\). This derivation explains why the scree plot is about eigenvalues rather than component numbers alone: each plotted height represents the variance along that component. If PCA is based on a correlation matrix, the eigenvalues are on that matrix's scale; if it is based on a covariance matrix, their scale depends on the measurement units and variances in the input data.

Constructing and Reading the Plot

Start with the eigenvalues in their PCA order, from largest to smallest. Plot one point per component and connect adjacent points with straight line segments if a connected scree plot is used. Label both axes, and make clear whether the eigenvalues came from a covariance matrix or a correlation matrix. Do not reorder the values by some other criterion: the component number is part of the interpretation.

One practical way to make a visual judgment more deliberate is to calculate the adjacent drops:

D_j=\lambda_j-\lambda_{j+1},\qquad j=1,\ldots,p-1.

These differences do not create an automatic elbow test. They simply make the pattern easier to describe. If the drops are large through a particular component and then become consistently much smaller, that change supports a candidate bend after that component. If drops remain similar, or decrease gradually, the plot may have no convincing elbow. When eigenvalues tie, their difference is zero, but a tie alone does not establish where components should be retained.

Confirm what was analyzed.
Check whether the PCA used a covariance or correlation matrix, and verify the preprocessing that determines the eigenvalue scale.

Plot the ordered eigenvalues.
Use component number on the horizontal axis and eigenvalue on the vertical axis; retain the PCA ordering.

Look for a change in slope.
Describe where a steep early decline, if present, gives way to a more gradual tail. Compare adjacent drops as a check on the visual description.

State the uncertainty.
Report an elbow as a candidate location, or say that no clear elbow is apparent. Do not present the plot as if it proves a unique answer.

Worked Examples

Worked Example: A Pronounced Bend After Two Components

A hypothetical insurer standardizes six related claim-management measurements before fitting PCA to a correlation matrix. The ordered eigenvalues are \(3.20, 1.80, 0.50, 0.25, 0.15, 0.10\). Construct the key numerical comparisons and identify a candidate elbow.

Method. Treat the listed values as eigenvalues, plot them in the given order, and calculate each adjacent drop. The order already satisfies the required nonincreasing arrangement.

Step 1: Calculate the drops. From component 1 to 2, \(D_1=3.20-1.80=1.40\). From 2 to 3, \(D_2=1.80-0.50=1.30\). From 3 to 4, \(D_3=0.50-0.25=0.25\). From 4 to 5, \(D_4=0.25-0.15=0.10\). From 5 to 6, \(D_5=0.15-0.10=0.05\).

Step 2: Interpret the pattern. The plot falls sharply through the first three plotted points: the drops into components 2 and 3 are 1.40 and 1.30. After component 3, the remaining drops are 0.25, 0.10, and 0.05, which are substantially smaller. The candidate elbow is therefore after component 2, where the plotted decline changes from the large drop into component 3 to the smaller drops that follow.

The comparison shows why it is useful to locate the change in slope, rather than to label every initial steep segment as an elbow. A common mistake is to say “retain three” simply because the large drops occur before the third component. The bend is after component 2: the \(1.30\) drop is the drop into component 3, and the later, much smaller drops mark the tail. This is a visual candidate, not a proof that exactly two components must be used.

Worked Example: A Smooth Decline With No Clear Elbow

A hypothetical operations team performs PCA on five centered, unscaled measurements. Its ordered covariance-matrix eigenvalues are \(5,4,3,2,1\). Decide whether the scree plot supports a distinct elbow.

Method. Plot the ordered values and compare the adjacent drops rather than selecting a component count based only on the fact that the eigenvalues decline.

Step 1: Find each drop. The calculations are \(D_1=5-4=1\), \(D_2=4-3=1\), \(D_3=3-2=1\), and \(D_4=2-1=1\).

Step 2: Read the shape. Every segment has the same vertical drop of 1. The plotted sequence is a steady decline, not a steep initial descent followed by a visibly flatter tail. There is no distinct elbow supported by these values.

A common mistake is to force an elbow at the first component because it has the largest eigenvalue. The largest value identifies the first component as the one with the greatest variance; it does not by itself identify a bend in the sequence. The appropriate conclusion here is that the scree plot does not give a clear elbow-based cutoff.

Worked Example: Converting Component Standard Deviations to Eigenvalues

A PCA summary reports component standard deviations of \(2.5, 2.0, 1.0, 0.6,\) and \(0.4\), in component order. Construct the eigenvalue sequence needed for a scree plot and assess the likely bend.

Method. When the reported component standard deviations are the square roots of the PCA eigenvalues, square each value before constructing an eigenvalue scree plot. Do not plot the standard deviations and describe them as eigenvalues.

Step 1: Square the values. The eigenvalues are \(2.5^2=6.25\), \(2.0^2=4.00\), \(1.0^2=1.00\), \(0.6^2=0.36\), and \(0.4^2=0.16\). They remain in nonincreasing order.

Step 2: Calculate adjacent drops. The drops are \(6.25-4.00=2.25\), \(4.00-1.00=3.00\), \(1.00-0.36=0.64\), and \(0.36-0.16=0.20\).

Step 3: Interpret the result. The drop into component 3 is still large, at 3.00. The drops after component 3 fall to 0.64 and 0.20. A candidate bend is therefore after component 3, where the decline becomes much more gradual.

The check is that each eigenvalue is the square of its corresponding reported standard deviation, and the squared sequence remains ordered. The common mistake is to use \(2.5,2.0,1.0,0.6,0.4\) as eigenvalues without checking what the summary reports. That would change the vertical scale and the adjacent-drop calculations. Confirm the quantity being plotted before interpreting a scree plot.

Why an Elbow Is a Judgment Aid

A bend is useful because it gives a compact visual description of how the successive variance contributions change. It is not a universal rule for how many components to retain. Some data produce a sharp bend; others produce a smooth curve, multiple possible bends, or a short sequence where the shape is difficult to judge. Two analysts can reasonably disagree about the location of a subtle elbow, particularly when neighboring drops are similar.

Several practical checks help keep the interpretation measured:

Use the correct scale and quantity. Eigenvalues from covariance-based PCA depend on the scales and units of the input variables. The eigenvalues from correlation-based PCA have a different interpretation and scale. Know which matrix was analyzed before comparing plots.
Keep the plot definition clear. The conventional eigenvalue scree plot has eigenvalue on the vertical axis. A logarithmic vertical axis changes the visual spacing of values, so an apparent change in slope may look different; describe the axis scale if one is used.
Do not invent a cutoff when the curve is smooth. It is acceptable to report that no distinct elbow is visible. A candidate retention decision may require additional evidence or a stated practical purpose.
Separate description from decision. “There is a visible bend after component 2” describes a plot. “Two components are sufficient for this analysis” is a broader conclusion that the plot alone does not establish.

Key takeaway: A scree plot displays ordered eigenvalues so that the decline in component variances can be inspected. An elbow is evidence of a change in that decline, not a universal threshold. State what the plot suggests, how clearly it suggests it, and what it cannot establish on its own.

Check Your Understanding

For each problem, calculate or compare the requested eigenvalue pattern before interpreting its scree plot.

A PCA produces the ordered eigenvalues \(7,2,1,0.5\). Calculate all adjacent drops and state the candidate elbow, if any. (Answer: Drops are \(5,1,0.5\); a candidate elbow is after component 1, where the decline becomes smaller.)
The ordered eigenvalues are \(3.2,2.4,1.8,1.35,0.99\). Calculate the drops and decide whether the values show a distinct elbow. (Answer: Drops are \(0.8,0.6,0.45,0.36\); they decline gradually, so there is no distinct elbow.)
A plotted sequence has eigenvalues \(5,3,1,0.8,0.7,0.6\). Compare the large and small adjacent drops and identify the likely bend. (Answer: Drops are \(2,2,0.2,0.1,0.1\); the candidate elbow is after component 2.)
A software summary gives component standard deviations \(3,2,1,0.5\). Convert them to eigenvalues, calculate the drops, and identify a candidate elbow. (Answer: Eigenvalues are \(9,4,1,0.25\); drops are \(5,3,0.75\); a candidate elbow is after component 2.)
Two plots show eigenvalues \(4,2,1,0.5\) and \(12,6,3,1.5\), respectively, on linear vertical axes. Compare their shapes and explain whether the difference in eigenvalue scale alone changes the bend location. (Answer: The second sequence is three times the first at every component, so the relative shape and candidate bend after component 1 are the same; the eigenvalue scale differs, but scale alone does not change the bend location.)

What You'll Learn