Scree Plots and Explained Variance

Why Summarize a PCA Spectrum?

PCA replaces a set of measured variables with ordered, uncorrelated components. For a genomic profile, this can turn thousands of correlated measurements into a shorter sequence of directions that summarizes variation among samples. The spectrum of PCA eigenvalues answers a focused descriptive question: how much sample variance is associated with each successive component? A scree plot displays that spectrum, while explained-variance percentages express it on a common scale.

These summaries can help select a manageable number of components for description or visualization. They do not, by themselves, identify biological mechanisms, establish that a component is useful for prediction, or prove that a particular number of underlying factors exists. The choice is a purpose-dependent summary of the observed variance.

Eigenvalues and Explained Variance

Let \(X\) be an \(n \times p\) data matrix, with samples in rows and variables in columns. Assume the columns have been centered. If variables have also been scaled to unit sample variance, the PCA is based on the sample correlation matrix; otherwise, it is based on the sample covariance matrix. The earlier material on centering and scaling before PCA explains why this choice matters: the variance spectrum depends on the variables and their scales.

Write the sample covariance matrix as \(S=X^\mathsf{T}X/(n-1)\). Its eigenvalues are ordered from largest to smallest, \(\lambda_1 \geq \lambda_2 \geq \cdots \geq \lambda_p \geq 0\). With \(X=UDV^\mathsf{T}\), the singular-value decomposition of the centered matrix, the corresponding eigenvalues are \(\lambda_j=d_j^2/(n-1)\). If the data are scaled, this relation uses the scaled \(X\).

The total variance is \(\sum_{j=1}^{p}\lambda_j=\operatorname{tr}(S)\), the sum of the variances of the original variables. Thus \(r_j\) is a fraction between zero and one, and the fractions across all \(p\) components sum to one. Report \(100r_j\) as a percentage. The cumulative percentage for the first \(k\) components is \(100R_k\). If centered data have rank less than \(p\), some eigenvalues are zero; they contribute no variance, but including them keeps the denominator and the definition unambiguous.

The trace identity follows because the trace of a matrix equals the sum of its eigenvalues. A second way to see the interpretation is that the variance of the score on component \(j\) is \(\lambda_j\); the first \(k\) orthogonal component scores therefore account for \(\sum_{j=1}^{k}\lambda_j\) units of variance. Dividing by the total variance expresses that amount as a fraction of the variance in the original centered data.

Reading a Scree Plot

A scree plot places component number \(j\) on the horizontal axis and its eigenvalue \(\lambda_j\), or sometimes its explained-variance percentage, on the vertical axis. Since the eigenvalues are ordered, the plotted sequence cannot increase. A steep initial decline indicates that early components account for substantially more variance than later ones. A flatter tail indicates that successive components contribute more similar, often smaller, amounts.

An apparent elbow is a bend where a steep decline changes to a comparatively flat tail. One descriptive rule is to retain components up to that bend, treating the later components as adding relatively little variance one at a time. This is a visual judgment, not a uniquely defined mathematical cutoff. Different readers may reasonably identify different bends, especially when the decline is gradual or the plot is noisy.

A plot of eigenvalues and one of explained-variance percentages have the same ordering and shape up to a constant vertical rescaling, because each percentage is the corresponding eigenvalue divided by the same total. Cumulative explained variance is different: it can only increase, so it is useful for checking a target such as 80% but does not display an elbow in the same way.

The Scaling Decision Changes the Spectrum

Explained-variance percentages are relative to the variables used in the PCA. With covariance-based PCA, variables with larger measurement variances can dominate the total. With correlation-based PCA, each nonconstant variable is first scaled to unit variance, so each contributes equally to the total variance before accounting for correlations. Neither choice is universally correct: it depends on whether the original measurement scales and variances should influence the analysis.

Choosing a Descriptive Dimension

A defensible choice combines the variance summary with the purpose of the analysis. Common descriptive approaches include:

• Cumulative-variance target: choose the smallest \(k\) for which \(R_k\) meets a prespecified target. This is transparent, but the target is a convention tied to the use case, not a universal law.
• Scree elbow: retain components before the plot becomes comparatively flat. This reflects diminishing individual contributions, but the visual judgment may be ambiguous.
• Fixed display dimension: use a small number of components for a visualization, while reporting how much variance they explain. A display constraint is not evidence that omitted components contain no structure.

These approaches need not agree. If an elbow suggests three components but a stated 90% target requires six, report the disagreement rather than presenting one choice as inevitable. For a descriptive summary, the analyst might prioritize interpretability or simplicity and show the retained cumulative variance. For compression, a specified variance target may be more relevant. In either case, describe the rule and its consequences.

The average-eigenvalue rule sometimes used with correlation-based PCA retains components with eigenvalues greater than 1, since the total variance of \(p\) standardized variables is \(p\) and the average eigenvalue is 1. This is a heuristic, not a guarantee that retained components are meaningful or that discarded ones are unimportant. More formal methods for assessing factor number are a separate topic.

Diagnostics, Pitfalls, and Limits

Check the calculation and the data definition before drawing a conclusion. The eigenvalues should be nonnegative apart from tiny numerical errors, should be ordered from largest to smallest, and should sum to the total variance. The component percentages should sum to 100% up to rounding, and cumulative percentages should not decrease. If these checks fail substantially, verify the matrix used, the component ordering, the denominator, and whether the reported eigenvalues are covariance eigenvalues or have been rescaled.

When \(p\) is large relative to \(n\), a centered data matrix has rank at most \(n-1\), so some of the \(p\) eigenvalues are zero. This is expected, not a calculation error. Conversely, a component explaining a small percentage is not automatically negligible: a low-variance direction may be important for a specific scientific question. Explained variance describes variation, not biological importance, causal effect, prediction accuracy, or measurement quality.

A frequent mistake is to interpret “the first three components explain 80% of the variance” as “the first three components explain 80% of the genes” or “80% of the biology.” The percentage refers to total sample variance in the chosen, centered and possibly scaled variables. Another mistake is to treat an elbow or a percentage threshold as proof of the correct number of latent processes. They are descriptive decision aids, and their appropriateness depends on the goal and preprocessing.