Explained Variance Ratios

What an Explained-Variance Ratio Measures

A principal component is a direction through the variables, and its eigenvalue measures the variance of the component scores in that direction. An explained-variance ratio expresses that component's variance as a share of the total variance represented in the PCA. It is a descriptive summary of variation in the variables used to form the PCA; by itself, it is not a measure of predictive accuracy for some separate outcome.

The denominator must include the variance represented by all components in the chosen PCA. If only the first few components are retained, their ratios still use the total variance from the full eigenvalue list. This makes the ratios useful for describing how much of the original total variance is retained by those components.

Definition: Suppose a PCA of \(p\) variables has eigenvalues \(\lambda_1,\lambda_2,\ldots,\lambda_p\), ordered so that \(\lambda_1\geq\lambda_2\geq\cdots\geq\lambda_p\geq0\). The total variance is \(\sum_{j=1}^{p}\lambda_j\). The explained-variance ratio for component \(j\) is \(\lambda_j/\sum_{r=1}^{p}\lambda_r\). The cumulative explained-variance ratio through component \(k\) is \(\sum_{j=1}^{k}\lambda_j/\sum_{r=1}^{p}\lambda_r\).

The component-wise ratios are between 0 and 1 and sum to 1 across all components when the total variance is positive. Their percentage versions sum to 100%, subject to small differences from rounding. The cumulative ratio cannot decrease as components are added: each additional eigenvalue is nonnegative, so each addition either increases the cumulative amount or leaves it unchanged.

Why the Eigenvalues Give Total Variance

For covariance-matrix PCA, let \(S\) be the sample covariance matrix of the centered variables. Its diagonal entries are the sample variances of the individual variables. Therefore, the sum of those diagonal entries, \(\operatorname{tr}(S)\), is the total variance across the variables. The eigenvalues provide another way to calculate the same quantity.

Write the eigenvalue decomposition.
For a symmetric covariance matrix, write \(S=Q\Lambda Q^{T}\), where the columns of \(Q\) are orthonormal eigenvectors and \(\Lambda\) is diagonal with entries \(\lambda_1,\ldots,\lambda_p\).

Take the trace.
The trace is the sum of the diagonal entries. The identity \(\operatorname{tr}(AB)=\operatorname{tr}(BA)\) gives the following equality.

Use orthonormality.
Since \(Q^{T}Q=I\), the trace of \(S\) equals the sum of the diagonal entries of \(\Lambda\), which is the sum of its eigenvalues.

\operatorname{tr}(S) =\operatorname{tr}(Q\Lambda Q^{T}) =\operatorname{tr}(\Lambda Q^{T}Q) =\operatorname{tr}(\Lambda) =\sum_{j=1}^{p}\lambda_j.

The variance of the \(j\)th principal component score is \(\lambda_j\). Dividing that variance by the sum of all component variances therefore gives the share of total variance in that direction. For PCA based on a correlation matrix of \(p\) nonconstant variables, each diagonal entry is 1, so the trace, and hence the sum of the eigenvalues, is \(p\). For covariance-matrix PCA, the total instead equals the sum of the original variables' sample variances. In either case, use the eigenvalues from the PCA actually performed.

Worked Example: Ratios From a Covariance-Matrix PCA

A hypothetical risk analytics team performs PCA on three centered measurements in their original units. The covariance-matrix eigenvalues, in descending order, are 6, 3, and 1. Calculate each component's explained-variance ratio, the cumulative ratio through the second component, and the share not captured by the first two. Report ratios to four decimal places and percentages to two decimal places.

Method. Add all eigenvalues for the denominator, calculate each eigenvalue's share of that total, and then add the first two ratios. This avoids treating the retained eigenvalues as the full total.

Total variance. \(6+3+1=10\).

Component 1. \(6/10=0.6000\), or 60.00%.

Component 2. \(3/10=0.3000\), or 30.00%.

Component 3. \(1/10=0.1000\), or 10.00%.

Cumulative through component 2. \(0.6000+0.3000=0.9000\), or 90.00%. The share not captured by the first two is \(1-0.9000=0.1000\), or 10.00%.

Check. The three component ratios sum to \(0.6000+0.3000+0.1000=1.0000\), and the uncaptured 10.00% agrees with component 3's ratio. The first two components capture 90.00% of total variance in these three measurements, not necessarily 90.00% of variation in any external outcome.

Calculating and Reading Cumulative Ratios

The cumulative ratio through component \(k\) is the share captured by the first \(k\) components together. It is not a new ratio with a denominator consisting only of those first \(k\) eigenvalues. The denominator remains the total across all \(p\) components. Thus the cumulative ratio measures the fraction of the full variance retained when only the first \(k\) component directions are represented.

A useful check is to calculate the cumulative amount in either of two equivalent ways: add the first \(k\) component ratios, or divide the sum of the first \(k\) eigenvalues by the total eigenvalue sum. These must agree before rounding.

Worked Example: Cumulative Ratios From Correlation-Matrix PCA

A hypothetical portfolio study standardizes four measurements and performs PCA on their correlation matrix. The eigenvalues are 1.6, 1.0, 0.9, and 0.5. Calculate each component's ratio and the cumulative ratio through component 3. Use four decimal places for ratios and two decimal places for percentages.

Method. Because this is a correlation matrix for four nonconstant standardized variables, the total variance is 4. The eigenvalue sum confirms the denominator: \(1.6+1.0+0.9+0.5=4.0\).

Component 1. \(1.6/4.0=0.4000\), or 40.00%.

Component 2. \(1.0/4.0=0.2500\), or 25.00%. Its cumulative ratio with component 1 is \(0.4000+0.2500=0.6500\), or 65.00%.

Component 3. \(0.9/4.0=0.2250\), or 22.50%. The cumulative ratio through component 3 is \(0.4000+0.2500+0.2250=0.8750\), or 87.50%.

Component 4. \(0.5/4.0=0.1250\), or 12.50%. Including it gives \(0.8750+0.1250=1.0000\), or 100.00%.

Check. The first three eigenvalues sum to \(1.6+1.0+0.9=3.5\), and \(3.5/4.0=0.8750\), matching the cumulative calculation. These percentages describe variation in the standardized measurements, since the analysis used their correlation matrix.

Variance Captured Is Not Predictive Variance Explained

In PCA, “explained variance” has a specific meaning: it is variance in the input variables represented by component directions. PCA uses the variables supplied to it; an external response or target is not part of the explained-variance-ratio calculation. Consequently, a large ratio does not establish that the component predicts an outcome well, and a small ratio does not establish that it is useless for prediction.

For example, a low-variance direction may align closely with an outcome even if it accounts for little variation among the predictors. Conversely, the direction accounting for most predictor variation may have little association with that outcome. Assessing prediction requires examining the outcome and evaluating a predictive model; the PCA ratios alone cannot answer that question.

Worked Example: High Predictor Variance Does Not Guarantee Prediction

Suppose a hypothetical PCA of two centered predictor measurements produces orthogonal component scores \(Z_1\) and \(Z_2\), with variances 9 and 1. A separate centered outcome \(Y\) is defined by \(Y=Z_2\), and the component scores have covariance \(\operatorname{Cov}(Z_1,Z_2)=0\). Calculate the PCA ratios, then determine what these facts say about prediction of \(Y\) from either component. Treat the stated score variances and relationship as exact.

Method. First calculate the ratios using the total predictor variance. Then use the given relationship between the outcome and component scores; do not infer predictive value from the PCA ratios.

Total predictor variance. \(9+1=10\).

Ratios. For \(Z_1\), the ratio is \(9/10=0.9000\), or 90.00%. For \(Z_2\), it is \(1/10=0.1000\), or 10.00%.

Association with the outcome. Since \(Y=Z_2\), \(\operatorname{Var}(Y)=\operatorname{Var}(Z_2)=1\). Also, \(\operatorname{Cov}(Z_1,Y)=\operatorname{Cov}(Z_1,Z_2)=0\). Thus the component that captures 90.00% of predictor variance has zero covariance with this outcome.

Prediction using \(Z_2\). The relationship \(Y=Z_2\) means that using \(Z_2\) reproduces the outcome exactly in this constructed example. It is the component with a 10.00% explained-variance ratio, yet it fully determines \(Y\).

Check. The ratios still sum to 100.00%, as required. They summarize the predictors' variance; they do not rank components by predictive usefulness for \(Y\). The common mistake avoided here is interpreting “90% of predictor variance” as “90% of outcome variation predicted.”

Common Denominator Error and Practical Checks

A frequent calculation error is to divide each retained eigenvalue by the sum of only the retained eigenvalues. That calculation answers what fraction of the retained subset's variance each retained component represents. It does not give the component's explained-variance ratio relative to the original total.

Common exam trap: If the eigenvalues are 8, 2, and 1, the total is 11. The correct ratio for the first component is \(8/11\), approximately 0.7273. If only the first two components are retained, using \(8/(8+2)=0.8000\) is incorrect for the explained-variance ratio: it omits the third eigenvalue from the total. The correct cumulative ratio through the first two components is \((8+2)/11=10/11\), approximately 0.9091.

Before finalizing a calculation, check the following:

Use all eigenvalues from the PCA in the denominator, including those not retained.
Confirm that the sum of all component ratios is 1 before rounding.
Confirm that each cumulative ratio is nondecreasing and that the final cumulative ratio is 1.
State whether the PCA used a covariance matrix or a correlation matrix when interpreting what variables' variance is being summarized.
Do not translate a captured-variance percentage directly into predictive performance for an outcome.

Key takeaway: A component's explained-variance ratio is its eigenvalue divided by the sum of all eigenvalues. Add the first \(k\) eigenvalues, but keep the full eigenvalue sum as the denominator, to obtain the cumulative variance captured by the first \(k\) components. These ratios describe input-variable variation, not predictive success.

Check Your Understanding

For each problem, use the full eigenvalue list as the denominator. Ratios may be rounded to four decimal places and percentages to two decimal places.

A covariance-matrix PCA has eigenvalues 7, 2, and 1. Calculate the first component's ratio, the second component's ratio, and the cumulative ratio through component 2. (Answer: \(0.7000\), \(0.2000\), and \(0.9000\), respectively; the cumulative percentage is 90.00%.)
A PCA has eigenvalues 4.5, 3, and 1.5. Find the total variance and the percentage represented by each component. (Answer: total \(9\); component percentages 50.00%, 33.33%, and 16.67%.)
A correlation-matrix PCA of four nonconstant variables has eigenvalues 1.8, 1.2, 0.7, and 0.3. Calculate the cumulative ratio through component 3 and the percentage left in component 4. (Answer: \(0.9250\), or 92.50%, captured through component 3; 7.50% left.)
A covariance-matrix PCA has eigenvalues 12, 6, and 2. A candidate divides 12 by \(12+6\) to report the first component's explained-variance ratio. Identify the correct first-component ratio and the cumulative ratio through component 2. (Answer: correct first ratio \(12/20=0.6000\); cumulative ratio \(18/20=0.9000\).)
Two orthogonal component scores have variances 16 and 4, and a centered outcome satisfies \(Y=Z_2\). Calculate the component ratios and identify which component predicts the outcome exactly. (Answer: ratios \(0.8000\) and \(0.2000\); \(Z_2\), despite its smaller ratio, predicts \(Y\) exactly.)

What You'll Learn