Introduction
Principal component analysis (PCA) reduces a collection of variables to a smaller set of derived scores. The word “component” can make it sound as though PCA has selected or renamed some of the original variables. That is not generally what happens: each component is usually a weighted combination of several variables. Its score for an observation is calculated from those variables using the component’s loading vector.
This distinction matters when interpreting a PCA result. A component may account for a large share of the total variance and still combine variables in a way that is difficult to summarize. Explained variance describes how much variation a direction captures; it does not, by itself, tell us whether the direction corresponds to one original variable or has a clear practical meaning. This topic is aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.
The score \(C_j\) is a new variable calculated for each observation. It is not one of the input variables unless its loading vector has a particular form: \(a_j=e_k\) or \(a_j=-e_k\) for some \(k\), where \(e_k\) has a 1 in position \(k\) and zeros elsewhere. In those cases the score is exactly \(X_k\) or \(-X_k\). The sign does not change the information represented, because PCA eigenvectors can be multiplied by \(-1\) without changing their validity.
Why a Component Is a Weighted Combination
For a centered observation \(x=(x_1,\ldots,x_p)^\mathsf{T}\), the score is obtained by taking a weighted sum of its variable values. The loading vector specifies the weights, including their signs. A positive loading means that the variable contributes in the same direction as the score; a negative loading means it contributes in the opposite direction. A loading of zero means that variable does not contribute to that component.
Unit length, \(a_j^\mathsf{T}a_j=1\), is the usual eigenvector normalization. It makes the loading vector’s direction, rather than an arbitrary rescaling of its entries, determine the component. Under this normalization, the variance of the score is \(a_j^\mathsf{T}\Sigma a_j\). If \(a_j\) is an eigenvector with eigenvalue \(\lambda_j\), this variance equals \(\lambda_j\).
This also explains the role of explained variance. For a covariance-matrix PCA, the total variance is the trace of \(\Sigma\), equivalently the sum of its eigenvalues. The proportion attributed to component \(j\) is \(\lambda_j/\sum_{k=1}^p\lambda_k\). This proportion measures variance captured by the component, not how many original variables it uses or how easy its loading pattern is to interpret.
Worked Calculations
Worked Example: A High-Variance Sum of Two Variables
Problem. A hypothetical risk-monitoring team studies two centered measurements, \(X_1\) and \(X_2\), in the same units. Their covariance matrix is \(\Sigma=\begin{pmatrix}4&3\\3&4\end{pmatrix}\). Find the first principal component, its explained-variance ratio, and its score for an observation with centered values \((2,0)\). Decide whether it is an original variable.
Method. Find the eigenvalues and a unit eigenvector for the largest eigenvalue, then use the loading vector to calculate the score. This avoids treating the component label “PC1” as if it named one of the original measurements.
For a \(2\) by \(2\) matrix, the characteristic equation is \(\det(\Sigma-\lambda I)=0\). Thus \((4-\lambda)^2-9=0\), so \(4-\lambda=3\) or \(4-\lambda=-3\). The eigenvalues are \(\lambda_1=7\) and \(\lambda_2=1\).
For \(\lambda_1=7\), the equation \((\Sigma-7I)a_1=0\) gives \(-3a_{11}+3a_{21}=0\), so the two loadings are equal. Normalizing gives \(a_1=(1/\sqrt{2},1/\sqrt{2})^\mathsf{T}\). Therefore \(C_1=(X_1+X_2)/\sqrt{2}\). The score for the stated observation is \((2+0)/\sqrt{2}=\sqrt{2}\), approximately \(1.4142\).
The total variance is \(7+1=8\), so the first component’s explained-variance ratio is \(7/8=0.875\), or \(87.5\%\). Its loading vector has two nonzero entries, so it is a combination, not either original variable. Check: the score variance is \(7\), matching the first eigenvalue. A mistaken interpretation would call PC1 “\(X_1\)” because it captures most of the variance; the loadings show that it uses both measurements equally.
Worked Example: A Large Share With a Mixed Loading Pattern
Problem. A hypothetical insurer summarizes three centered operational measurements \(X_1,X_2,X_3\), recorded on comparable scales. Suppose their covariance matrix is \(\Sigma=I+8uu^\mathsf{T}\), where \(u=(1,2,-2)^\mathsf{T}/3\). Find the leading component and its explained-variance ratio. Then calculate its score for an observation \(x=(1,0,-1)^\mathsf{T}\) and describe the loading pattern.
Method. Use the given rank-one form to identify the eigenvalues and eigenvector. This makes clear both how much variance the component captures and which variables form its score, rather than inferring interpretation from the percentage alone.
First, \(u^\mathsf{T}u=(1+4+4)/9=1\), so \(u\) is a unit vector. Multiplication by \(\Sigma\) gives \(\Sigma u=u+8u(u^\mathsf{T}u)=u+8u=9u\). Thus \(u\) is an eigenvector with eigenvalue \(9\). For any vector \(w\) perpendicular to \(u\), \(u^\mathsf{T}w=0\), so \(\Sigma w=w\). The other two eigenvalues are therefore \(1\) and \(1\).
The total variance is \(9+1+1=11\). The leading component explains \(9/11=0.81818\ldots\), or \(81.82\%\) to two decimal places. Its score is \(C_1=(X_1+2X_2-2X_3)/3\). At \(x=(1,0,-1)^\mathsf{T}\), this is \((1+0+2)/3=1\).
The component uses all three measurements. \(X_1\) contributes with weight \(1/3\); \(X_2\) and \(X_3\) contribute with weights \(2/3\) and \(-2/3\). This is a contrast, not a selected original variable. The large explained-variance ratio does not remove the need to examine that contrast and decide whether it has a useful interpretation in context. Check: the eigenvalues sum to \(11\), and the leading ratio is less than \(1\), as required.
Worked Example: Recognizing a Component That Is an Original Variable Up to Sign
Problem. A hypothetical analysis of two centered measurements has covariance matrix \(\Sigma=\begin{pmatrix}1&0\\0&4\end{pmatrix}\). Identify the first component and its explained-variance ratio. A software output reports its loading vector as \((0,-1)^\mathsf{T}\). Is the component a combination of both original variables?
Method. Match the largest eigenvalue to its eigenvector and apply the sign-aware criterion for when a component equals an original variable up to sign.
The eigenvalues of this diagonal matrix are \(1\) and \(4\). The largest is \(4\), with eigenvector \((0,1)^\mathsf{T}\); the reported vector \((0,-1)^\mathsf{T}\) is the same eigenvector direction with its sign reversed. The corresponding score is \(C_1=-X_2\), not a combination involving \(X_1\).
The total variance is \(1+4=5\), so the explained-variance ratio is \(4/5=0.8\), or \(80\%\). The loading vector is \(-e_2\), so this component is the second original variable up to sign. Check: \(\operatorname{Var}(-X_2)=\operatorname{Var}(X_2)=4\), agreeing with the eigenvalue. Calling this a combination merely because its loading is \(-1\), rather than \(1\), would ignore PCA’s sign ambiguity.
Why PCA Chooses a Direction, Not a Variable
The variance calculation also gives a concise justification for why PCA is framed in terms of directions. For any unit vector \(a\), the variance of the score \(a^\mathsf{T}X\) is \(a^\mathsf{T}\Sigma a\). The first principal direction is the unit vector that maximizes this quantity. A Lagrange multiplier makes the maximizing condition explicit.
Because \(\Sigma\) is symmetric, differentiating with respect to \(a\) gives \(\nabla_a L=2\Sigma a-2\lambda a\). At a constrained stationary point this gradient is zero, so \(\Sigma a=\lambda a\). Thus a maximizing direction is an eigenvector, and its variance is its associated eigenvalue. The largest eigenvalue supplies the greatest variance among unit directions. Nothing in this optimization requires the maximizing vector to have just one nonzero entry; it may assign weights to many variables.
Interpreting Explained Variance Carefully
A high explained-variance ratio is useful evidence that a component summarizes a substantial amount of variation in the analysis variables. It is not a complete interpretation. A component’s loadings show which variables contribute, in what directions, and with what relative weights. For instance, a score with positive loadings for some measurements and negative loadings for others may represent a contrast between groups of measurements. Whether that contrast is meaningful depends on what the measurements represent and how they were prepared for PCA.
The scale used for PCA also affects the loadings and the variance proportions. Covariance-matrix PCA is sensitive to the units and variances of the variables. PCA on standardized variables uses a correlation matrix and puts the variables on a common scale. Neither choice turns a component into an original variable automatically. Interpretation must match the actual variables and scaling used.
A component may be a useful summary for later analysis even if its name is not self-evident. In that case, describe it through its loading pattern rather than assigning a confident label unsupported by the weights. Also distinguish the component’s loading vector, which defines the score, from the score itself, which is the value computed for a particular observation.
Common Mistakes
- Calling PC1 the “most important original variable.” PC1 is a direction with the largest variance among unit directions. It may combine many variables.
- Assuming a large explained-variance percentage guarantees easy interpretation. The percentage measures captured variance; inspect the loadings to understand the combination.
- Ignoring negative loadings. A negative weight means the corresponding variable contributes in the opposite direction in the score. It is not evidence that the variable was omitted.
- Treating a loading of \(-1\) on one variable as a mixture. A vector equal to \(e_k\) or \(-e_k\), with zeros elsewhere, represents that original variable up to sign.
- Giving the component a meaning without reference to the variables. Explain the actual weighted pattern before proposing a short label.
Check Your Understanding
For each problem, distinguish the component’s variance contribution from what its loadings say about its interpretation.
- A two-variable covariance matrix is \(\begin{pmatrix}5&0\\0&2\end{pmatrix}\). What is the leading component’s loading direction and explained-variance ratio? Is it an original variable? (Answer: Loading direction \(e_1=(1,0)^\mathsf{T}\) up to sign; ratio \(5/7\approx0.7143\); yes, it is \(X_1\) up to sign.)
- A three-variable component has loading vector \((0,-1,0)^\mathsf{T}\). Does it combine multiple original variables, and which variable does it represent? (Answer: No; it is \(-X_2\), or \(X_2\) up to sign.)
- A unit loading vector is \((1/\sqrt{2},-1/\sqrt{2})^\mathsf{T}\). Write its score and explain why it is not one original variable. (Answer: \(C=(X_1-X_2)/\sqrt{2}\); both variables have nonzero loadings, so it is a contrast rather than either variable alone.)
- A PCA has eigenvalues \(12,3,1\). Calculate the first component’s explained-variance ratio and state what cannot be concluded from that ratio alone. (Answer: \(12/(12+3+1)=12/16=0.75\); it explains \(75\%\), but this alone does not show that its loading pattern is simple or that it equals an original variable.)
- A component has loading vector \((2/3,2/3,1/3)^\mathsf{T}\). For a centered observation \((3,0,3)^\mathsf{T}\), calculate the score and decide whether the component is an original variable up to sign. (Answer: Score \(=(2/3)(3)+(2/3)(0)+(1/3)(3)=2+0+1=3\); no, it has three nonzero loadings, so it is a combination.)