Why Compare Principal Component Scores?
PCA represents each observation by coordinates on new, mutually perpendicular axes. These coordinates are the principal component scores. Comparing score profiles can reveal which observations have similar patterns across the directions retained for analysis. For example, in a hypothetical portfolio study, two branches with similar scores on the first few components have similar positions in the reduced PCA representation.
The comparison must be interpreted carefully. A distance calculated from only selected components measures separation after projection onto those axes. It need not equal the observations’ distance in the full input space. And if the PCA was applied to standardized variables, the relevant full-space distance is in standardized units, not necessarily in the original measurement units.
A positive score places an observation in the positive direction of the chosen component axis relative to the origin of the input data. A negative score places it in the opposite direction. A score near zero means the observation is near the center along that axis. It does not necessarily mean the observation is average on every original variable.
Component signs are arbitrary: reversing an eigenvector reverses every score on that component. Therefore, statements such as “this observation has a high score on component 1” depend on the chosen sign. Distances between score profiles do not: reversing the sign for all observations on an axis leaves their pairwise score differences in absolute value unchanged.
Comparing Observations on Selected Components
Suppose two observations \(i\) and \(r\) are compared on the first \(k\) components. Their Euclidean distance in that selected score space is
This calculation gives equal weight to a one-unit difference on each included score axis. It is often useful to inspect the individual score differences as well as their combined distance: a short distance says the profiles are close overall on the selected axes, while the signed differences show how their positions contrast.
Worked Example: Finding the Closest Pair on Two Scores
A hypothetical analyst represents three branches by their scores on the first two components. Which pair is closest in this two-component representation, and how do the branch profiles differ?
| Branch | Component 1 score | Component 2 score |
|---|---|---|
| North | 2 | 1 |
| East | 1 | -1 |
| West | -2 | 0 |
Method. Calculate the Euclidean distance for each pair using both score coordinates.
For North and East, the score differences are \(2-1=1\) and \(1-(-1)=2\). Thus, \(d_{\text{North,East}}^{(2)}=\sqrt{1^2+2^2}=\sqrt{5}\approx 2.236\).
For North and West, the differences are \(2-(-2)=4\) and \(1-0=1\). Thus, \(d_{\text{North,West}}^{(2)}=\sqrt{4^2+1^2}=\sqrt{17}\approx 4.123\).
For East and West, the differences are \(1-(-2)=3\) and \(-1-0=-1\). Thus, \(d_{\text{East,West}}^{(2)}=\sqrt{3^2+(-1)^2}=\sqrt{10}\approx 3.162\).
The smallest distance is \(\sqrt{5}\), so North and East are closest on these two scores. Their component 1 scores are both positive and differ by 1, while their component 2 scores are on opposite sides of zero and differ by 2. The distance is a summary of both differences, not a claim that the branches match on each component.
Check and trap. The other two distances, approximately 4.123 and 3.162, are larger than 2.236, confirming the closest pair. Comparing only component 1 would incorrectly ignore the component 2 contrast.
What a Score Distance Represents
For a PCA based on an orthonormal set of eigenvectors, using every component preserves Euclidean distance in the space supplied to PCA. Using only some components gives the distance after projecting onto their axes. The relationship follows directly from the geometry of an orthogonal change of coordinates.
With all \(p\) components, the score vector is \(t_i=V^\mathsf{T}z_i\), where \(V^\mathsf{T}V=I\).
The score difference is \(t_i-t_r=V^\mathsf{T}(z_i-z_r)\).
Its squared length is \((z_i-z_r)^\mathsf{T}VV^\mathsf{T}(z_i-z_r)=(z_i-z_r)^\mathsf{T}(z_i-z_r)\), since the full square matrix \(V\) is orthogonal and \(VV^\mathsf{T}=I\).
With \(k<p\), the squared distance is a partial sum of the full sum of squared score differences. The omitted squared differences are nonnegative, so removing them cannot increase the distance.
This argument explains both what is preserved and what is lost. The full score vector merely expresses the same centered input vector in a different orthonormal coordinate system. A truncated score vector describes only its projection onto the retained component axes; separation along omitted axes is not counted.
The equality concerns the PCA input vectors, not automatically the raw data in their original units. If variables were standardized before PCA, \(z_i\) contains centered, scale-adjusted values, and the distance is Euclidean distance in that standardized space. If PCA used a covariance matrix without standardization, the distance is in the centered original-variable units. The preprocessing must be known before giving a score distance a substantive interpretation.
Worked Example: A Projected Distance Is Not the Full Distance
Two hypothetical observations have the following score differences on three PCA components: component 1 difference \(3\), component 2 difference \(0\), and component 3 difference \(-2\). Find their distance on the first two components and their full distance. Assume the PCA used all three orthonormal directions for the input space being considered.
Method. Square and add the differences for the components specified. For the full distance, include all three components.
Using only components 1 and 2 gives \( (d^{(2)})^2=3^2+0^2=9\), so \(d^{(2)}=\sqrt{9}=3\).
Using all three gives \(d^2=3^2+0^2+(-2)^2=9+0+4=13\), so \(d=\sqrt{13}\approx3.606\).
Thus the first-two-component distance is 3, whereas the full distance in the PCA input space is approximately 3.606. The omitted component contributes \( (-2)^2=4\) to the squared distance.
Check and trap. The full squared distance is \(9+4=13\), which is at least the retained squared distance of 9. Treating 3 as the original-space distance would omit the component 3 difference; the correction is to include it when the question asks for all-component distance. Both distances are correctly stated only relative to the input scaling used for PCA.
Selected Components Can Change Which Pair Seems Closest
Dropping components can change the ordering of pairwise distances as well as their numerical values. A pair that is close on the retained components may differ substantially on a discarded component. Therefore, a nearest-pair conclusion based on a selected score space is specifically a conclusion about that space.
Worked Example: A Close Pair on Retained Scores
Three hypothetical observations have scores on three components. The analysis retains only components 1 and 2. Identify the closest pair using the retained scores, then compare the result with the distance using all three scores.
| Observation | Component 1 | Component 2 | Component 3 |
|---|---|---|---|
| A | 2 | 0 | 5 |
| B | 1 | 1 | -5 |
| C | -1 | 1 | -5 |
Method. First calculate each distance using components 1 and 2 only. Then include component 3 to obtain the full score-space distances.
For A and B, the retained score differences are \(2-1=1\) and \(0-1=-1\). Their retained distance is \(\sqrt{1^2+(-1)^2}=\sqrt{2}\approx1.414\).
For B and C, the retained differences are \(1-(-1)=2\) and \(1-1=0\). Their retained distance is \(\sqrt{2^2+0^2}=2\).
For A and C, the retained differences are \(2-(-1)=3\) and \(0-1=-1\). Their retained distance is \(\sqrt{3^2+(-1)^2}=\sqrt{10}\approx3.162\). Thus A and B are closest on the first two components.
Now include component 3. The A–B difference there is \(5-(-5)=10\), so their full distance is \(\sqrt{1^2+(-1)^2+10^2}=\sqrt{102}\approx10.100\). The B–C component 3 difference is \(-5-(-5)=0\), so their full distance remains \(\sqrt{2^2+0^2+0^2}=2\). The A–C component 3 difference is \(5-(-5)=10\), giving full distance \(\sqrt{3^2+(-1)^2+10^2}=\sqrt{110}\approx10.488\).
With all three scores, B and C are closest, not A and B. The first-two-component conclusion is still correct for the retained representation; it simply does not describe all three dimensions.
Check and trap. Each full squared distance equals its retained squared distance plus the square of the component 3 difference: \(2+100=102\), \(4+0=4\), and \(10+100=110\). Assuming the closest pair must remain closest after truncation would miss the large omitted difference between A and B.
Interpreting Score Profiles Carefully
A distance is useful for summarizing similarity, but it does not explain the direction of a contrast. To describe a pair, report the relevant score differences: for example, one observation may be higher on component 1 but lower on component 2. Component interpretations depend on the loading patterns, so a score alone does not identify which original variables produced the position. The component axes summarize combinations of variables, not the variables individually.
Also distinguish a score from a standardized score. In ordinary PCA, score differences are measured in the coordinates produced by the eigenvectors. Dividing component \(j\)'s scores by \(\sqrt{\lambda_j}\), where \(\lambda_j\) is its eigenvalue, creates a different scaled comparison. It gives components different relative weight in the distance and does not preserve the original Euclidean distance. Do not make this extra rescaling unless the problem explicitly defines it.
For an actuarial analyst, this distinction matters when screening observations, identifying groups for review, or communicating what a reduced representation shows. A statement such as “these two observations are close” should identify the components used and the preprocessing basis. It should not imply that the observations are close on every recorded measure unless that conclusion is justified in the full input space.
Common Mistakes
- Calling a truncated distance the original distance. It omits differences on discarded components. State that it is a distance in the selected score space.
- Ignoring preprocessing. Correlation-based PCA generally operates on standardized variables; covariance-based PCA uses centered variables in their original units. Score-distance meaning follows the PCA input.
- Interpreting the sign as an absolute property. All scores on an axis can be reversed by changing the eigenvector sign. Pairwise distances remain unchanged, but verbal descriptions of positive and negative positions must use the chosen orientation.
- Assuming a close score profile means agreement on every original variable. Components combine variables, and omitted directions may contain differences that are not visible in the selected scores.
- Rescaling scores without being asked. Dividing by component standard deviations changes the distance definition and its weighting.
Check Your Understanding
For each problem, use the stated component scores and preprocessing. Distances may be rounded to three decimals.
- Two observations have score differences \(2\) and \(-1\) on components 1 and 2. Find their distance on these two components. (Answer: \(\sqrt{5}\approx2.236\))
- Observations D and E have score differences \(1, 2, 2\) on three orthonormal components. Find the distance on the first two components and the full distance. (Answer: \(\sqrt{5}\approx2.236\) on the first two; \(3\) using all three)
- On the first two components, the distances for pairs R–S, R–T, and S–T are \(2\), \(\sqrt{13}\), and \(3\), respectively. Which pair is closest in this selected score space? (Answer: R–S)
- Two observations differ by \(2\) and \(1\) on retained components, and by \(4\) on an omitted component. Assuming a full orthonormal PCA basis, calculate the retained and full distances. (Answer: \(\sqrt{5}\approx2.236\) retained; \(\sqrt{21}\approx4.583\) full)
- PCA was performed after standardizing two variables. Two observations have full score differences \(3\) and \(4\). What is their Euclidean distance in the PCA input space, and what qualification should accompany it? (Answer: \(5\); it is the distance in standardized-variable space, not necessarily in the original measurement units)