What a PCA Biplot Shows
A PCA biplot places two kinds of objects in one display: observations, represented by points, and variables, represented by directions or vectors from the origin. The distinction matters. A point’s coordinates describe an observation’s scores on components; a vector’s coordinates describe how a variable is represented in those component directions. The two sets of coordinates are related, but they are not interchangeable.
A biplot is also a geometric approximation. A two-component display omits information in later components, and the exact meaning of distances, lengths, and angles depends on the scaling convention used to construct the coordinates. A careful interpretation therefore begins by identifying what was plotted, not by reading the picture as if all plotted distances had a universal meaning.
Using the singular value decomposition \(Z=U\Sigma V^{\mathsf T}\), the rank-two approximation is \(Z_2=U_2\Sigma_2V_2^{\mathsf T}\). A family of biplot factorizations is \(A=U_2\Sigma_2^\alpha\) for observations and \(B=V_2\Sigma_2^{1-\alpha}\) for variables, where \(0\leq\alpha\leq1\). The row coordinates of \(A\) are points; the row coordinates of \(B\) are variable vectors. Their product gives \(AB^{\mathsf T}=Z_2\). The choice of \(\alpha\) changes the plotted geometry, while preserving this rank-two reconstruction.
Interpreting Points, Vectors, and Their Products
Observation points show relative positions in the retained component coordinates. Points close together have similar displayed scores on those components; this does not establish that they are similar on every original variable, because omitted components may distinguish them. A variable vector starts at the origin and points toward increasing values of that variable in the displayed representation. Its direction is meaningful relative to the component axes and the other variable vectors.
For a particular observation and variable, the dot product of their plotted coordinates is the corresponding entry of \(Z_2\), the rank-two reconstruction, when the coordinates come from a matching biplot factorization. It is not automatically a correlation, a fitted response, or the original unscaled value. Those interpretations require the relevant centering, scaling, and biplot convention to be specified.
Worked Example: Reading an Observation–Variable Pair
In a hypothetical analysis of branch-level service measures, a two-component biplot uses a factorization in which the displayed point and vector coordinates reconstruct the centered-and-scaled data matrix. One branch has point coordinates \((1.2,-0.4)\). The vector for a service measure has coordinates \((0.8,0.6)\), and a second measure has coordinates \((0.5,-0.2)\). Find each rank-two reconstructed value for this branch and explain what the signs indicate.
Method. For each measure, take the dot product of the branch point with that measure’s vector.
For the first measure: \(1.2(0.8)+(-0.4)(0.6)=0.96-0.24=0.72\).
For the second measure: \(1.2(0.5)+(-0.4)(-0.2)=0.60+0.08=0.68\).
Thus, the two-component reconstructions are \(0.72\) and \(0.68\). Both are positive, so in this centered-and-scaled representation the branch is above the column mean on each measure as approximated by the retained components. The values are approximations, not necessarily the exact entries of the full data matrix. The common mistake is to call \(0.72\) a correlation: it is a row–variable dot product in the stated factorization, not a correlation coefficient.
Variable Vectors and Component Correlations
A useful interpretation of variable directions is available when the plotted variable coordinates are correlations between the variables and the principal component scores. The relationship can be derived directly. Suppose the columns of \(Z\) are centered and have sample variance 1, and PCA is performed on the sample covariance matrix \(Z^{\mathsf T}Z/(n-1)\). Then the score vector for component \(k\) is \(T_{\cdot k}=ZV_{\cdot k}\), and its sample variance is \(\lambda_k\).
The covariance of standardized variable \(j\) and score \(k\) is \(Z_{\cdot j}^{\mathsf T}T_{\cdot k}/(n-1)\). Since \(Z^{\mathsf T}ZV_{\cdot k}=(n-1)\lambda_kV_{\cdot k}\), the covariance is \(\lambda_kv_{jk}\). Dividing by the standard deviations \(1\) and \(\sqrt{\lambda_k}\) gives the correlation formula below.
The condition \(\lambda_k>0\) is essential: if \(\lambda_k=0\), the score has zero variance and its correlation with a variable is undefined. The expression on the right must not be treated as a defined correlation in that case. When the plotted variable coordinates are these correlations, the vector for variable \(j\), restricted to the first two components, is \((\sqrt{\lambda_1}v_{j1},\sqrt{\lambda_2}v_{j2})\). These are often called correlation coordinates.
If all components are included, the dot product between two variables’ correlation-coordinate vectors equals their correlation, provided both variables are standardized and all relevant components have positive variance. With only two components, the dot product is a partial sum: it approximates the correlation using the retained component directions. Consequently, an angle between two vectors can suggest a relationship, but it need not reproduce the original-variable correlation exactly.
Worked Example: Calculating an Angle from Correlation Coordinates
A hypothetical two-component display gives two standardized variables correlation-coordinate vectors \(a\) and \(b\). Their coordinate sums are reported as \(a^{\mathsf T}a=0.9441\), \(b^{\mathsf T}b=0.3204\), and \(a^{\mathsf T}b=0.4158\). Calculate the cosine of the angle between the plotted vectors, then state what the result does and does not show.
Method. Use the dot-product definition of the cosine. The vector lengths are \(\sqrt{a^{\mathsf T}a}\) and \(\sqrt{b^{\mathsf T}b}\), so their product is the denominator.
First calculate the product under the square root: \(0.9441 \times 0.3204 = 0.30248964\). Its square root is \(\sqrt{0.30248964}\approx0.5499906\).
Therefore, \(\cos(\theta)=0.4158/0.5499906\approx0.7560\), rounded to four decimal places. The corresponding angle is about \(40.9^\circ\). The vectors point in broadly similar directions in the two-component display. Because these are correlation coordinates, their dot product \(0.4158\) is the retained two-component contribution to the variables’ correlation, not necessarily their full correlation. The common mistake is to multiply the squared lengths incorrectly or to treat a two-component angle as an exact summary of the full data relationship.
How Scaling Changes the Picture
In the factorization \(A=U_2\Sigma_2^\alpha\), \(B=V_2\Sigma_2^{1-\alpha}\), choosing \(\alpha=1\) gives observation coordinates \(U_2\Sigma_2\), which are the component scores, and variable coordinates \(V_2\). This is called principal scaling. Choosing \(\alpha=0\) gives observation coordinates \(U_2\) and variable coordinates \(V_2\Sigma_2\), called standard scaling. A symmetric choice, commonly \(\alpha=1/2\), divides the singular-value scaling between points and vectors.
| Convention | Observation coordinates | Variable coordinates | Emphasis |
|---|---|---|---|
| Principal scaling, \(\alpha=1\) | \(U_2\Sigma_2\) | \(V_2\) | Observation score geometry |
| Standard scaling, \(\alpha=0\) | \(U_2\) | \(V_2\Sigma_2\) | Variable representation in the factorization |
| Symmetric scaling, \(\alpha=1/2\) | \(U_2\Sigma_2^{1/2}\) | \(V_2\Sigma_2^{1/2}\) | A compromise between the two |
The dot product reconstruction remains the same because each component’s powers multiply to \(\Sigma_k\). However, the lengths and angles among points or among variable vectors can change when the singular values are allocated differently. Thus, distances between points are most directly interpreted under a convention that preserves the score geometry, while variable-vector comparisons depend on the variable scaling used. Even then, scaling choices and omitted components limit the conclusions.
Worked Example: Comparing Principal and Standard Scaling
For a two-component PCA, suppose the first two singular values are \(3\) and \(1\). One observation has row coordinates \(U_i=(0.6,0.8)\), and one variable has row coordinates \(V_j=(0.8,0.6)\). Compare the point and vector coordinates under principal scaling and standard scaling. Check that their dot product is preserved.
Principal scaling. With \(\alpha=1\), the point is \(U_i\Sigma=(0.6(3),0.8(1))=(1.8,0.8)\), while the variable vector is \(V_j=(0.8,0.6)\). Their dot product is \(1.8(0.8)+0.8(0.6)=1.44+0.48=1.92\). The point length is \(\sqrt{1.8^2+0.8^2}=\sqrt{3.88}\approx1.9698\), and the vector length is \(\sqrt{0.8^2+0.6^2}=1\).
Standard scaling. With \(\alpha=0\), the point is \(U_i=(0.6,0.8)\), while the vector is \(V_j\Sigma=(0.8(3),0.6(1))=(2.4,0.6)\). Their dot product is \(0.6(2.4)+0.8(0.6)=1.44+0.48=1.92\). The point length is \(1\), while the vector length is \(\sqrt{2.4^2+0.6^2}=\sqrt{6.12}\approx2.4739\).
The reconstructed entry is \(1.92\) under both conventions, but the plotted lengths differ substantially. It would be a mistake to compare point-to-point distances from one convention with distances from the other, or to assume that a longer vector in one plot automatically represents a stronger original-variable association.
Practical Checks and Common Traps
Before interpreting a biplot, establish whether the input columns were centered or standardized and identify the actual point and vector coordinates. In R, for example, a PCA fit may provide scores and eigenvectors, but a plotted biplot can apply additional scaling. The name of a plotting function alone is not enough to determine whether plotted coordinates equal raw scores, eigenvectors, correlation coordinates, or rescaled versions.
Check that both axes use the same physical plotting scale before making angle comparisons. If the horizontal and vertical axes are stretched differently, the visual angle can differ from the angle calculated from the numerical coordinates. Also remember that reversing a component’s sign reverses both its point and variable coordinates on that axis; the underlying PCA representation is unchanged, but the display is reflected.
Check Your Understanding
For each question, identify the relevant coordinate convention or calculation before interpreting the plot.
- A standardized variable has eigenvector coefficients \(v_{j1}=0.6\) and \(v_{j2}=0.8\), and the corresponding eigenvalues are \(0.64\) and \(0.36\). Find its first two correlation coordinates. (Answer: \((0.48, 0.48)\))
- Two variable vectors have coordinates \((0.8,0.6)\) and \((0.4,-0.3)\). Calculate their dot product and the cosine of the angle between them. (Answer: Dot product \(0.14\); cosine \(0.28\))
- In a biplot factorization, an observation has \(U_i=(0.5,0.5)\), a variable has \(V_j=(0.4,0.2)\), and the singular values are \(2\) and \(1\). Under principal scaling, calculate the point–vector dot product. (Answer: \(0.5(2)(0.4)+0.5(1)(0.2)=0.50\))
- A component has eigenvalue zero. Can the correlation between a standardized variable and that component’s score be interpreted as zero using the correlation-coordinate formula? Explain. (Answer: No. The score has zero variance, so the correlation is undefined.)
- A plot uses standard scaling, but an analyst compares its observation-point distances with distances from a principal-scaling plot and concludes that the observations changed. Is this conclusion justified? (Answer: No. The scaling convention changes point coordinates and distances; compare like scaling with the same data and axes.)