What Loadings and Scores Describe
Principal component analysis (PCA) replaces a set of measured variables with new, mutually uncorrelated directions through the data. Its components can be useful summaries, but their meaning is not automatic: one must interpret the loadings, which describe the directions, separately from the scores, which locate samples along those directions. In genomic data, this distinction matters when deciding whether a component is associated with a pattern across genes or whether a particular sample has an unusual position on that pattern.
Suppose \(X\) is an \(n\)-by-\(p\) data matrix whose columns have been centered and, when appropriate, scaled. Let \(v_k\) be the unit-length eigenvector for the \(k\)th principal component, with eigenvalue \(\lambda_k\). The score of sample \(i\) on that component is the projection of its centered data vector onto \(v_k\). We use the standard PCA convention that the sample covariance matrix is calculated with denominator \(n-1\).
These terms are related but not interchangeable. The eigenvector coefficient describes the component’s direction in variable space. A score is a sample-specific coordinate in that direction. A large positive loading coefficient does not mean every sample has a large positive score, and a large score does not mean a particular variable defines the component.
From Eigenvectors to Variable–Component Associations
For centered, standardized variables, each column has sample variance 1. The sample variance of component \(k\)’s scores is \(\lambda_k\), and the sample covariance between variable \(j\) and those scores is \(\lambda_k v_{jk}\). Dividing that covariance by the product of their sample standard deviations, \(1\) and \(\sqrt{\lambda_k}\), gives the correlation loading.
The squared loading coefficients for a component sum to 1: \(\sum_{j=1}^{p}v_{jk}^2=1\). Thus \(v_{jk}^2\) is the variable’s share of the squared length of that component direction. Correlation loadings answer a related but different question: how strongly, and in which direction, is a standardized variable associated with the component scores? They also depend on the component’s eigenvalue.
The sign of an entire eigenvector is arbitrary. Replacing \(v_k\) by \(-v_k\) changes every loading coefficient and score on that component to its negative, but changes neither the fitted PCA directions nor the relationships among variables and samples. Interpret signs relative to the displayed orientation of a component, not as an absolute biological label.
Worked Example: A Sample’s Scores Are Not Its Original Measurements
Consider two standardized variables with correlation 0.6. Their covariance matrix is therefore
The first component has eigenvalue \(\lambda_1=1.6\) and eigenvector \(v_1=(0.7071,0.7071)^\mathsf{T}\). The second has eigenvalue \(\lambda_2=0.4\) and eigenvector \(v_2=(0.7071,-0.7071)^\mathsf{T}\). For a hypothetical sample whose centered standardized profile is \(x_i=(1,0)^\mathsf{T}\), its scores are
Both scores are positive, so the point lies in the upper-right quadrant of a plot of its two raw scores. That location does not mean the sample is above the centre on both variables. Its given profile has the first measurement 1 standard deviation above the centre and the second exactly at the centre. The score axes are directions formed from the variables; they are not the original variable axes.
The correlation loadings are \((0.8944,0.4472)\) for variable 1 and \((0.8944,-0.4472)\) for variable 2 across the first two components. For example, the first component loading for variable 1 is \(\sqrt{1.6}(0.7071)=0.8944\). The two variables both relate positively to PC1, while their PC2 correlations have opposite signs. Neither fact changes the sample’s stated original profile of \((1,0)\).
Reading a Correlation Biplot
A biplot displays sample positions and variable directions together, but the coordinates must be identified before interpreting it. One useful choice for standardized data is a correlation biplot. Plot variable \(j\) at \((r_{j1},r_{j2})\), where \(r_{jk}=\sqrt{\lambda_k}v_{jk}\) is its correlation loading on component \(k\). Draw the variable as an arrow from the origin. Plot sample \(i\) at \((z_{i1},z_{i2})\), where \(z_{ik}=t_{ik}/\sqrt{\lambda_k}\) is its score divided by the component-score standard deviation. These standardized scores make the axes comparable in units of component standard deviations.
In this display, an arrow pointing strongly right indicates positive association with PC1; one pointing left indicates negative association. Arrows pointing in similar directions indicate variables that tend to vary together in the displayed plane. Opposite directions suggest a negative relationship in that plane, while near-perpendicular directions suggest little association in that plane. These are approximations to full-data relationships when important variation lies outside the two plotted components.
A sample’s position can be read by projecting its point onto an arrow: a positive projection suggests a value above the centre for that variable in the pattern represented by the plotted components; a negative projection suggests a value below the centre. This is a geometric summary, not a claim that every individual measurement can be recovered exactly from the plot. Arrow length also matters: a short arrow means the variable is weakly represented by the two displayed components, so its direction and the sample’s projection on it are less informative.
Worked Example: Separating Variable Contribution From Sample Extremity
Suppose a fitted PCA on standardized expression measurements has a first-component eigenvector for three genes equal to \(v_1=(0.80,0.50,0.3317)^\mathsf{T}\). The squared coefficients are approximately
Gene 1 therefore accounts for 64% of the squared length of this component direction, the largest share among these three genes. This is a statement about the component’s variable weights, not about any sample’s expression level. Its correlation with PC1 would additionally depend on \(\sqrt{\lambda_1}\): the correlation loading is \(\sqrt{\lambda_1}(0.80)\).
Now suppose three hypothetical samples have PC1 scores \(3.00\), \(-2.00\), and \(0.20\), and the component eigenvalue is 2.10. The score standard deviation is \(\sqrt{2.10}=1.449\). The first sample is \(3.00/1.449=2.07\) component-score standard deviations above the centre on PC1; the second is \(2.00/1.449=1.38\) standard deviations below it. These positions tell us which samples are far along the component direction. They do not change the fact that gene 1 has the largest loading coefficient in the direction.
A component score combines measurements across genes. Thus the sample with score 3.00 is not necessarily the sample with the largest value for gene 1, and it is not evidence that gene 1 alone produced the score. To make a claim about one gene’s measured value, inspect that gene’s measurement for the sample; to describe the component’s variable pattern, inspect its loadings.
Worked Example: Interpreting a Two-Component Correlation Biplot
Consider a hypothetical biplot with the following correlation-loading coordinates for four genes. Sample coordinates are standardized scores \(z_{i1},z_{i2}\).
| Gene or sample | PC1 coordinate | PC2 coordinate |
|---|---|---|
| Gene A arrow | 0.80 | 0.10 |
| Gene B arrow | 0.75 | 0.20 |
| Gene C arrow | -0.10 | 0.85 |
| Sample S1 point | 1.50 | 0.20 |
| Sample S2 point | -0.20 | 1.40 |
Genes A and B point in similar directions, so they have similar associations with the two displayed components. Their coordinates are both strongly positive on PC1. Sample S1 lies to the right, in the direction of both arrows, so its projected two-component pattern is relatively high for A and B compared with the centre. This does not establish that both measured gene values are individually high; it describes their pattern in the displayed component plane.
Gene C points mainly upward, indicating a strong positive association with PC2 and little association with PC1 in this plot. Sample S2 lies mainly upward too, so its position is aligned with the displayed pattern for C. Its negative PC1 coordinate also indicates that it is on the opposite side from the positive PC1 direction of A and B. These statements concern component coordinates and projections, not causal effects or a biological mechanism.
The arrow lengths for A and B are \(\sqrt{0.80^2+0.10^2}=0.8062\) and \(\sqrt{0.75^2+0.20^2}=0.7762\), respectively. Neither reaches 1, so some of each gene’s relationship to the full set of components lies outside this two-component display. S1’s distance from the origin is \(\sqrt{1.50^2+0.20^2}=1.513\); that describes its distance in this plotted score plane, not the contribution of a particular gene.
Checks, Alternatives, and Common Pitfalls
Before interpreting a loading plot or biplot, check what data entered PCA. Centering and scaling affect the result: PCA on unscaled measurements can be dominated by variables with large numerical variance, while standardizing gives variables equal sample variance before analysis. The earlier material on centering and scaling before PCA establishes these choices; the interpretation of loadings is conditional on them. In genomic applications, also consider whether a technical or batch pattern is driving the component before assigning it biological meaning.
Check the axes and scaling printed on the figure. A plot may show raw scores and eigenvector coefficients, or use another biplot scaling; arrow length and sample distance then have different numerical meanings. Correlation biplots are convenient for interpreting standardized-variable associations, but are not the only valid display. A scores-only plot is often clearer for comparing sample positions, while a loading-only plot is clearer for comparing variable directions. A table of loadings can preserve precise values when a plot is crowded.
Common failures include treating the sign of a component as biologically fixed, reading a score as an original measurement, and interpreting a two-component picture as though it captured every variable relationship. A component can also have a large eigenvalue without any single variable having a dominant coefficient: the direction may combine many variables. Conversely, a large coefficient does not by itself mean the component explains a large fraction of total variation; that requires comparing its eigenvalue with total variance, as in the earlier material on explained variance.
- Wrong: “The sample is above the centre on both genes because its point is upper-right.” Correction: The point gives two component coordinates. Check each original variable value or use its projection on the variable arrow; a coordinate can be zero or below centre even when both component scores are positive.
- Wrong: “The sample farthest from the origin identifies the most important gene.” Correction: Distance describes a sample’s position in the plotted components. Variable importance for a component is assessed from its loading coefficients, with the chosen scaling made explicit.
- Wrong: “A positive loading means the variable is intrinsically positive.” Correction: Its sign is relative to the arbitrary orientation of the component; flipping the component flips all its scores and loadings together.
Biplots are descriptive summaries, not tests of association, proof of causation, or guarantees of biological interpretation. Their usefulness depends on the preprocessing, the variance captured by the displayed components, and whether the plotted coordinates are clearly defined. Distinguishing directions from sample positions is the central safeguard: loadings characterize components; scores locate samples.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- For standardized variables, what does \(v_{jk}\) describe, and how does it differ from \(t_{ik}\)?
- A sample has original standardized profile \((0,1)\). Can positive scores on two components establish that both original measurements are above their centres? Explain.
- A variable has loading coefficient \(v_{jk}=0.6\) on a component. What share of that component’s squared direction length is associated with this coefficient?
- In a correlation biplot, what does a short variable arrow indicate?
- Why can two analysts display opposite signs for the same principal component and still obtain equivalent PCA results?
Solutions. (1) \(v_{jk}\) is the coefficient of variable \(j\) in component \(k\)’s direction; \(t_{ik}\) is sample \(i\)’s projection onto that direction. (2) No. Scores are coordinates along combinations of variables, not the original measurements. The profile states that the first measurement is at its centre and the second is one standard deviation above it. (3) Its share is \(v_{jk}^2=0.6^2=0.36\), or 36%. (4) The variable is not well represented by the two displayed components; some of its relationships with the full component set lie outside the plot. (5) An eigenvector and its negative describe the same axis. Flipping the eigenvector flips all scores and loadings on that component, preserving the underlying PCA relationships.