What a Loading Can—and Cannot—Tell You
A fitted principal component can summarize a pattern of variation across several variables. Interpreting that pattern requires care: software and reports may use the word loading for different quantities, and neither quantity by itself explains why the variables vary together. A useful interpretation identifies the variables most aligned with the component, describes their relative directions, and avoids treating a descriptive association as a causal or predictive result.
Here, consider PCA of standardized variables. This setting makes the components depend on the correlation matrix and puts variables on a common scale. It also lets us distinguish two related but different measures: the coefficients used to form a component and the correlations between variables and that component.
In R, the columns of prcomp’s rotation matrix are eigenvector coefficients. A report may instead call the variable-component correlations “loadings.” Therefore, before interpreting a table, check its definition or how it was calculated. Treating the two quantities as interchangeable can materially change what a reported magnitude means.
From Coefficients to Correlation Loadings
The distinction follows directly from the eigenvector equation. Since \(R a_k=\lambda_k a_k\), the covariance of standardized variable \(Z_j\) with the component score is the \(j\)th entry of \(R a_k\), namely \(\lambda_k a_{jk}\). The variance of \(Z_j\) is 1, and the variance of \(T_k\) is \(\lambda_k\). Dividing the covariance by the product of the standard deviations gives the correlation loading:
This derivation also explains why a large coefficient does not always mean a strong variable-component correlation. The coefficient describes the variable’s weight in the component formula; the correlation loading additionally depends on the component’s eigenvalue. In particular, a component with a small eigenvalue can have a large-magnitude coefficient but a modest correlation loading.
Worked Interpretations
Worked Example: A Shared Pattern in Two Standardized Variables
Suppose a hypothetical PCA uses two standardized measurements of a policy-service process. Their correlation matrix is \(R=\begin{pmatrix}1&0.6\\0.6&1\end{pmatrix}\). For the first principal component, find the eigenvalue, coefficients, correlation loadings, and coefficient contributions. Describe the pattern.
Method. Find the eigenpair of the stated correlation matrix, then calculate correlation loadings using \(\sqrt{\lambda_k}a_{jk}\) and contributions using \(a_{jk}^2\). This avoids assuming that coefficients and correlations are the same.
The two eigenvalues are \(1+0.6=1.6\) and \(1-0.6=0.4\). The first component has the larger eigenvalue, \(1.6\), and its unit eigenvector is \(a_1=(1,1)^\mathsf{T}/\sqrt{2}\). The coefficients are therefore \(1/\sqrt{2}\approx0.7071\) for each variable.
For either variable, the correlation loading is \(\sqrt{1.6}(1/\sqrt{2})=\sqrt{0.8}\approx0.8944\). Each coefficient contribution is \((1/\sqrt{2})^2=0.5\), or 50%. The contributions total 100%.
Interpretation and check. Both variables have equally large, positive correlations with this component, so it describes a shared direction in which they tend to vary together. The eigenvalue 1.6 is larger than the other eigenvalue 0.4, as required for the first component. This pattern is descriptive; it does not say that a change in one measurement causes a change in the other.
Worked Example: A Contrast in a Feasible Three-Variable PCA
A hypothetical analyst standardizes three claim-process measures. Their fitted correlation matrix is \(R=\begin{pmatrix}1&-0.4&0.1\\-0.4&1&0.1\\0.1&0.1&1\end{pmatrix}\). For the leading component, find the coefficient vector, correlation loadings, and coefficient contributions. Explain the sign pattern.
Method. Use the symmetry between the first two variables to check the contrast direction, then verify it is the leading component. This avoids using a coefficient vector that is algebraically convenient but incompatible with the stated PCA.
Multiplying \(R\) by \((1,-1,0)^\mathsf{T}/\sqrt{2}\) gives \((1.4,-1.4,0)^\mathsf{T}/\sqrt{2}\). Thus the unit vector \(a_1=(1,-1,0)^\mathsf{T}/\sqrt{2}\) has eigenvalue 1.4.
To verify it is the leading component, the orthogonal direction \((1,1,0)^\mathsf{T}/\sqrt{2}\), together with the third-variable direction, has the matrix \(\begin{pmatrix}0.6&\sqrt{2}(0.1)\\\sqrt{2}(0.1)&1\end{pmatrix}\). Its eigenvalues are \((1.6+\sqrt{0.24})/2\approx1.0450\) and \((1.6-\sqrt{0.24})/2\approx0.5551\). Both are less than 1.4, so the stated contrast is the leading component.
The first and second correlation loadings are \(\sqrt{1.4}/\sqrt{2}=\sqrt{0.7}\approx0.8367\) and \(-\sqrt{0.7}\approx-0.8367\). The third is zero. Contributions are \(0.5\), \(0.5\), and \(0\), respectively: 50%, 50%, and 0%.
Interpretation and check. The component contrasts the first two measures: one aligns positively and the other negatively with the displayed direction. The third has no coefficient in this component. The equal and opposite first two coefficients are consistent with the matrix’s symmetry, and the contributions sum to 100%. This is a description of a direction in the observed variation, not evidence that the measures cause one another.
Worked Example: A Large Coefficient but a Modest Correlation Loading
Two hypothetical standardized measures have correlation 0.96. Consider their second principal component. Calculate its eigenvalue, coefficients, correlation loadings, and contributions. What distinction should an analyst make when describing the result?
Method. Calculate the eigenpair from the two-variable correlation matrix and then apply the correlation-loading formula. This avoids describing a large coefficient as if it necessarily implied a strong variable-component correlation.
The eigenvalues are \(1+0.96=1.96\) and \(1-0.96=0.04\). The second component is the contrast direction \(a_2=(1,-1)^\mathsf{T}/\sqrt{2}\), with eigenvalue 0.04. Its coefficients are approximately \(0.7071\) and \(-0.7071\).
The correlation loadings are \(\sqrt{0.04}(1/\sqrt{2})\approx0.1414\) and \(-0.1414\). The coefficient contributions are \(0.5\) and \(0.5\), or 50% each. The squared correlation loadings sum to \(0.1414^2+(-0.1414)^2=0.04\), matching the eigenvalue.
Interpretation and check. The coefficients have equal magnitudes and opposite signs, so the component formula is a contrast. Yet its variable-component correlations are modest because its eigenvalue is small. The eigenvalues sum to \(1.96+0.04=2\), the trace of this two-variable correlation matrix. Thus both the coefficients and the correlation loadings are consistent with the stated PCA.
Interpreting Magnitudes, Signs, and Contributions
A magnitude is meaningful only relative to the definition of the reported quantity and the other variables in that component. For eigenvector coefficients, larger absolute values mean greater weight in the score formula. For correlation loadings, larger absolute values mean a stronger linear association with the displayed component score. A coefficient of zero excludes a variable from that score formula; a correlation loading of zero means no linear correlation with that component.
Within a displayed component orientation, coefficients or correlation loadings with the same sign indicate alignment in the same direction, while opposite signs indicate opposing directions. A sign describes a relationship to the component direction, not an intrinsically positive or negative property of a variable. Avoid describing signs without naming the variables and the convention being used.
Coefficient contributions answer a narrower question: how the squared coefficients are distributed across variables in the component direction. They do not say how much of a variable’s total variation the component explains. For standardized-variable PCA, a correlation loading squared is the variable’s variance associated with that component; dividing by \(\lambda_k\) expresses that squared correlation as a share of the component’s total variance. Keep these interpretations separate.
| Quantity reported | What its magnitude describes | What it does not establish |
|---|---|---|
| Eigenvector coefficient \(a_{jk}\) | Variable \(j\)’s weight in the component score formula | The strength of its correlation with the component without considering \(\lambda_k\) |
| Correlation loading \(\sqrt{\lambda_k}a_{jk}\) | Variable \(j\)’s linear association with the component score | A causal effect or a relationship with an outcome not included in the PCA |
| Coefficient contribution \(a_{jk}^2\) | Variable \(j\)’s share of squared coefficients for that component | Predictive importance, causal importance, or the variable’s total variance explained |
Keep Descriptions Within the Evidence
PCA is an unsupervised method: it uses the input variables to summarize their variation, without using a target outcome to construct the components. A loading pattern can therefore support a statement such as, “these standardized measures align with the component in opposite directions.” By itself, it cannot support a statement that one measure causes another, that either measure changes an outcome, or that a variable is important to a predictive model.
Those stronger claims need evidence appropriate to the question. Causal interpretation requires a design and assumptions that address confounding and direction of effect. Predictive importance requires an analysis connected to a specified prediction target and an appropriate evaluation. A component that explains substantial input variation is not automatically useful for prediction.
A common exam trap is to read a reported coefficient of 0.8 as a correlation of 0.8. If it is an eigenvector coefficient and the component eigenvalue is 0.09, the correlation loading is \(\sqrt{0.09}(0.8)=0.3(0.8)=0.24\), not 0.8. Conversely, if 0.8 is explicitly a correlation loading, do not multiply it by \(\sqrt{\lambda_k}\) again. First identify the convention; then calculate and interpret.
A second trap is proposing an impossible eigenvector for a two-variable correlation matrix. If its variables have correlation \(r\), its eigenvectors for distinct eigenvalues are proportional to \((1,1)\) or \((1,-1)\); their two entries have equal magnitudes. For example, an asserted two-variable component with distinct eigenvalue 1.44 and coefficients 0.8 and 0.6 cannot be an eigenpair of a correlation matrix. The coefficient calculations might be arithmetically possible in isolation, but the PCA premise is not.
Check Your Understanding
For each problem, identify the quantity being reported before interpreting it. Give probabilities to four decimal places if any arise.
- A two-variable correlation matrix has correlation 0.5. For its first component, state the eigenvalue, eigenvector coefficients, correlation loadings, and coefficient contributions. (Answer: eigenvalue 1.5; coefficients \(1/\sqrt{2},1/\sqrt{2}\); correlation loadings \(\sqrt{0.75}\approx0.8660\) each; contributions 50% each.)
- A two-variable correlation matrix has correlation \(-0.3\). For its leading component, give the eigenvalue, coefficient signs, correlation loadings, and contributions. (Answer: eigenvalue 1.3; coefficients have opposite signs; loadings \(+\sqrt{0.65}\approx0.8062\) and \(-\sqrt{0.65}\approx-0.8062\); contributions 50% each.)
- Three standardized variables have pairwise correlations 0.2. For the first component, calculate the coefficient contribution of each variable and its correlation loading. (Answer: eigenvalue 1.4; equal coefficients \(1/\sqrt{3}\); contributions \(33.33\%\) each; correlation loadings \(\sqrt{1.4/3}\approx0.6831\) each.)
- A fitted component has eigenvalue 0.09, and a variable’s eigenvector coefficient is 0.8. Calculate its correlation loading and state what the coefficient and loading each describe. (Answer: correlation loading \(0.24\); the coefficient is its weight in the score formula, while the loading is its correlation with the component.)
- A report says a variable has a large positive correlation loading on a component that explains much of the input variation. What is a defensible conclusion, and name one conclusion that does not follow. (Answer: it has a strong positive linear association with the displayed component; this does not establish a causal effect or predictive importance for an outcome.)