Tutorials › Exam PA - Predictive Analytics › Reading prcomp Output Correctly

Home  /  Tutorials  /  Exam PA / Principal Components Analysis

Exam PA · Principal Components Analysis · Tutorial 625 of 966

Reading prcomp Output Correctly

Interpret the main components of a prcomp object and verify what they tell you about principal component directions, scores, and variance.

Advanced 13 min read

What You'll Learn

  • Identify what sdev, rotation, and x contain in a prcomp object.
  • Check expected component dimensions against the number of observations and variables.
  • Relate singular values, component score variances, and reported standard deviations.
  • Distinguish a full-component fit from a rank-limited fit.
  • Verify PCA output using matrix calculations in R.

What the Main prcomp Components Represent

A call to R's prcomp function returns more than a set of principal component directions. Its output records the directions, the transformed observations, and a scale for the variation along each direction. Reading those pieces together lets you check whether an interpretation is consistent with the analysis that was run.

Suppose the input has \(n\) observations and \(p\) numeric variables. Let \(A\) denote the matrix after the centering and, if requested, scaling specified in the call to prcomp. Its rows correspond to observations and its columns to variables. The earlier material on centering and scaling explains how those choices form \(A\); here the focus is on interpreting the returned components.

Definition: In a prcomp object, rotation contains the principal component directions as columns, x contains the observation scores on those directions, and sdev contains the standard deviation of the scores for each reported component. If \(k\) components are returned, their usual dimensions are \(p \times k\) for rotation and \(n \times k\) for x. The vector sdev has length \(k\). If the call uses retx = FALSE, x is not returned.

For a given component, a column of rotation has one coefficient for each input variable. Multiplying an observation's centered and scaled variable values by those coefficients gives its score on that component. Therefore, rotation is not a table of observation scores, and x is not a table of variable coefficients. Confusing these two components reverses the roles of variables and observations.

Why sdev Is a Singular Value Divided by a Scale Factor

The relationship among the three components follows from the singular value decomposition of \(A\). For a full-component fit, write the SVD as \(A=UDV^\mathsf{T}\), with singular values \(d_j\) on the diagonal of \(D\). The columns of \(V\) are orthonormal directions. Since the scores are \(AV\), the full score matrix is \(UD\).

$$ \text{rotation}=V,\qquad x=AV=UD,\qquad \text{sdev}_j=\frac{d_j}{\sqrt{n-1}}. $$

The last identity is not an arbitrary software convention. The \(j\)th score column is \(d_j\) times the \(j\)th column of \(U\). SVD columns of \(U\) have squared entries summing to 1, and the score column has mean zero when \(A\) is centered. Its sample variance is consequently its sum of squared entries divided by \(n-1\), or \(d_j^2/(n-1)\). Taking the square root gives the reported sdev.

$$ \operatorname{Var}_{\mathrm{sample}}(x_{\cdot j}) =\frac{\sum_{i=1}^n x_{ij}^2}{n-1} =\frac{d_j^2}{n-1} =\text{sdev}_j^2. $$

Thus, the eigenvalue associated with a component is \(\text{sdev}_j^2\), using the sample covariance convention. The proportion of total variance represented by that component is \(\text{sdev}_j^2\) divided by the sum of the eigenvalues for all components in the analysis. This denominator matters: if only some components are reported, the sum of their sdev values squared is the variance represented by those reported components, not necessarily the total variance in the input.

Important distinction: The full-SVD identities use all the directions in \(V\). If a fit retains only \(k\) components, write \(V_k\), \(U_k\), and \(D_k\) for the corresponding retained parts. Then rotation \(=V_k\) and \(x=AV_k=U_kD_k\), but in general \(A\ne U_kD_kV_k^\mathsf{T}\). The latter product is a rank-\(k\) approximation, not necessarily the original matrix.

Worked Example: Checking a Full Two-Variable Fit

Worked Example: Checking a Full Two-Variable Fit

A hypothetical dataset has four observations and two variables. After centering, its matrix is \(A=\begin{pmatrix}-3&1\\-1&-1\\1&-1\\3&1\end{pmatrix}\). Assume prcomp is run without scaling and returns both components. Verify the directions, scores, sdev values, and variance proportions. The method is to calculate the sample covariance matrix and then check the corresponding output relationships.

1. Check centering and the covariance matrix. The first column sums to \(-3-1+1+3=0\), and the second sums to \(1-1-1+1=0\). The sums of squares are \(20\) and \(4\). The cross-product sum is \((-3)(1)+(-1)(-1)+(1)(-1)+(3)(1)=0\). Dividing these sums by \(n-1=3\) gives \(\operatorname{Cov}(A)=\begin{pmatrix}20/3&0\\0&4/3\end{pmatrix}\).

2. Identify the directions. The covariance matrix is diagonal, with eigenvalues \(20/3\) and \(4/3\), in decreasing order. Its directions are the first and second coordinate axes, so rotation can be the identity matrix. Each direction has length 1, as required for an orthonormal set.

3. Calculate scores and sdev. Multiplication by the identity leaves the centered data unchanged, so \(x=AI=A\). The score sample variances are \(20/3\) and \(4/3\). Therefore sdev is \(\left(\sqrt{20/3},\sqrt{4/3}\right)\), approximately \((2.582,1.155)\), to three decimal places. The singular values are \(\sqrt{20}\) and \(2\); dividing them by \(\sqrt{3}\) gives the same sdev values.

4. Check the variance proportions. Total variance is \(20/3+4/3=8\). The first component's proportion is \((20/3)/8=5/6\), and the second's is \((4/3)/8=1/6\). They add to 1, as expected for this full two-component fit. The calculation avoids the common mistake of treating sdev itself, rather than sdev squared, as variance.

Reading Dimensions and Checking Output in R

Dimensions provide a quick diagnostic before interpreting numerical values. For a matrix with \(n\) rows and \(p\) columns, a full fit ordinarily returns up to \(\min(n,p)\) component directions. A centered matrix has rank at most \(\min(n-1,p)\), so one or more returned directions can have zero variance in particular data. When a fit is restricted to \(k\) components, rotation has \(p\) rows and \(k\) columns, x has \(n\) rows and \(k\) columns if it is retained, and sdev has length \(k\).

The argument rank. can request a limit on the number of components. The argument tol can also omit components whose standard deviations are small relative to the first component's standard deviation. As a result, check the object rather than assuming that every possible component is present. In R, dim(result$rotation), dim(result$x), and length(result$sdev) give the basic size checks; the x dimension check applies only when retx = TRUE.

The following calculations check score variances and the retained variance in an existing result, assuming x is present:

$$ \operatorname{apply}(x,\,2,\,\operatorname{var}) \ \approx\ \text{sdev}^2, \qquad \sum_j \text{sdev}_j^2 =\text{variance represented by the reported components}. $$

In the first check, apply calculates the sample variance of each score column, and sdev squared gives the corresponding target. Small numerical discrepancies can occur from floating-point arithmetic. In a full fit, the sum of the component variances equals the total sample variance of the centered or scaled variables analyzed. In a truncated fit, it generally equals only the retained variance.

Worked Example: What a Rank-Limited Fit Does Not Reconstruct

Worked Example: What a Rank-Limited Fit Does Not Reconstruct

Consider the centered matrix \(A=\begin{pmatrix}3&1\\-3&1\\0&-2\end{pmatrix}\), with three observations and two variables. Suppose prcomp retains only one component, so \(k=1\). Verify the returned dimensions and show why the retained component does not reproduce all of \(A\). The method is to calculate the covariance eigenstructure and then form the rank-one reconstruction.

1. Calculate the covariance matrix. Both column sums are zero. Their sums of squares are \(18\) and \(6\), and their cross-product sum is \(3(1)+(-3)(1)+0(-2)=0\). With \(n-1=2\), the covariance matrix is \(\begin{pmatrix}9&0\\0&3\end{pmatrix}\). Its leading eigenvalue is \(9\), with direction \((1,0)^\mathsf{T}\).

2. Find the retained output. The one-column rotation matrix is \(V_1=\begin{pmatrix}1\\0\end{pmatrix}\), so rotation has dimension \(2\times1\). The scores are \(x=AV_1=\begin{pmatrix}3\\-3\\0\end{pmatrix}\), with dimension \(3\times1\). Their sample variance is \((3^2+(-3)^2+0^2)/2=9\), so sdev is \(3\). This agrees with \(d_1/\sqrt{n-1}=\sqrt{18}/\sqrt2=3\).

3. Check what is discarded. The retained rank-one reconstruction is \(xV_1^\mathsf{T}=\begin{pmatrix}3&0\\-3&0\\0&0\end{pmatrix}\), which is not \(A\): the second column \((1,1,-2)^\mathsf{T}\) has been discarded. The retained variance is \(9\), while the full variance is \(9+3=12\), so the retained proportion is \(9/12=0.75\).

Common mistake and correction. It is incorrect to claim that \(A=U_1D_1V_1^\mathsf{T}\) for this one-component fit. That product is the displayed rank-one reconstruction and omits the second direction. The valid identities are rotation \(=V_1\) and \(x=AV_1=U_1D_1\); the original matrix is generally not equal to its truncated reconstruction.

Worked Example: Interpreting a Standardized Correlated Fit

Worked Example: Interpreting a Standardized Correlated Fit

A hypothetical three-observation dataset is \(M=\begin{pmatrix}1&1\\-1&0\\0&-1\end{pmatrix}\). Run prcomp with center = TRUE and scale. = TRUE, retaining both components. Verify that the data are already centered and have unit sample standard deviations, and interpret the output. The method is to calculate the sample correlation matrix, diagonalize it, and check the scores directly.

1. Verify the analyzed matrix. Each column sums to zero. Each has sum of squares \(2\), so its sample variance is \(2/(3-1)=1\) and its sample standard deviation is 1. The cross-product sum is \(1\), so the sample correlation is \(1/2\). Centering and scaling therefore leave this particular matrix unchanged.

2. Find the directions and variances. The correlation matrix is \(\begin{pmatrix}1&1/2\\1/2&1\end{pmatrix}\). Its unit eigenvectors are \(v_1=(1,1)^\mathsf{T}/\sqrt2\) and \(v_2=(1,-1)^\mathsf{T}/\sqrt2\), with eigenvalues \(3/2\) and \(1/2\). Thus the two columns of rotation are those vectors, and sdev is \((\sqrt{3/2},\sqrt{1/2})\), approximately \((1.225,0.707)\), to three decimals.

3. Check the score matrix. Multiplying each row by the directions gives \(x=\begin{pmatrix}\sqrt2&0\\-1/\sqrt2&-1/\sqrt2\\-1/\sqrt2&1/\sqrt2\end{pmatrix}\). The first score column has mean zero and sum of squares \(2+1/2+1/2=3\), hence sample variance \(3/2\). The second has mean zero and sum of squares \(0+1/2+1/2=1\), hence sample variance \(1/2\). Both agree with sdev squared.

4. Interpret the variance share. Standardizing two nonconstant variables to unit sample variance gives total variance \(2\). The component proportions are \((3/2)/2=0.75\) and \((1/2)/2=0.25\). This verifies the output without treating the loading coefficients as scores or mistaking standard deviations for eigenvalues.

Common Checks Before Reporting a Result

A concise review of a prcomp object should connect the requested analysis to the actual returned values. Check which variables were analyzed and whether they were centered and scaled; then inspect how many components were retained. For each reported component, rotation gives its variable direction, x gives the observation scores, and sdev squared gives its sample variance.

  • Check orientation. Rotation has variables in rows and components in columns; x has observations in rows and components in columns.
  • Check the variance scale. Compare var of each score column with the square of the corresponding sdev, not with sdev itself.
  • Check truncation. If fewer than all components are retained, do not equate the rank-limited reconstruction with the original analyzed matrix.
  • Check the denominator. A retained component's share of total variance uses the variance of all analyzed components as the denominator, not just the retained variance.
  • Check missing scores. If retx = FALSE, x is absent by design; its absence does not mean the component directions were not calculated.

These checks are useful when PCA is used to create features for a predictive model. A model using component scores depends on the observation coordinates in x, while an explanation of the directions depends on rotation. Keeping those interpretations separate prevents a correct PCA calculation from becoming an incorrect model description. This treatment is aligned with the SOA Exam PA (Predictive Analytics) syllabus.

Check Your Understanding

For each problem, identify the relevant prcomp component and check the dimensions or variance relationship before interpreting it.

  1. A full fit analyzes 8 observations on 3 centered variables and retains all three components. State the dimensions of rotation and x, and identify the number of sdev values. (Answer: rotation \(3\times3\), x \(8\times3\), and 3 sdev values.)
  2. A reported component has sdev \(=2.5\). What is its sample score variance, and what quantity does that value represent? (Answer: \(2.5^2=6.25\), the sample variance of that component's scores.)
  3. A centered matrix has covariance eigenvalues \(5,2,1\), but the fit retains only the first two components. What are their sdev values and what proportion of total variance do they jointly represent? (Answer: sdev values \(\sqrt5,\sqrt2\); retained proportion \(7/(5+2+1)=0.875\).)
  4. A fit uses 10 observations, 4 variables, and rank. = 2. State the expected dimensions of rotation and x when retx = TRUE. If the two reported sdev values are 3 and 1, find the retained variance and explain why it need not be the total variance. (Answer: rotation \(4\times2\), x \(10\times2\); retained variance \(3^2+1^2=10\), which excludes any variance in omitted components.)
  5. A full fit has 5 variables, 4 observations, and uses centered data. It returns 4 directions, with sdev values \(2,1,0,0\). What is the total variance represented, and what is the centered matrix's rank? (Answer: total variance \(2^2+1^2=5\); rank \(2\), since only two singular values are nonzero.)

Previous

Next

Back to Tutorials
Clinical Biostats