Introduction
A PCA result can look plausible even when a calculation is wrong. A misplaced transpose may give scores in the wrong order, an uncentered data matrix may make the mean dominate the analysis, and an eigenvalue may be reported as a percentage without dividing by total variance. These mistakes matter because PCA is a sequence of linked calculations: an error in centering or scaling changes the matrix, which can change the components, scores, and interpretation.
A reliable check begins by stating what matrix is being analyzed and how it was constructed. For a data matrix with observations in rows and variables in columns, PCA is ordinarily applied to centered variables, or to standardized variables when using the correlation matrix. The eigenvectors of that chosen matrix determine the component directions. This tutorial is aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.
Start With the Matrix Being Analyzed
Centering removes each variable’s sample mean before calculating variation around that mean. Without centering, a cross-product matrix contains information about both the means and the variation around them. In fact,
The additional term is not generally zero. Thus, eigenvectors of \(X^{T}X\) need not describe directions of greatest variation around the sample means. A special data set may happen to give the same leading direction with or without centering, but that coincidence does not justify skipping centering.
Worked Example: Detecting an Uncentered Cross-Product
Three hypothetical service centers have two recorded measurements, \(x_1\) and \(x_2\): \((11,1)\), \((12,3)\), and \((13,5)\). Find the centered sample covariance matrix, compare it with the uncentered cross-product matrix divided by \(n-1\), and identify the leading covariance-PCA direction.
Method. Compute the column means, center each observation, and form \(X_c^TX_c/(n-1)\). Do not treat the raw cross-product as a covariance matrix.
The means are \(\bar{x}_1=(11+12+13)/3=12\) and \(\bar{x}_2=(1+3+5)/3=3\). The centered rows are \((-1,-2)\), \((0,0)\), and \((1,2)\). Therefore,
For the leading eigenvalue, the characteristic equation is \((1-\lambda)(4-\lambda)-4=0\), or \(\lambda^2-5\lambda=0\). The eigenvalues are \(5\) and \(0\). Solving \(S v=5v\) gives \(v_2=2v_1\), so a unit leading eigenvector is \(v=(1,2)^T/\sqrt{5}\). Its variance is \(v^TSv=5\).
The raw cross-products are \(\sum x_1^2=434\), \(\sum x_2^2=35\), and \(\sum x_1x_2=112\). Dividing by \(n-1=2\) gives \[ X^TX/(n-1)= \begin{pmatrix} 217&56\\ 56&17.5 \end{pmatrix}, \] not the centered covariance matrix. For a direct check, the mean contribution in the identity above is \((3/2)\begin{pmatrix}144&36\\36&9\end{pmatrix}\), which, added to \(S\), gives exactly this raw matrix.
As a score check, the first centered observation has score \((-1,-2)v=-5/\sqrt{5}=-\sqrt{5}\), and the third has score \(\sqrt{5}\). The middle observation has score \(0\). These scores are centered around zero, as expected. The common mistake is to interpret the raw cross-product as variation around the means; the correction is to center first and then divide by \(n-1\).
Do Not Mix Covariance and Correlation PCA
Covariance PCA uses the variables in their original measurement units. A variable with a larger numerical scale or variance can consequently have substantial influence. Correlation PCA standardizes each variable to have sample variance one before finding components. Neither choice is universally correct: the calculation must match the intended analysis. In particular, do not use eigenvectors from one matrix and eigenvalues or explained-variance calculations from the other.
A useful check is to calculate the matrix explicitly. If \(S\) is the covariance matrix and the standard deviations are \(s_1,\ldots,s_p\), then \(R_{jk}=S_{jk}/(s_js_k)\). For two variables, the diagonal entries of \(R\) must be one, and its off-diagonal entry is their correlation.
Worked Example: Finding a Covariance–Correlation Mismatch
Suppose two hypothetical measurements have sample covariance matrix \(S=\begin{pmatrix}9&3\\3&4\end{pmatrix}\). Find its correlation matrix and its leading covariance-PCA direction. Then check whether that direction is an eigenvector of the correlation matrix.
Method. First convert each covariance to a correlation using the standard deviations. Then calculate the leading eigenvector of \(S\). A direction obtained from \(S\) must not be assumed to be a principal direction of \(R\).
The standard deviations are \(s_1=\sqrt{9}=3\) and \(s_2=\sqrt{4}=2\). The correlation is \(3/(3\cdot2)=0.5\), so
The covariance eigenvalues solve \[ (9-\lambda)(4-\lambda)-9=0, \] so \(\lambda^2-13\lambda+27=0\), with leading eigenvalue \[ \lambda_1=\frac{13+\sqrt{61}}{2}\approx 10.405. \] An eigenvector satisfies \((9-\lambda_1)v_1+3v_2=0\), giving \(v_2/v_1=(\lambda_1-9)/3\approx0.4683\). Normalizing \((1,0.4683)^T\) gives approximately \(v=(0.906,0.424)^T\).
The correlation matrix has leading eigenvector \((1,1)^T/\sqrt{2}\) and leading eigenvalue \(1.5\). As a direct mismatch check, apply \(R\) to the covariance-PCA direction:
The second coordinate is \(0.5(0.906)+0.424=0.877\). The resulting vector is not a scalar multiple of \((0.906,0.424)^T\), so it is not an eigenvector of \(R\). The check confirms that the covariance-PCA direction cannot be reused as a correlation-PCA direction. The common mistake is to combine a covariance eigenvector with a correlation-matrix interpretation; the correction is to use one matrix consistently throughout.
Calculate Scores in the Correct Order
Let \(P\) be a \(p\times p\) matrix whose columns are orthonormal eigenvectors of the selected covariance or correlation matrix. For a centered observation written as a row vector \(z^T\), its component scores are the row vector \(z^TP\). If the observation is written as a column vector \(z\), the same calculation is \(P^Tz\). The orientation follows from the convention that eigenvectors are columns of \(P\).
A quick dimensional check helps: a row observation \(z^T\) has size \(1\times p\), and \(P\) has size \(p\times p\), so \(z^TP\) has one score for each component. Multiplying \(Pz\) is a different operation and generally gives different values.
Worked Example: Checking Score Multiplication and Variance Ratios
A three-variable PCA has orthonormal eigenvector matrix \[ P= \begin{pmatrix} 2/3&-2/3&1/3\\ 2/3&1/3&-2/3\\ 1/3&2/3&2/3 \end{pmatrix}, \] with eigenvalues \(6,2,1\), respectively. For the centered column observation \(z=(3,0,0)^T\), calculate its scores and the explained-variance ratio of each component. Check the scores by reconstruction.
Method. With eigenvectors stored in columns, use \(t=P^Tz\) for a column observation. Divide each eigenvalue by the sum of all eigenvalues to obtain its explained-variance ratio.
The scores are \[ t=P^Tz=(2,-2,1)^T. \] For example, the first score is \((2/3)(3)+(2/3)(0)+(1/3)(0)=2\). The second is \((-2/3)(3)+(1/3)(0)+(2/3)(0)=-2\), and the third is \((1/3)(3)+(-2/3)(0)+(2/3)(0)=1\).
If the multiplication is incorrectly written as \(Pz\), the result is the first column of \(P\) multiplied by 3, namely \((2,2,1)^T\). Its second entry has the wrong sign. To check the correct result, reconstruct: \(Pt=(2,-2,1)^T\) mapped back to the variable coordinates gives \[ P \begin{pmatrix}2\\-2\\1\end{pmatrix} = \begin{pmatrix} 4/3+4/3+1/3\\ 4/3-2/3-2/3\\ 2/3-4/3+2/3 \end{pmatrix} = \begin{pmatrix}3\\0\\0\end{pmatrix} =z. \]
The total variance is \(6+2+1=9\). The explained-variance ratios are \(6/9=2/3\), \(2/9\), and \(1/9\); they sum to one. The eigenvalues \(6,2,1\) are variances, not percentages. The common mistake is either multiplying in the wrong order or reporting eigenvalues themselves as ratios; the corrections are \(P^Tz\) and division by total variance.
Signs and Explained Variance: Two Final Checks
An eigenvector and its negative represent the same principal direction. If \(v\) is an eigenvector, then \(Sv=\lambda v\) implies \(S(-v)=\lambda(-v)\). The score for an observation changes sign when \(v\) is replaced by \(-v\), but the component’s variance and explained-variance ratio do not change. A negative score is therefore not evidence of a calculation error by itself. Check whether the reported eigenvector and its score use the same sign convention.
This derivation also explains why the matrix choice must remain consistent. Covariance eigenvalues sum to the total variance in the original units; correlation eigenvalues sum to \(p\), the number of standardized variables. Using the wrong denominator can produce ratios that do not sum to one. In numerical work, calculate the ratios from all eigenvalues of the matrix actually analyzed, then verify that their sum is one, apart from rounding.
A Compact PCA Error Audit
Confirm that rows are observations and columns are variables. Compute column means before forming covariance PCA.
State whether the analysis uses the covariance matrix or the correlation matrix. Confirm that standardized variables, if used, have unit sample variance.
For a proposed eigenvector \(v_j\) and eigenvalue \(\lambda_j\), verify \(Sv_j\approx\lambda_jv_j\), or \(Rv_j\approx\lambda_jv_j\) for correlation PCA.
Use the transpose required by the data orientation, and divide each eigenvalue by the total variance of that same matrix.
For a quick two-variable audit, an eigenvector check is often more informative than comparing rounded component labels. If the matrix-vector product is not approximately a scalar multiple of the proposed direction, either the matrix, eigenvector, or multiplication is wrong. When values are rounded, small discrepancies are expected; a large coordinate error is not.
These checks are useful because PCA errors can survive superficial review: scores may still look like reasonable numbers, and percentages may still be between zero and one. Confirming the centering, scaling, eigenvector equation, score orientation, and ratio denominator tests the actual chain of calculations rather than just the final presentation.
Check Your Understanding
For each problem, identify the relevant check before accepting the reported PCA result.
- A centered two-variable data set has sample covariance matrix \(\begin{pmatrix}4&0\\0&1\end{pmatrix}\). What are the covariance-PCA eigenvalues and explained-variance ratios? (Answer: Eigenvalues \(4,1\); ratios \(0.8,0.2\).)
- A covariance matrix is \(\begin{pmatrix}16&4\\4&1\end{pmatrix}\). Find the correlation between the variables and the correlation matrix. (Answer: Correlation \(4/(4\cdot1)=1\); \(R=\begin{pmatrix}1&1\\1&1\end{pmatrix}\).)
- Eigenvectors are columns of \(P\), and a centered column observation is \(z\). Which expression gives its scores, \(Pz\) or \(P^Tz\)? (Answer: \(P^Tz\).)
- A PCA matrix has eigenvalues \(5,3,2\). A candidate reports the first component’s explained-variance ratio as \(5\). What is the corrected ratio, and what check should be applied to all ratios? (Answer: \(5/(5+3+2)=0.5\); all ratios should sum to \(1\).)
- A unit eigenvector is reported as \(v=(0.6,0.8)^T\), and an observation’s score on it is \(4\). If the eigenvector is instead reported as \(-v\), what happens to the score and the component variance? (Answer: The score becomes \(-4\); the component variance is unchanged.)