Why Principal Component Signs Can Flip

Why a Principal Component Can Have Either Sign

The entries of a principal component are often interpreted as describing how variables contribute to a direction. Their signs can therefore look important. Yet the direction itself does not distinguish between moving along a line one way and moving along the same line the opposite way. If an eigenvector is replaced by its negative, it points along the same axis. The numerical scores reverse sign, but the underlying projection does not move to a different axis.

This matters when comparing PCA output from different software runs, data-processing choices, or implementations. Two sets of results may report opposite signs for a component and its scores while describing precisely the same PCA direction. The reliable comparison is based on the component axis and quantities invariant to sign, rather than on an isolated positive or negative entry.

Definition: Let \(v\) be a unit eigenvector of a covariance or correlation matrix, with eigenvalue \(\lambda\). The vector \(-v\) is also a unit eigenvector with the same eigenvalue. The two vectors are different signed representations of the same one-dimensional principal axis.

Throughout, let \(z\) be a centered observation, or a centered and scaled observation when PCA is performed on standardized variables. A component score is the projection \(v^\mathsf{T}z\). A one-component reconstruction in the centered coordinate system is that score multiplied by the component vector. Adding the mean back, and undoing any scaling when relevant, returns to the original data units.

The Algebra of a Sign Flip

The central relationships follow directly from replacing \(v\) by \(-v\). The score changes sign because the projection is linear in the component vector. The reconstructed contribution does not change because both the score and component vector change sign.

s=v^\mathsf{T}z,\qquad s_{\text{flip}}=(-v)^\mathsf{T}z=-s

\widehat{z}_1=sv,\qquad \widehat{z}_{1,\text{flip}}=s_{\text{flip}}(-v)=(-s)(-v)=sv=\widehat{z}_1

The principal axis is also unchanged as a geometric line: every point of the form \(av\), for a real number \(a\), is also of the form \(b(-v)\), by taking \(b=-a\). Reversing the vector changes which direction is called positive, not the set of points on the axis.

Theorem: If \(v\) is a unit eigenvector of a symmetric PCA matrix \(C\), with eigenvalue \(\lambda\), then \(-v\) is also a unit eigenvector for \(\lambda\). For every centered observation \(z\), the scores from \(v\) and \(-v\) are opposites, while their one-component reconstructions are identical.

To verify the eigenvector statement, use \(Cv=\lambda v\). Then \(C(-v)=-Cv=-\lambda v=\lambda(-v)\). Also, \((-v)^\mathsf{T}(-v)=v^\mathsf{T}v=1\), so the negative vector remains a unit vector. The eigenvalue, which measures variance along the component, is unchanged.

Why the Projected Variance Is Unchanged

For a centered random vector \(Z\) with covariance matrix \(C\), the variance of the score in direction \(v\) is \(v^\mathsf{T}Cv\). When \(v\) is a unit eigenvector with eigenvalue \(\lambda\), this variance is \(\lambda\). Reversing the sign leaves the variance unchanged, as shown below.

\operatorname{Var}(v^\mathsf{T}Z) =v^\mathsf{T}Cv =v^\mathsf{T}(\lambda v) =\lambda v^\mathsf{T}v =\lambda

\operatorname{Var}((-v)^\mathsf{T}Z) =(-v)^\mathsf{T}C(-v) =v^\mathsf{T}Cv =\lambda

The same invariance can be seen from any finite collection of centered observations. If their scores are \(s_1,\ldots,s_n\), the flipped scores are \(-s_1,\ldots,-s_n\). Their mean is negated, but each deviation from that mean is also negated. Squaring the deviations therefore gives the same sample variance. The result does not depend on whether the scores are summarized with a population or sample variance convention.

Worked Example: Flip a Score and Check Its Reconstruction

Suppose the first principal component is the unit eigenvector \(v=(3/5,4/5)^\mathsf{T}\), and the centered observation is \(z=(2,1)^\mathsf{T}\). Find the score and one-component reconstruction, then repeat the calculations using \(-v\). The method is direct projection followed by reconstruction; it avoids treating the sign of a score as a change in the underlying point.

Original score. Take the dot product:

s=v^\mathsf{T}z =\frac{3}{5}(2)+\frac{4}{5}(1) =\frac{6}{5}+\frac{4}{5} =2

Original reconstruction. Multiply the score by the component vector:

\widehat{z}_1=sv =2\begin{pmatrix}3/5\\4/5\end{pmatrix} =\begin{pmatrix}6/5\\8/5\end{pmatrix}

Flipped score. The negative component is \(-v=(-3/5,-4/5)^\mathsf{T}\). Its score is:

s_{\text{flip}}=(-v)^\mathsf{T}z =-\frac{3}{5}(2)-\frac{4}{5}(1) =-\frac{6}{5}-\frac{4}{5} =-2

Flipped reconstruction. Multiply this score by \(-v\):

\widehat{z}_{1,\text{flip}} =(-2)\begin{pmatrix}-3/5\\-4/5\end{pmatrix} =\begin{pmatrix}6/5\\8/5\end{pmatrix}

The check is exact: both reconstructions are \((6/5,8/5)^\mathsf{T}\), while the scores are \(2\) and \(-2\). As a useful arithmetic check, \(-2\bigl(2(3/5)+1(4/5)\bigr)=-2(2)=-4\). That product is not the flipped score: the flipped score is \((-v)^\mathsf{T}z=-2\). Applying an extra factor of \(-2\) to the original dot product would calculate a different quantity.

Worked Example: Confirm the Projected Variance From Data

A hypothetical centered two-variable data set has four observations, written as row vectors: \((2,0),(-2,0),(0,1),(0,-1)\). Its sample covariance matrix uses divisor \(n-1=3\). Verify that the first component’s score variance is the same for either sign of the component. The method is to compute the covariance, scores, and sample variance explicitly; this checks the algebra against data.

Covariance matrix. The data are centered, and the sum of the outer products is:

\sum_{j=1}^{4}z_jz_j^\mathsf{T} = \begin{pmatrix}4&0\\0&0\end{pmatrix} +\begin{pmatrix}4&0\\0&0\end{pmatrix} +\begin{pmatrix}0&0\\0&1\end{pmatrix} +\begin{pmatrix}0&0\\0&1\end{pmatrix} = \begin{pmatrix}8&0\\0&2\end{pmatrix}

Dividing by \(3\) gives \(C=\begin{pmatrix}8/3&0\\0&2/3\end{pmatrix}\). Its leading eigenvector is \(v=(1,0)^\mathsf{T}\), with eigenvalue \(8/3\). The scores are \(2,-2,0,0\), whose mean is zero. Their sample variance is:

\frac{2^2+(-2)^2+0^2+0^2}{4-1} =\frac{8}{3}

Reverse the component. For \(-v=(-1,0)^\mathsf{T}\), the scores are \(-2,2,0,0\). Their mean is also zero, and their sample variance is \(((-2)^2+2^2+0^2+0^2)/3=8/3\). This equals the eigenvalue and the original score variance. The sign changes each score, not the variance.

Check the projection. For the first observation \((2,0)^\mathsf{T}\), the original score and one-component reconstruction are \(2\) and \(2(1,0)^\mathsf{T}=(2,0)^\mathsf{T}\). With \(-v\), they are \(-2\) and \((-2)(-1,0)^\mathsf{T}=(2,0)^\mathsf{T}\). Both calculations reconstruct the same point.

Sign Changes With More Than One Component

With several retained components, each component can be sign-flipped independently. Let \(V\) be a matrix whose columns are orthonormal component vectors, and let \(t=V^\mathsf{T}z\) be the vector of scores. A collection of sign changes can be represented by a diagonal matrix \(D\) whose diagonal entries are \(1\) or \(-1\). The new component matrix is \(VD\), and the new score vector is \(Dt\). Because \(D^\mathsf{T}D=I\), the reconstruction is unchanged:

$$ (VD)(Dt)=V D^2t=Vt $$

This applies whether all components are retained or only some are retained: the reconstruction from the retained components stays the same when their signs are reversed. It does not mean that any change to the component vectors is harmless. The invariance established here is for independent sign changes, not for replacing an axis by a different direction.

Worked Example: Flip One Component in a Two-Component Reconstruction

Consider the orthonormal vectors \(v_1=(3/5,4/5)^\mathsf{T}\) and \(v_2=(-4/5,3/5)^\mathsf{T}\), and the centered observation \(z=(2,1)^\mathsf{T}\). The vectors can be eigenvectors of the symmetric matrix \(C=\begin{pmatrix}5.8&2.4\\2.4&7.2\end{pmatrix}\): direct multiplication gives \(Cv_1=9v_1\) and \(Cv_2=4v_2\). Find both scores, reconstruct using both components, then flip only \(v_1\). This calculation checks that the sign of one coordinate does not alter the combined reconstruction.

Scores before the flip.

t_1=v_1^\mathsf{T}z =\frac{3}{5}(2)+\frac{4}{5}(1)=2,\qquad t_2=v_2^\mathsf{T}z =-\frac{4}{5}(2)+\frac{3}{5}(1) =-\frac{8}{5}+\frac{3}{5}=-1

Reconstruct using both components.

t_1v_1+t_2v_2 =2\begin{pmatrix}3/5\\4/5\end{pmatrix} +(-1)\begin{pmatrix}-4/5\\3/5\end{pmatrix} =\begin{pmatrix}6/5\\8/5\end{pmatrix} +\begin{pmatrix}4/5\\-3/5\end{pmatrix} =\begin{pmatrix}2\\1\end{pmatrix}

Flip only the first component. Its new score is \(-2\), and the second score remains \(-1\). The reconstruction is:

(-2)(-v_1)+(-1)v_2 =2v_1+(-1)v_2 =\begin{pmatrix}2\\1\end{pmatrix}

The two-component reconstruction is exactly the original centered observation in both calculations. The eigenvalues are also unchanged: \(C(-v_1)=9(-v_1)\), while the second component was not changed. This example’s arithmetic is consistent with the one-component rule: reversing a component and its score together preserves their product.

What Sign Reversal Does and Does Not Mean

A sign flip changes the reported score values and the signs of all entries in that component vector. It does not change the component’s eigenvalue, the variance it captures, the axis it spans, or any reconstruction using that component and its corresponding scores. A software routine may choose either sign according to its implementation details; a sign convention can make a display consistent, but it does not make one sign statistically more correct.

A sign flip should also be distinguished from other forms of nonuniqueness. If eigenvalues are tied, a whole set of orthonormal directions within the tied eigenspace may represent valid choices, not merely a set of independent sign reversals. That situation can change individual component coordinates even though the tied subspace is retained. For distinct eigenvalues, the eigenvector ambiguity for a given eigenvalue is its sign, assuming the eigenvector is normalized.

Key takeaway: Compare PCA results only after allowing for sign. If a component changes from \(v\) to \(-v\), its scores must change from \(s\) to \(-s\); multiplying each new score by its new component vector must give the same reconstruction as before.

A common exam trap is to see opposite signs in two reports and conclude that one result is wrong, or to reverse the component but leave its scores unchanged. The first conclusion ignores the arbitrary orientation of an axis. The second breaks the projection-reconstruction calculation: using \(-v\) with the original score \(s\) produces \(-sv\), not \(sv\). A careful check pairs each component vector with scores computed from that same signed vector.

Scores: reverse sign when the component vector reverses.
Eigenvalues and projected variance: do not change.
Reconstruction: does not change when the score and its component are both reversed.
Interpretation: describe relative patterns consistently, but do not treat the component’s overall sign as an intrinsic property.

These invariances help distinguish a harmless software sign convention from a substantive difference in PCA results. They are useful when checking calculations, comparing output, and explaining components without overclaiming that a particular sign orientation carries independent meaning. This treatment is aligned with the SOA Exam PA (Predictive Analytics) syllabus.

Check Your Understanding

For each problem, use the score as the dot product of the centered observation and the stated component. Check that any reconstruction uses the score paired with that same signed component.

A unit component is \(v=(5/13,12/13)^\mathsf{T}\), and \(z=(1,2)^\mathsf{T}\). Find the score and one-component reconstruction, then give them using \(-v\). (Answer: Original score \(29/13\), reconstruction \((145/169,348/169)^\mathsf{T}\); flipped score \(-29/13\), same reconstruction.)
A covariance matrix is \(C=\begin{pmatrix}2&0\\0&6\end{pmatrix}\), and \(z=(3,-2)^\mathsf{T}\). Use the leading component and its negative to find the scores, eigenvalue, and one-component reconstruction. (Answer: Leading eigenvalue \(6\); scores \(-2\) and \(2\); reconstruction \((0,-2)^\mathsf{T}\) in both cases.)
Use \(v_1=(3/5,4/5)^\mathsf{T}\), \(v_2=(-4/5,3/5)^\mathsf{T}\), and \(z=(1,3)^\mathsf{T}\). Find the two scores and full reconstruction. Then flip only \(v_2\) and give its new score. (Answer: Scores \(3,1\); reconstruction \((1,3)^\mathsf{T}\); after the flip the scores are \(3,-1\), and the reconstruction remains \((1,3)^\mathsf{T}\).)
A centered sample has \(n=3\) scores \(2,-1,-1\) on a component. Find their sample variance. What are the scores and sample variance after the component sign is reversed? (Answer: Sample variance \(3\); flipped scores \(-2,1,1\), sample variance \(3\).)
A component vector is changed from \(v\) to \(-v\), but a data analyst retains the old score \(s=4\) when reconstructing. If the original one-component reconstruction was \(4v\), what does the analyst obtain, and what score should be used with \(-v\) to preserve the reconstruction? (Answer: The analyst obtains \(-4v\), the negative of the original reconstruction; the correct paired score is \(-4\), giving \((-4)(-v)=4v\).)

What You'll Learn