Computing Principal Component Scores

From Centered Observations to Scores

Principal component analysis (PCA) represents each observation using coordinates along a set of component directions. Those coordinates are the principal component scores. They are useful because they describe where an observation lies relative to the sample center, measured along directions chosen by PCA.

To calculate scores, the essential sequence is simple: center each observation, then take its inner product with each component direction. The details matter. Multiplying the original, uncentered observation by a direction generally gives a different number, and a sign has meaning only relative to the orientation of the direction being used.

In this definition, each direction is a \(p\)-dimensional column vector. For ordinary PCA, the directions are unit length and mutually orthogonal. The product \(ZV\) is defined because \(Z\) has \(p\) columns and \(V\) has \(p\) rows. The resulting score matrix has \(n\) rows and \(k\) columns: one score per observation and selected component.

For a single observation, write its centered row as \(z_i=x_i-\bar{x}\). Its row of scores is \(t_i=z_iV\). Equivalently, the score on direction \(v_j\) is \(t_{ij}=z_i v_j\), where the row-vector and column-vector multiplication is an inner product. Use the same variables, units, and centering convention used to define the PCA directions.

What the Score Says About an Observation

A score is a signed coordinate along a component direction. If the direction \(v_j\) is fixed, a positive score means the centered observation has a positive component along \(v_j\); a negative score means it lies in the opposite direction. A score of zero means the centered observation has no component along that direction. It does not mean that the observation is at the sample center: the observation may have nonzero scores on other components.

The comparison is always with the sample center. A positive score does not mean that the original measurements are positive, nor does it necessarily mean that every variable is above its mean. The score combines all centered variables according to the entries of the direction. For example, a direction may assign positive weight to one variable and negative weight to another.

Matrix Calculation and a Justification

When there are several observations, compute the centered data matrix first and multiply once by the direction matrix. This keeps the row and column roles clear: a row of \(Z\) is an observation, while a column of \(V\) is a component direction.

The entry in row \(i\), column \(j\) of the product is \(z_i v_j\). In particular, the directions go in the columns of \(V\). If directions are instead written as rows, the corresponding calculation must use a transpose. Checking dimensions before multiplying is a quick way to catch this orientation error.

The inner-product formula follows from the meaning of a coordinate along a unit direction. Consider a centered row \(z\) and a unit direction \(v\). If \(t\) is the coordinate of \(z\) along \(v\), the part of \(z\) along the direction is \(tv\), and the remaining part \(r=z-tv\) must be perpendicular to \(v\). Perpendicularity means \(rv=0\), so

Since \(v\) is a unit vector, \(v^Tv=1\). Therefore \(t=zv\), the inner product used to calculate the score. For multiple mutually orthogonal unit directions, this argument applies separately to each direction, giving \(t_j=zv_j\) for every \(j\). Placing those results side by side gives the row calculation \(zV\).

Centering also gives a useful check on a complete score matrix. Each column of \(Z\) sums to zero because each variable was centered at its sample mean. Consequently,

Thus the sample mean of the scores on each component is zero, apart from possible rounding. This is a check on the calculation, not a statement that every individual score is zero.

Reading a Score Matrix

A score matrix preserves the observation order of the original data: row \(i\) contains the scores for observation \(i\). Column \(j\) contains the scores for component \(j\). The first column is not automatically “the first variable”; it is the coordinate on the first component direction.

The size of a score indicates the distance along that particular direction from the center, in the units implied by the measurements and direction. A score should be interpreted relative to other scores on the same component. Comparing numerical sizes across components can require care because the components may have different variances. For this calculation, the immediate interpretation is the signed coordinate; the score itself does not say which original variable caused that position.

Common Mistake: Skipping the Centering Step

A frequent error is to multiply the raw observation by a component direction. For example, in the first worked example, using raw \(x_1=(13,24)\) with \(v_1=(3/5,4/5)^T\) gives \(13(3/5)+24(4/5)=39/5+96/5=27\). That is not the score relative to the stated sample center. The correct calculation subtracts \((10,20)\) first: \((3,4)v_1=9/5+16/5=5\). The value 27 mixes the location of the origin with the observation's position, which is why PCA scores from centered data use deviations from the mean.

Another trap is to calculate only one inner product when the question asks for several components. Each direction produces its own score, and the direction matrix must contain every requested direction as a separate column. Finally, interpret a negative score as “opposite the stated direction,” not as “below the mean on every variable.”