Reconstructing Observations With Principal Components

From Component Scores Back to Observations

Principal component analysis (PCA) transforms an observation from its original variable coordinates into scores along component directions. Those scores can also be used in reverse: combine each score with its component direction to recover the centered observation. This is useful when interpreting PCA results, checking calculations, or forming an approximation that uses only some of the components.

The reconstruction depends on keeping the coordinate system straight. PCA scores describe deviations from the variable means if PCA was performed on centered data. Therefore, a reconstruction from those scores is initially centered; the means must be added back to express it on the original scale. If PCA was performed on standardized data, the reconstructed standardized coordinates must first be rescaled and then shifted by the original means.

A direction \(v_j\) has one coordinate for each variable. Multiplying it by its score gives that component’s contribution to the reconstructed observation. Adding the contributions across components combines the information in the retained scores. When all \(p\) component directions form an orthonormal basis, using all \(p\) scores recovers the centered vector exactly, and adding \(\bar{x}\) recovers the original observation.

Why the Reconstruction Formula Works

Let \(V\) be the matrix whose columns are the \(p\) orthonormal component directions, so \(V^\mathsf{T}V=I\). The score vector is \(t_i=V^\mathsf{T}z_i\). Since \(V\) is a square orthogonal matrix, \(VV^\mathsf{T}=I\) as well. Multiplying the score vector by the direction matrix therefore gives

Written as a sum of component contributions, \(Vt_i=\sum_{j=1}^{p}t_{ij}v_j\). This establishes the full reconstruction formula. If only the first \(k\) scores are used, the same sum stops at \(k\), so the result contains only the part of the centered observation represented in those directions. The omitted component contributions are not included in that approximation.

This argument assumes that the directions are orthonormal and that the scores and directions use the same ordering and sign convention. In PCA based on a data matrix with fewer independent directions than variables, some directions may not be needed to span the observed centered data; the full-basis identity applies to the complete orthonormal coordinate system, while reconstruction using available component directions recovers the part represented in their span.

Using Fewer Components

When only \(k\) components are retained, the reconstruction is an approximation:

The coordinate-wise difference between the original and reconstructed observation is \(z_i-\hat{z}_{i,k}\), equivalently \(x_i-\hat{x}_{i,k}\). These differences show how the approximation changes each variable for that particular observation. Because the omitted directions are orthogonal to the retained directions, the squared Euclidean length of this difference is the sum of the squares of the omitted scores, provided the component directions form an orthonormal basis:

To see why, expand the omitted part as \(\sum_{j=k+1}^{p}t_{ij}v_j\). Its squared length is the sum of all products \(t_{ij}t_{i\ell}v_j^\mathsf{T}v_\ell\). Orthonormality gives \(v_j^\mathsf{T}v_\ell=0\) for \(j\ne\ell\) and \(v_j^\mathsf{T}v_j=1\), leaving only \(\sum_{j=k+1}^{p}t_{ij}^2\). This is an observation-specific calculation; it does not by itself give a claim about average reconstruction error across a data set.

Reconstructing Standardized Data

For PCA based on standardized variables, the scores and directions reconstruct standardized coordinates, not the original measurements. Suppose variable \(r\) has original mean \(\bar{x}_r\) and standard deviation \(s_r>0\). The standardized coordinate for observation \(i\) is \(z_{ir}=(x_{ir}-\bar{x}_r)/s_r\). After reconstructing \(\hat{z}_{ir,k}\), return to the original scale with

The order matters: first reconstruct in the coordinate system used by PCA, then multiply by each variable’s standard deviation and add that variable’s mean. Adding the means directly to standardized coordinates, or multiplying all variables by a single standard deviation, does not reverse standardization correctly.

Reading and Checking a Reconstruction

A reconstruction should always be labelled with both its coordinate system and the number of components used. A centered reconstruction is expressed as deviations from the means. A reconstruction on the original scale is expressed in the variables’ original units. For standardized PCA, these are different again: reconstructed standardized coordinates must be converted back variable by variable before they can be interpreted in original units.

Several checks help catch arithmetic and interpretation errors:

• Confirm that each score is a dot product of the centered (or standardized) observation with the stated direction.
• For a full reconstruction, verify that the sum of score-weighted directions returns the centered vector.
• For a truncated reconstruction, confirm that only retained score contributions were added.
• When returning to original units, add each variable’s own mean; for standardized PCA, also multiply by its own standard deviation.
• If checking a truncated reconstruction with the squared-length identity, use the omitted scores and an orthonormal set of directions.

A particularly common exam trap is to confuse scores with reconstructed coordinates. A score is one number for one component; it is not itself a value of an original variable. The original-variable contribution comes from multiplying that score by the corresponding direction vector. Another trap is restoring means too early: adding the means to the centered observation before calculating scores changes the projections and no longer follows the stated PCA setup.