Rank-K Reconstruction and PCA Compression

From PCA Coordinates to Reconstructed Profiles

PCA can represent each profile using a smaller set of component scores, but those scores are useful for compression only if we can reconstruct an approximation to the original measurements. In genomic data, a profile might be a sample measured on many features. Retaining fewer components means storing fewer numbers, at the cost of losing some variation in those profiles.

Let \(X\) be an \(n\times p\) data matrix after the chosen centering and scaling steps: rows are profiles, columns are features, and \(x_{ij}\) is the processed value for profile \(i\) and feature \(j\). As in the earlier material on centering and scaling, the chosen preprocessing determines the scale on which PCA operates. Write the singular value decomposition as \(X=UDV^\mathsf{T}\), with singular values \(d_1\ge d_2\ge\cdots\ge d_r>0\), where \(r\) is the rank of \(X\). The columns of \(U\) and \(V\) are orthonormal.

Each row of \(X_k\) is a reconstructed profile. Its values are obtained by taking that profile’s \(k\) scores and combining them with the corresponding feature loadings. With \(k=r\), the reconstruction is exact. With fewer components, the omitted directions contribute to the reconstruction error.

Why Truncation Minimizes Squared Error

The rank-\(k\) reconstruction is not just one convenient way to compress the matrix: among all matrices of rank at most \(k\), it has the smallest total squared error measured by the Frobenius norm. This result applies to a fixed, finite matrix and the stated squared-error objective. It does not say that this objective is the right one for every scientific purpose.

A short derivation explains both the optimality and the error formula. Let \(P\) be the orthogonal projection onto the row space of \(A\). Because every row of \(A\) lies in this space, \(A(I-P)=0\). Right multiplication by an orthogonal projection cannot increase the Frobenius norm, so

The eigenvectors of \(X^\mathsf{T}X\) are the right singular vectors \(v_j\), with eigenvalues \(d_j^2\). If the projection space has dimension at most \(k\), the greatest possible squared length retained by projecting onto it is \(\sum_{j=1}^{k}d_j^2\): a \(k\)-dimensional subspace captures most squared length by spanning the eigenvectors with the largest eigenvalues. Since \(\|X\|_F^2=\sum_{j=1}^{r}d_j^2\), the least possible squared residual is therefore \(\sum_{j=k+1}^{r}d_j^2\). The row space spanned by \(v_1,\ldots,v_k\) achieves this bound, and projecting \(X\) onto that space gives \(X_k\). Thus

This formula also connects truncation to explained variance. The total squared magnitude of the processed matrix is \(\sum_{j=1}^{r}d_j^2\), and the fraction retained by the first \(k\) components is \(\sum_{j=1}^{k}d_j^2/\sum_{j=1}^{r}d_j^2\). The remaining fraction is the squared reconstruction error divided by the total squared magnitude. This is a statement about the chosen matrix scale and squared-error measure, not a guarantee that every feature or profile is reconstructed equally well.

Worked Reconstructions for Hypothetical Profiles

Assumptions, Diagnostics, and Alternatives

The optimality result is exact for the chosen matrix, rank constraint, and unweighted sum of squared entrywise errors. It does not require that the entries follow a probability distribution. Its practical interpretation, however, depends on how the matrix was constructed. Centering, feature scaling, transformations, and the presence of missing values affect which differences count as large. The examples above use complete, centered matrices; PCA reconstruction of a matrix with missing entries requires a different treatment.

When inspecting a proposed compression, check the singular values and calculate the tail sum for the chosen \(k\). Also examine errors by profile or feature: a modest total error can conceal a large error for one profile or a particular feature. These checks do not change the theorem; they help establish whether its overall squared-error summary matches the intended use. Detailed analysis of reconstruction residual patterns is a distinct diagnostic task.

PCA truncation is one option among several. Keeping selected original features preserves their direct interpretation but does not generally minimize total squared error at a fixed number of retained features. Lossless storage preserves all recorded values but does not reduce the information in the matrix. Rank-\(k\) PCA instead uses dense linear combinations of features and gives the best rank-at-most-\(k\) approximation under Frobenius loss. Those components need not correspond to single genes or biological processes, and the result is not evidence of a causal or biological mechanism.

• Changing scale changes the target. If features are standardized before PCA, squared error is measured on that standardized scale. Transforming a reconstruction back to the original units does not make its errors comparable across features in the same way.
• A small total error is not a per-feature guarantee. The theorem minimizes the sum across all entries. It does not minimize the largest individual error or ensure equal accuracy across profiles.
• Compression and reconstruction quality can conflict. Increasing \(k\) cannot increase the truncated-SVD squared error, but may increase storage and may provide little practical benefit.
• Components are not always uniquely oriented. If singular values are repeated, different orthonormal bases can span the same optimal subspace. The reconstruction can be well-defined even when individual component directions are not unique.