PCA With Missing Profile Measurements

Why Missing Measurements Change the PCA Problem

PCA is usually introduced for a rectangular data matrix: every profile has a value for every feature. In genomic profiles, however, some measurements may be unavailable because of assay limits, quality-control filtering, or other causes. A standard singular value decomposition cannot be applied directly to a matrix with blank entries. Replacing those entries is not a neutral bookkeeping step: the values inserted can change the covariance structure and hence the principal components.

The goal is to describe the patterns in the measured profiles without treating unobserved values as if they had been measured. Three common approaches differ in how they handle this distinction. Mean imputation fills each gap with a feature mean before ordinary PCA. Iterative low-rank reconstruction repeatedly estimates missing entries from a low-rank approximation. Observed-entry least squares estimates a low-rank structure by measuring fit only where values were observed.

PCA is commonly performed after centering each feature, and sometimes after scaling features to comparable variances. With missing values, the preprocessing rule must be stated. A practical choice is to estimate each feature mean and standard deviation from its observed entries, then center and, if appropriate, scale those entries. If missingness depends on the unobserved value, these observed-entry estimates need not represent the full feature distribution. Scaling also needs care: a feature with very few observed values can have an unstable estimated standard deviation.

Missingness assumptions matter but do not determine the reconstruction on their own. Missing completely at random (MCAR) means the chance an entry is missing does not depend on observed or unobserved data. Missing at random (MAR) means that, conditional on observed information, missingness does not further depend on the missing value. Missing not at random (MNAR) means missingness still depends on the unobserved value after conditioning on observed information. These labels describe the missingness process; none guarantees that an imputation or low-rank model is correct.

Mean Imputation: A Simple Baseline With Structural Consequences

For each feature, mean imputation replaces every missing entry by that feature's mean among observed profiles. Ordinary PCA is then applied to the resulting complete matrix. If the entries have been centered using those observed means, each imputed value is zero on the centered scale. This makes the procedure easy to implement, but it places every missing profile exactly at the feature average, suppressing its contribution to variation in that feature.

Mean imputation is most defensible as a transparent baseline or when only a small fraction of entries is missing and the goal is exploratory description. Even under MCAR, filling gaps with means generally reduces variation and can alter correlations. Under MAR or MNAR, the observed mean may itself be shifted relative to the mean among all profiles. Standard PCA after single imputation also treats the filled values as known, so its usual variance summaries do not reflect uncertainty about them.

Low-Rank Reconstruction and the Observed-Entry Objective

A rank-\(k\) PCA reconstruction represents the systematic profile structure using \(k\) components. With complete data, the rank-\(k\) truncated singular value decomposition gives the least-squares matrix approximation. With missing data, the relevant observed-entry loss is instead

Only measured entries contribute to this loss. Fitting the same expression as an unweighted sum over a completed matrix would count invented values as observations. The masked objective is nonconvex in \(U\) and \(V\) jointly, and its minimum need not be unique. In particular, enough entries must be observed in informative patterns to determine the factors; a profile or feature with too few observed entries may have poorly determined coordinates.

An iterative low-rank reconstruction is one way to seek a useful fit. A common version starts by filling missing centered entries with zero (or another initial estimate), computes a rank-\(k\) approximation, replaces only the missing entries with their reconstructed values, and repeats. The observed entries remain fixed throughout. This is an algorithmic recipe, not a guarantee of the global minimum of the masked loss. It can depend on initialization and may settle at an unsatisfactory fixed point.

Observed-entry least squares names the fitting target itself: minimize \(\mathcal{L}(U,V)\) over the factors. It can be optimized by alternating least squares. Holding \(V\) fixed, update each profile's row of \(U\) by least squares using that profile's observed features; then hold \(U\) fixed and update each feature's row of \(V\) using its observed profiles. If each block update solves its least-squares problem, the objective does not increase at that update, although the joint problem remains nonconvex and may converge to a local solution. A row update requires enough informative observed features for its factor coordinates to be determined; the analogous condition applies to feature updates.

Iterative low-rank filling and observed-entry least squares are therefore related but not identical. The former is a family of reconstruction algorithms that alternately completes and decomposes a matrix. The latter specifies an objective that ignores missing cells; an optimizer such as alternating least squares can target it directly. A filling algorithm should not be assumed to minimize the masked objective unless its update rule and convergence properties justify that claim.

Assumptions, Diagnostics, and Method Choice

A low-rank reconstruction is meaningful only if a low-rank signal is a reasonable approximation for the profiles after preprocessing. For a statistical interpretation of least-squares fitting, one might additionally posit that observed measurements equal a low-rank signal plus errors with mean zero and comparable variance. If errors have substantially different variances, unweighted squared error gives noisy measurements the same influence as precise ones. If missingness depends on unobserved measurement values, the observed entries can be a selected subset, and a masked fit alone does not correct that selection.

Useful checks include the fraction of missingness by profile and feature, the number and pattern of observations per row and column, and whether missingness is associated with known technical or biological covariates. Inspect the observed-entry residuals \(X_{ij}-(UV^\mathsf{T})_{ij}\) for systematic feature, profile, or batch patterns. Compare results across plausible ranks, initializations, and preprocessing choices. Large changes in the leading subspace or in reconstructed values are evidence that conclusions depend on modeling choices. Convergence of an algorithm is not, by itself, evidence that the model is appropriate.

These approaches are not substitutes for understanding why data are missing. Mean imputation is simple and reproducible, but can be a poor choice when missingness is substantial or correlated with profile structure. Low-rank methods can use cross-feature structure to estimate gaps, but their estimates inherit the chosen rank and factor model. Masked least squares avoids pretending that gaps were measured, but it can still produce unstable fits when there is insufficient information. None automatically supplies uncertainty intervals for each reconstructed value.

A practical report should state how features were centered and scaled, how missing entries were handled, what rank was used, and which objective the fitting procedure targeted. It should distinguish observed measurements from model-based reconstructions. If results are sensitive to these choices, present that sensitivity rather than treating one completed matrix as definitive.

Common Pitfall

A frequent error is to say, “The low-rank reconstruction is the value that was missing,” or to treat an iterative update as if it were a measured observation. The correction is that the method supplies a model-dependent estimate. Its accuracy depends on the low-rank approximation, the observation pattern, preprocessing, and missingness process. Another error is to claim that observed-entry least squares solves the same problem as ordinary PCA after filling gaps. It does not: the former excludes missing cells from its loss, while the latter's PCA is affected by every imputed value.