Gene Scaling and Its Effects on PCA

Why Gene Scaling Changes PCA

PCA identifies directions of greatest variation among cells. But “greatest” depends on the units and scales of the gene measurements. If one gene varies much more across cells than another, PCA on centered expression can devote most of its first component to the high-variance gene. Standardizing each gene before PCA changes the criterion: each gene is put on a unit-variance scale, so low-variance genes can contribute as much as high-variance genes.

That change is not automatically an improvement. A gene may have low variance because it carries little biological signal, but it may also be measured on a different scale. Standardization can make a weak, noisy pattern influential. The choice should therefore be understood as a modeling decision about which genes’ variation PCA should treat as comparable.

Definition: Let \(X\) be an \(n\times p\) expression matrix, with cells in rows and genes in columns. Let \(X_c\) denote the matrix after subtracting each gene’s mean across the \(n\) cells. The sample covariance matrix is \(S=X_c^\mathsf{T}X_c/(n-1)\). PCA on centered expression uses the eigenvectors of \(S\). For gene \(j\), let \(s_j\) be its sample standard deviation, and let \(D\) be the diagonal matrix with entries \(s_1,\ldots,s_p\). Gene-wise standardization forms \(Z=X_cD^{-1}\) and applies PCA to \(R=Z^\mathsf{T}Z/(n-1)=D^{-1}SD^{-1}\), the sample correlation matrix when all gene standard deviations are positive.

Both procedures center the genes first. They differ in whether each gene is also divided by its standard deviation. In centered-expression PCA, the input retains the genes’ original measurement scales, and the covariance matrix records their variances and covariances. In standardized PCA, each column of \(Z\) has sample mean zero and sample variance one; the covariance matrix of \(Z\) is the correlation matrix of the original centered genes.

The algebraic rule is straightforward, but expression values must be on a meaningful numeric scale for the chosen analysis. Standardization does not make an unsuitable input scale suitable, remove outliers, or correct technical artifacts. The examples below use hypothetical, already-prepared expression measurements and focus only on the effect of scaling.

What the Scaling Does to the PCA Criterion

For a unit vector \(v\), the variance of the cell scores \(X_cv\) is \(v^\mathsf{T}Sv\). The first covariance-PCA direction maximizes this quantity subject to \(v^\mathsf{T}v=1\). For standardized data, the corresponding criterion is \(a^\mathsf{T}Ra\), with \(a^\mathsf{T}a=1\). These are different optimization problems unless scaling leaves the feature geometry unchanged.

The connection follows by writing the standardized score for a cell as \(X_cD^{-1}a\). Thus, in the original centered-expression coordinates, the standardized-PCA direction has coefficients \(D^{-1}a\), not generally \(a\). The unit-length constraint applies to \(a\), the direction in standardized coordinates. This is why a loading vector from standardized PCA should be read as a direction among standardized genes; it is not a vector of effects per original expression unit.

Key result: If every gene has a positive sample standard deviation, PCA of \(Z=X_cD^{-1}\) is PCA of the correlation matrix \(R=D^{-1}SD^{-1}\). It gives each gene unit marginal variance before comparing directions. PCA of \(X_c\), in contrast, uses the original gene variances in \(S\); a high-variance gene can therefore have greater influence on the leading components.

A short derivation makes the distinction explicit. If \(v\) is a unit direction in the original coordinates, the sample variance of its cell scores is

\frac{1}{n-1}\lVert X_cv\rVert^2 =\frac{1}{n-1}v^\mathsf{T}X_c^\mathsf{T}X_cv =v^\mathsf{T}Sv.

The largest value of this Rayleigh quotient is the largest eigenvalue of \(S\), attained at a corresponding unit eigenvector. Replacing \(X_c\) with \(Z=X_cD^{-1}\) gives the quotient \(a^\mathsf{T}Ra\) instead. This is not merely a numerical rescaling of the final scores: it changes which direction is selected to maximize variance.

Worked Example: A High-Variance Gene Dominates Covariance PCA

Suppose two hypothetical genes have sample covariance matrix

S= \begin{pmatrix} 100&8\\ 8&1 \end{pmatrix}.

The diagonal entries are the sample variances: gene 1 has variance \(100\), while gene 2 has variance \(1\). The sample standard deviations are \(10\) and \(1\). The first principal component uses the eigenvector for the larger eigenvalue. The characteristic equation is

\det(S-\lambda I) =(100-\lambda)(1-\lambda)-64 =\lambda^2-101\lambda+36=0.

The eigenvalues are \(\lambda=(101\pm\sqrt{10057})/2\), giving approximately \(100.6423\) and \(0.3577\). For the larger eigenvalue, the first row of \(Sv=\lambda v\) gives

\frac{v_2}{v_1} =\frac{\lambda-100}{8} =\frac{100.6423-100}{8} \approx 0.08029.

A unit eigenvector in the leading direction is therefore approximately \((0.99679,\,0.08003)^\mathsf{T}\). The direction is dominated by gene 1, the higher-variance gene. The fraction of total variance accounted for by this component is \(100.6423/(100+1)\approx0.9965\), or about \(99.65\%\). This concentration reflects the covariance structure and the large scale of gene 1; it does not by itself establish that gene 1 is biologically more important.

Standardization Can Change Which Genes Lead

Dividing each centered gene by its standard deviation removes differences in marginal variance. It does not remove relationships between genes: the off-diagonal entries of \(R\) still measure their correlations. Consequently, a group of genes with modest individual variance may define a leading component after standardization if they share a strong pattern across cells.

Worked Example: A Shared Pattern Leads After Standardization

For a second hypothetical pair of genes, suppose the covariance matrix is

S= \begin{pmatrix} 100&8\\ 8&1 \end{pmatrix}.

The standard deviations are \(10\) and \(1\), so their correlation is \(8/(10\cdot1)=0.8\). Standardizing produces

R= \begin{pmatrix} 1&0.8\\ 0.8&1 \end{pmatrix}.

The eigenvectors of \(R\) are proportional to \((1,1)^\mathsf{T}\) and \((1,-1)^\mathsf{T}\), with eigenvalues \(1.8\) and \(0.2\). The leading standardized-PC direction is therefore \((1,1)^\mathsf{T}/\sqrt{2}\): the two genes contribute equally in standardized coordinates. It accounts for \(1.8/(1.8+0.2)=90\%\) of the standardized total variance. The corresponding centered-expression PCA was dominated by gene 1, as the previous calculation showed. The correlation between genes is the same in both descriptions, but the variance criterion changes their relative influence.

The standardized loading vector describes weights on standardized gene values. In this example, a one-unit change in either standardized gene contributes equally to the component score. In original units, however, one unit of the first gene is not equivalent to one unit of the second: standardization divides the first gene’s centered values by \(10\) and the second’s by \(1\). Interpret loadings in the coordinate system used to fit the PCA.

Worked Example: A Low-Variance Gene Can Gain Equal Weight

Consider another hypothetical covariance matrix:

S= \begin{pmatrix} 100&2\\ 2&1 \end{pmatrix}.

The standard deviations are \(10\) and \(1\), and the correlation is \(2/(10\cdot1)=0.2\). For centered-expression PCA, the larger eigenvalue is

\lambda_1 =\frac{101+\sqrt{9817}}{2} \approx100.0404.

The eigenvector equation gives \(v_2/v_1=(\lambda_1-100)/2\approx0.02019\), so a unit leading direction is approximately \((0.99980,\,0.02019)^\mathsf{T}\). Gene 2 contributes little to this direction. The correlation matrix after standardization is instead

R= \begin{pmatrix} 1&0.2\\ 0.2&1 \end{pmatrix}, \qquad a_1=\frac{1}{\sqrt{2}} \begin{pmatrix}1\\1\end{pmatrix}, \qquad \lambda_1(R)=1.2.

In standardized coordinates, the two genes have equal leading-component weights, and the first component accounts for \(1.2/2=60\%\) of the total standardized variance. The lower-variance gene has gained influence because its values are divided by its smaller standard deviation. Whether that influence is useful depends on whether its variation is reliable and relevant.

Assumptions, Diagnostics, and Failure Modes

PCA is a descriptive linear decomposition of a numeric matrix; it does not require gene independence. To interpret the comparison above, the cells and genes must be consistently defined for both analyses, centering and standard deviations must be computed across the same cells, and each gene to be standardized must have positive, meaningfully estimated variance. Standardization also assumes that treating a one-standard-deviation change in each included gene as comparable is appropriate for the scientific objective.

Before trusting a scaled PCA, check how the operation behaves for the actual data. In particular:

Inspect gene standard deviations. Very small standard deviations produce large multipliers \(1/s_j\). A nearly constant gene can then contribute strongly after scaling.
Check for zero-variance genes. If \(s_j=0\), division by \(s_j\) is undefined. Such a gene cannot be standardized by this formula and must be excluded from this calculation or handled by an explicitly justified alternative.
Compare the leading directions under both choices. Large changes in loadings or cell scores indicate that the scaling decision materially affects the result; they are not a reason to report only the more appealing plot.
Examine genes with large standardized loadings. Ask whether their variation is reproducible and interpretable, rather than assuming that unit variance makes them informative.
Check sensitivity to unusual observations. A small number of cells can affect means, standard deviations, covariances, and correlations, and hence can alter either PCA.

An especially important failure mode is mistaking variance equalization for noise removal. If a low-variance gene’s variation is mostly measurement noise, standardizing it still gives it variance one; that can make the noise compete with stronger patterns. Conversely, covariance PCA can underweight a genuinely informative gene simply because its numerical scale is small. Neither choice determines biological relevance by itself.

Choosing Between Covariance and Correlation PCA

Choice	What determines a gene’s influence	Useful when	Main caution
PCA on centered expression	Original gene variances and covariances	Differences in variability on the chosen scale are meaningful	High-variance genes may dominate the leading components
PCA after gene-wise standardization	Correlations; each included gene has unit variance	Genes should contribute comparably despite different marginal scales	Low-variance or noisy genes can be amplified

A practical comparison is to fit both versions on the same prepared expression matrix and examine whether the leading patterns are stable and scientifically interpretable. If they differ, report the scaling choice and understand which genes or groups of genes account for the change. A scaled analysis is not simply “fairer” to every gene; it encodes a different definition of variation.

This tutorial concerns how gene-wise scaling changes PCA’s covariance structure and leading directions. It does not imply that every gene should enter PCA, nor that a particular scaling choice is universally best. The standard deviations are estimated from the cells at hand, and scaling cannot distinguish biological signal from technical variation. Results should be interpreted with those limits in view.

Common mistake: “Standardization makes all genes equally important, so the leading components are unbiased.” Standardization makes each included gene’s sample variance equal to one; it does not make their signals equally reliable or biologically relevant. The correction is to treat scaling as an explicit modeling choice, inspect the genes gaining influence, and compare conclusions under the relevant alternatives.

Check Your Understanding

Use the definitions and calculations above to answer these questions. Brief solutions follow.

What matrix does PCA use after each gene is centered and divided by its sample standard deviation?
In the matrix \(S=\begin{pmatrix}100&2\\2&1\end{pmatrix}\), what is the correlation between the genes?
Why can a gene with small variance have a large influence after standardization?
What should be done with a gene whose sample standard deviation is zero before applying the stated standardization formula?
Does unit variance after standardization show that a gene’s variation is reliable? Explain.

Solutions. (1) The PCA input has covariance matrix \(R=D^{-1}SD^{-1}\), the correlation matrix. (2) The correlation is \(2/(10\cdot1)=0.2\). (3) Dividing by its small standard deviation magnifies its centered values, giving the gene unit variance and allowing its pattern to contribute comparably in standardized coordinates. (4) The division is undefined, so the gene must be excluded from this calculation or handled by a justified alternative. (5) No. Standardization sets the observed sample variance to one; it does not distinguish reliable signal from noise.