Centering Data for PCA

Why PCA Centers the Data

Principal component analysis (PCA) describes directions of variation among observations. To describe variation, it must distinguish how observations differ from one another from where the whole data cloud happens to sit on the coordinate axes. Centering makes that distinction: it moves the data centroid to the origin by subtracting each variable’s mean from its observed values.

This operation is simple, but its interpretation matters. PCA on uncentered data can be driven by the variables’ overall levels relative to the origin, rather than by their variation around the data cloud’s center. Centering does not change the differences between observations, but it changes the reference point from which those differences are measured.

The subtraction is performed column by column: all observations of variable 1 have the mean of variable 1 subtracted from them, and so on. Centering does not subtract one overall mean from the entire data table, and it does not subtract each observation’s row mean. For the standard PCA interpretation here, the input is a rectangular set of observations on numeric variables, with values available to calculate each variable’s mean.

What Centering Guarantees

Every column of centered values sums to zero. This follows directly from the definition:

Thus, the average centered value of each variable is zero. In geometric terms, the centroid of the centered observations is at the origin. Individual observations need not be at the origin, and centering does not make their variability zero.

The same idea applies to any direction through the data. If \(w\) is a direction vector and \(w^\mathsf{T}x_i\) is the projection of observation \(i\) onto that direction, the mean of these projected values is \(w^\mathsf{T}\bar{x}\). The centered projection is therefore \(w^\mathsf{T}(x_i-\bar{x})\). For a unit direction \(w\), the sample variance of the projected observations is:

This expression measures squared deviations of projected observations from their projected mean. It does not measure squared projected distance from the origin unless that mean is zero. PCA uses variation around the centroid; centering puts that reference point at zero.

Why the Centroid Is the Reference Point

A useful justification is that the centroid is the point that minimizes the total squared distance from the observations. Let \(a\) be any proposed reference point and write each observation as \(x_i=\bar{x}+(x_i-\bar{x})\). Expanding the squared distances gives:

The cross term is zero because the centered observations sum to zero. The final term is nonnegative, so the total is minimized when \(a=\bar{x}\). This explains why centering is not an arbitrary adjustment: it chooses the reference point that leaves the squared distances measuring spread around the observations’ own centroid.

The same reasoning applies after projection onto any fixed direction. Projected observations have mean \(w^\mathsf{T}\bar{x}\); their deviations from that mean are the projections of \(x_i-\bar{x}\). Hence a direction’s PCA variation criterion is based on those centered projections. The detailed construction of the covariance matrix and the procedure for finding principal directions are separate steps.

Variation Is Not Distance From the Origin

A direct comparison shows why centering matters. Consider three hypothetical observations with values \((1,2)\), \((3,2)\), and \((5,2)\). Their centroid is \((3,2)\), so their centered observations are \((-2,0)\), \((0,0)\), and \((2,0)\). There is variation in the first variable and none in the second.

For a unit direction \(w\), a projected value is \(w^\mathsf{T}x_i\). PCA compares the squared deviations of these projections from their mean. Along the first coordinate direction \(w=(1,0)\), the original projections are \(1,3,5\), with mean \(3\); the centered projections are \(-2,0,2\), and their squared deviations sum to \(4+0+4=8\). Along the second coordinate direction \(w=(0,1)\), all three projections are \(2\), so their centered deviations and their squared deviations sum are zero.

If instead one sums squared projections from the origin, the first direction gives \(1^2+3^2+5^2=35\), while the second gives \(2^2+2^2+2^2=12\). Those totals include the variables’ levels relative to the origin. In particular, the second variable has a nonzero total despite having no variation at all. Centering removes that misleading contribution before PCA evaluates variation.

Centering does not mean that a large mean is irrelevant in every practical analysis. It means that ordinary PCA is being used to study variation around the data centroid, rather than to prioritize variables or directions merely because their values are far from zero.

Practical Interpretation and Common Traps

For complete numeric data with \(n>1\), centering each variable provides a consistent reference point for measuring variation. It also means that translating all observations by the same constant vector leaves the centered data unchanged. The calculations depend on the observations included: if the data set changes, its means and centered values may change as well.

• Center columns, not rows. A separate mean is calculated for each variable across observations.
• Do not confuse centering with scaling. Centering subtracts a mean; it does not divide by a standard deviation or otherwise change the units.
• Do not interpret uncentered squared values as variation. Squared distance from the origin includes both the spread around the centroid and the centroid’s distance from the origin.
• Check the result. Each centered column should sum to zero, apart from small rounding differences if values were rounded during calculation.

Centering is appropriate for the standard PCA question of finding directions that describe variation around the observations’ centroid. It does not, by itself, make different variables comparable in scale, address missing data, or turn categorical labels into suitable numeric measurements. Those are separate data-preparation decisions.