Within-Block Centering and Feature Scaling

Why Center and Scale Within Blocks?

An omics data set may combine measurements recorded on very different scales: a transformed expression value, a protein intensity, and a metabolite concentration need not have comparable numerical ranges or units. Even after appropriate transformations and normalization, features within and across blocks can differ in their means and variability. If features are to be used together in a method sensitive to numerical scale, those differences can affect the analysis.

Centering and feature scaling are two distinct operations. Centering subtracts a feature’s mean, so its transformed values are expressed relative to that feature’s average. Scaling divides those centered values by a measure of spread. In the common standardization procedure, that measure is the sample standard deviation, giving each nonconstant feature a sample mean of zero and a sample standard deviation of one.

The operations are performed feature by feature within each block. They are not performed by subtracting one grand mean from the entire data set, nor by treating every measurement in a block as if it were one feature. The resulting standardized value describes a sample’s position on one feature relative to the reference samples used to calculate that feature’s mean and standard deviation.

Definition: Let $x_{ij}^{(b)}$ be the measured value for sample $i$, feature $j$, and omics block $b$. For a feature observed in $n$ reference samples, its sample mean and sample standard deviation are $\bar{x}_{\cdot j}^{(b)}$ and $s_j^{(b)}$. The centered value is $x_{ij}^{(b)}-\bar{x}_{\cdot j}^{(b)}$. If $s_j^{(b)}>0$, the standardized value, or z-score, is $$ z_{ij}^{(b)}=\frac{x_{ij}^{(b)}-\bar{x}_{\cdot j}^{(b)}}{s_j^{(b)}}. $$ The calculation is repeated separately for each feature in each block.

What the Standardized Values Mean

The original value $x_{ij}^{(b)}$ has the units of the measured feature. Its centered value retains those units, but is measured relative to the reference-sample mean. The z-score has no physical measurement units: it is the number of sample standard deviations by which the observation lies above or below that mean. A z-score of $1.5$ means the value is 1.5 sample standard deviations above the feature’s reference mean; a score of $-0.5$ means it is half a sample standard deviation below.

For a complete set of $n$ observations with $s_j^{(b)}>0$, standardizing with the sample standard deviation gives

\frac{1}{n}\sum_{i=1}^{n}z_{ij}^{(b)}=0 \qquad\text{and}\qquad \frac{1}{n-1}\sum_{i=1}^{n}\left(z_{ij}^{(b)}\right)^2=1.

The second expression is the sample variance of the z-scores. The denominator $n-1$ matters: using the sample standard deviation makes the sample variance of the standardized observations equal to one. It does not imply that every future sample or a different population has mean zero and variance one.

Worked Matrix Calculation

Worked Example: Standardizing Two Omics Blocks

Consider three hypothetical samples measured on two features in each of two blocks. The values below are already on scales suitable for this calculation.

Sample	Block A, feature A1	Block A, feature A2	Block B, feature B1	Block B, feature B2
1	2	10	100	1
2	4	10	110	3
3	6	13	120	5

For feature A1, the mean is $\bar{x}= (2+4+6)/3=4$. Its sample standard deviation is $s=\sqrt{[(2-4)^2+(4-4)^2+(6-4)^2]/(3-1)}=\sqrt{8/2}=2$. Thus the three z-scores are $(2-4)/2=-1$, $(4-4)/2=0$, and $(6-4)/2=1$.

For feature A2, $\bar{x}=(10+10+13)/3=11$ and $s=\sqrt{[(-1)^2+(-1)^2+2^2]/2}=\sqrt{3}\approx1.732$. Its z-scores are $-1/\sqrt{3}\approx-0.5774$, $-1/\sqrt{3}\approx-0.5774$, and $2/\sqrt{3}\approx1.155$.

For feature B1, $\bar{x}=(100+110+120)/3=110$ and $s=\sqrt{[(100-110)^2+(110-110)^2+(120-110)^2]/2}=10$. Its z-scores are $-1,0,1$. For B2, $\bar{x}=(1+3+5)/3=3$ and $s=\sqrt{[(-2)^2+0^2+2^2]/2}=2$, giving z-scores $-1,0,1$.

The standardized matrix is therefore:

Sample	Block A, feature A1	Block A, feature A2	Block B, feature B1	Block B, feature B2
1	-1.000	-0.5774	-1.000	-1.000
2	0	-0.5774	0	0
3	1.000	1.155	1.000	1.000

For example, sample 3’s A2 measurement is 1.155 sample standard deviations above the A2 mean in this reference set. The calculation does not say that A2 has the same scientific meaning or reliability as B1. Nor does standardizing each feature ensure that the blocks as a whole contribute equally to a later analysis.

Why the Basic Properties Hold

The mean-zero property follows directly from subtracting the sample mean. For one feature, write $z_i=(x_i-\bar{x})/s$, with $s>0$. Since $\sum_i(x_i-\bar{x})=0$,

\sum_{i=1}^{n}z_i =\frac{1}{s}\sum_{i=1}^{n}(x_i-\bar{x}) =0.

For the sample variance, use the definition $s^2=\sum_i(x_i-\bar{x})^2/(n-1)$. Then

\frac{1}{n-1}\sum_{i=1}^{n}z_i^2 =\frac{1}{n-1}\sum_{i=1}^{n}\frac{(x_i-\bar{x})^2}{s^2} =\frac{s^2}{s^2} =1.

These are algebraic properties of the observed reference data, not evidence that the feature values are normally distributed. Standardization requires neither normality nor independent observations for these identities. It does require a defined set of values from which to compute the mean and a positive, finite sample standard deviation.

Reference Samples and Practical Decisions

Worked Example: Applying a Training-Set Scale to a New Sample

Suppose a hypothetical feature has training-reference values 2, 4, and 6. Their mean is 4 and sample standard deviation is 2, so the fitted transformation for this feature is $z=(x-4)/2$. A new sample with value 8 receives $z=(8-4)/2=2$: its value is two training-set standard deviations above the training mean.

If the new value were instead included before estimating the scale, the four-value mean would be $(2+4+6+8)/4=5$ and the sample standard deviation would be $\sqrt{[(2-5)^2+(4-5)^2+(6-5)^2+(8-5)^2]/3}=\sqrt{20/3}\approx2.582$. Its resulting z-score would be $(8-5)/2.582\approx1.162$. That answers a different question: it describes the value relative to a reference set that includes the new sample.

When evaluating a predictive model, estimate the feature means and standard deviations using only the training data, then apply those same values to validation or test data. Re-estimating them using held-out samples lets information from those samples enter the preprocessing step and can bias the assessment. In a purely descriptive analysis of a fixed cohort, it may be appropriate to define the cohort itself as the reference set; state that choice clearly.

Assumptions, Diagnostics, and Failure Modes

The arithmetic is simple, but it does not make every feature suitable for standardization. Check the data and the intended comparison before calculating z-scores.

Nonzero variability. If all observed values for a feature are identical, $s=0$, so the z-score is undefined. Do not silently divide by zero or replace the result with an arbitrary value. A constant feature contains no variation in that reference set and usually needs to be omitted or handled explicitly.
Enough observed values. A sample standard deviation requires at least two observed values, and estimates from very few samples can be unstable. Record how missing values were handled. If each feature uses a different subset of samples, its mean and scale may refer to a different reference group.
Comparable samples and preprocessing. A feature’s mean and spread summarize the values supplied to the calculation. Confirm that the samples represent the intended reference population and that earlier transformations or normalizations appropriate to the assay have already been considered. Standardization does not repair an inappropriate input scale or a technical artifact.
Extreme values. The mean and sample standard deviation can be strongly affected by extreme observations. Inspect feature distributions and investigate unexpected values. A large z-score is a flag for relative distance from the mean, not proof of an error. Alternative scaling choices are useful in some settings, but require their own justification.
Block interpretation. Scaling each feature separately does not make all blocks equally informative, equally reliable, or equally influential in a combined analysis. It also does not remove correlation among features. Those are separate questions from whether each feature has been centered and scaled.

A useful diagnostic is to recompute the sample mean and sample standard deviation of each transformed feature on the reference observations. Up to rounding, they should be 0 and 1, respectively. If not, check that the same observations and missing-value convention were used in the calculation and diagnostic, that the scale used was the sample standard deviation, and that no later step changed the transformed values.

Choosing Among Basic Preprocessing Options

Operation	Transformation	Result and use
No centering or scaling	Keep $x_{ij}^{(b)}$	Retains original units and scale; features with large numerical ranges may dominate scale-sensitive calculations.
Centering only	$x_{ij}^{(b)}-\bar{x}_{\cdot j}^{(b)}$	Centers each feature at zero while retaining the feature’s original units and spread.
Centering and standardizing	$(x_{ij}^{(b)}-\bar{x}_{\cdot j}^{(b)})/s_j^{(b)}$	Expresses values in feature-specific sample standard deviation units; useful when feature scales should be comparable for the intended method.

The appropriate choice depends on the scientific meaning of the measurements and the analysis that follows. Standardization is not automatically preferable: if absolute variability or original units carry important meaning, dividing by the standard deviation may obscure it. Conversely, when a method is sensitive to feature scales and the features are intended to have comparable scale, leaving very different numerical ranges untreated can be consequential. Neither choice by itself determines how much influence an entire block has.

Common Mistake

Common pitfall: “Every standardized feature has variance one, so the omics blocks now contribute equally.” The correction is that variance one is a feature-wise property in the reference observations. A block can contain many features, correlated features, noisy features, or features with very different scientific reliability. Equal feature variance does not establish equal block influence or equal scientific importance.

Another frequent error is to interpret a z-score as a percentage or as a count of raw measurement units. It is neither: it is a signed distance from a feature-specific reference mean, measured in that feature’s sample standard deviations. Always identify the reference samples when interpreting it.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A feature has reference values 3, 5, and 7. Find its mean, sample standard deviation, and standardized values.
A centered feature has values $-4, 0, 4$. What are its sample standard deviation and standardized values?
What does a z-score of $-0.8$ mean in context, and does it mean that the raw value is negative?
Why is a constant feature not standardizable by the usual z-score formula?
In predictive evaluation, why should the training-set mean and standard deviation be applied to test samples rather than estimated using the combined training and test samples?

Solutions. (1) The mean is $(3+5+7)/3=5$; the sample standard deviation is $\sqrt{[(3-5)^2+(5-5)^2+(7-5)^2]/2}=2$. The z-scores are $-1,0,1$. (2) The mean is 0 and the sample standard deviation is $\sqrt{[(-4)^2+0^2+4^2]/2}=4$, so the z-scores are $-1,0,1$. (3) The measurement is 0.8 sample standard deviations below the reference mean. The raw value may still be positive; the z-score’s sign describes its position relative to the mean. (4) Its sample standard deviation is zero, making division by $s$ undefined. (5) Using test samples to estimate preprocessing values lets held-out information enter the model evaluation; applying the training reference scale avoids that leakage and gives a consistent transformation.

Operation	Transformation	Result and use
No centering or scaling	Keep \(x_{ij}^{(b)}\)	Retains original units and scale; features with large numerical ranges may dominate scale-sensitive calculations.
Centering only	\(x_{ij}^{(b)}-\bar{x}_{\cdot j}^{(b)}\)	Centers each feature at zero while retaining the feature’s original units and spread.
Centering and standardizing	\((x_{ij}^{(b)}-\bar{x}_{\cdot j}^{(b)})/s_j^{(b)}\)	Expresses values in feature-specific sample standard deviation units; useful when feature scales should be comparable for the intended method.