Standard, Pareto, and Robust Scaling Compared

Why Compare Scaling Rules?

After within-block centering and feature scaling, a measurement is no longer represented on its original numerical scale. The choice of scale determines how strongly a feature’s variation is represented relative to other features. This matters in multi-omics data because measurements can have very different spreads, and a single unusually large value can affect some scaling rules more than others.

This tutorial compares three feature-wise choices: unit-variance scaling, which divides centered values by their sample standard deviation; Pareto scaling, which divides by the square root of that standard deviation; and median-and-MAD scaling, which centers at the median and divides by a median-based measure of spread. These are deterministic preprocessing rules, not statistical models. They do not require normally distributed measurements to be calculated, though a normal-consistency factor used with the MAD has a specific interpretation under a normal distribution.

Definition: For feature \(j\), let \(x_{ij}\) be its measurement for sample \(i\), for \(i=1,\ldots,n\). Let \(\bar{x}_j\) and \(s_j\) be the sample mean and sample standard deviation. Let \(m_j\) be the sample median, and define the median absolute deviation as \(\mathrm{MAD}_j=\mathrm{median}_i(|x_{ij}-m_j|)\). Unit-variance scaling gives \(z_{ij}=(x_{ij}-\bar{x}_j)/s_j\). Pareto scaling gives \(p_{ij}=(x_{ij}-\bar{x}_j)/\sqrt{s_j}\). Median-and-MAD scaling gives \(r_{ij}=(x_{ij}-m_j)/(c\,\mathrm{MAD}_j)\), where \(c=1.4826\) is the usual normal-consistency factor. The raw MAD, without \(c\), is also used; the convention must be stated.

The factor \(1.4826\) makes the MAD a consistent estimate of the population standard deviation when observations come from a normal distribution, in the large-sample sense. It does not make the MAD exactly unbiased in every finite sample. Using the raw MAD instead changes the numerical scale by a constant factor; it does not change which features have larger or smaller MADs. Here, “robust scaling” refers specifically to median centering and MAD scaling, not to every method that may be called robust.

Assumptions and the Feature Weights

For these formulas, each feature must have numeric measurements on the samples being scaled, and the observations being compared must correspond to the same units of analysis. The spread estimates must be computed from usable values under a consistent missing-data rule. Each denominator must be positive. A constant feature has \(s_j=0\) and \(\mathrm{MAD}_j=0\), so none of the displayed rules can scale it as written. The formulas themselves make no normality assumption. The normal-consistency interpretation of \(c\,\mathrm{MAD}_j\) is the part that invokes a normal reference distribution.

The effect on feature influence follows directly by considering a squared difference between two samples, \(a\) and \(b\), for a single feature. Under unit-variance scaling, that difference is \((x_{aj}-x_{bj})/s_j\), because the common feature center cancels. Its square is the original squared difference multiplied by \(1/s_j^2\). Under Pareto scaling, the squared difference is multiplied by \(1/s_j\). Under median-and-MAD scaling, it is multiplied by \(1/(c\,\mathrm{MAD}_j)^2\). These are the respective feature weights in a sum of squared scaled differences. They describe the scaling’s algebraic effect; they do not by themselves establish that a feature is biologically important or reliable.

\begin{aligned} \text{Unit variance:}\quad & \frac{(x_{aj}-x_{bj})^2}{s_j^2}, &\text{weight} &= \frac{1}{s_j^2},\\ \text{Pareto:}\quad & \frac{(x_{aj}-x_{bj})^2}{s_j}, &\text{weight} &= \frac{1}{s_j},\\ \text{Median--MAD:}\quad & \frac{(x_{aj}-x_{bj})^2}{(c\,\mathrm{MAD}_j)^2}, &\text{weight} &= \frac{1}{(c\,\mathrm{MAD}_j)^2}. \end{aligned}

The standard deviation in Pareto scaling is not squared under the square root: its denominator is \(\sqrt{s_j}\), so the squared difference has weight \(1/s_j\). In the conventional Pareto rule, this is a partial reduction in the spread-based reweighting compared with unit-variance scaling. For instance, multiplying \(s_j\) by four divides the unit-variance weight by sixteen, but divides the Pareto weight by four. The convention is defined in terms of the numerical measurements as recorded; Pareto-scaled values do not generally have unit variance.

Worked Example: Calculating Spread Estimates

Consider two hypothetical continuous features measured on four aligned samples: feature A has values \(1,2,3,4\), and feature B has values \(0,1,1,8\). Compute the centers and scale estimates needed for all three methods.

Feature A. Its mean is \(\bar{x}_A=(1+2+3+4)/4=2.5\). Its squared deviations from the mean sum to \(2.25+0.25+0.25+2.25=5\), so its sample variance is \(5/(4-1)=5/3=1.6667\), and \(s_A=\sqrt{5/3}=1.291\). Its median is \(m_A=(2+3)/2=2.5\). The absolute deviations from that median are \(1.5,0.5,0.5,1.5\), whose median is \(1\). Thus \(\mathrm{MAD}_A=1\), and \(c\,\mathrm{MAD}_A=1.4826\).

Feature B. Its mean is \(\bar{x}_B=(0+1+1+8)/4=2.5\). The squared deviations sum to \(6.25+2.25+2.25+30.25=41\), giving sample variance \(41/3=13.67\) and \(s_B=\sqrt{41/3}=3.697\). Its median is \(m_B=1\). Its absolute deviations are \(1,0,0,7\). For four values, the usual median is the average of the two middle ordered values, so \(\mathrm{MAD}_B=(0+1)/2=0.5\), not zero. The normal-consistent robust scale is therefore \(1.4826(0.5)=0.7413\).

Feature B has greater standard deviation than feature A, but a lower MAD than would be suggested by that standard deviation alone. The value 8 contributes strongly to the standard deviation. The median and MAD are less affected by that single extreme value in this example.

Comparing the Scaled Values

The scale estimates do not fully convey the effect of the methods, so apply each rule to feature B. The calculations use the values \(0,1,1,8\) above. Results are rounded to three significant figures. These transformations are for illustration; they do not imply that the four hypothetical samples represent a population.

Worked Example: Three Transformations of One Feature

For feature B, unit-variance scaling centers at \(2.5\) and divides by \(3.697\). The transformed values are \((0-2.5)/3.697=-0.676\), \((1-2.5)/3.697=-0.406\), \((1-2.5)/3.697=-0.406\), and \((8-2.5)/3.697=1.49\).

For Pareto scaling, \(\sqrt{s_B}=\sqrt{3.697}=1.923\). Centering at \(2.5\) and dividing by \(1.923\) gives \(-1.30,-0.780,-0.780,\) and \(2.86\). These values are not unit-variance scores: the denominator is the square root of the standard deviation, not the standard deviation itself.

For median-and-MAD scaling, the center is \(1\) and the denominator is \(0.7413\). The transformed values are \((0-1)/0.7413=-1.35\), \(0\), \(0\), and \((8-1)/0.7413=9.44\). The outlying value is still far from the others. Robust scaling does not remove or cap extreme values; it makes the center and spread estimates less sensitive to them.

The values differ both because the scale estimates differ and, for median-and-MAD scaling, because the center differs. A transformed value is a position relative to the chosen center and scale, not a probability or a measure of biological importance.

The example also illustrates why the choice is substantive. Unit-variance scaling gives each feature sample variance one, provided the standard deviation is nonzero and the same sample standard deviation is used. Pareto scaling compresses the reweighting associated with large standard deviations. Median-and-MAD scaling relies on central order statistics, so an extreme value has less influence on those estimates, but the extreme observation remains extreme after transformation.

When the MAD Is Zero

A robust scale is not guaranteed to be positive. For example, if sufficiently many observations are tied at the median, more than half of the absolute deviations can be zero, making the usual median of those deviations zero. In that case, division by \(c\,\mathrm{MAD}_j\) is undefined. Do not replace the denominator with an arbitrary tiny positive number: that can produce enormous transformed values and make a feature with little observed variation dominate downstream calculations.

Worked Example: Detecting an Undefined Robust Scale

Suppose a hypothetical feature has measurements \(2,2,2,5,9\). Its median is \(2\), and the absolute deviations are \(0,0,0,3,7\). The middle absolute deviation is \(0\), so \(\mathrm{MAD}=0\) and \(c\,\mathrm{MAD}=0\). Median-and-MAD scaling cannot be applied to this feature as written.

A prespecified rule is needed. Depending on the analysis, one might exclude a feature classified as constant or nearly constant, or use another scale estimate such as the interquartile range with an explicitly chosen convention. That alternative must also be checked for zero values. The rule should be selected before inspecting results in a way that favors a desired outcome, and the number of affected features should be reported.

Choosing Among the Methods

Method	Center and scale	Main effect	Important limitation
Unit variance	Mean and sample standard deviation	Gives each nonconstant feature sample variance one	Standard deviation can be strongly affected by extreme observations
Pareto	Mean and square root of sample standard deviation	Reduces spread-based reweighting compared with unit variance	Does not equalize feature variances; values depend on measurement units and convention
Median-and-MAD	Median and normal-consistent MAD	Uses center and spread estimates less sensitive to extreme values	MAD can be zero; extremes remain in the transformed data

Unit-variance scaling is a natural choice when the goal is to place continuous features on a common sample-variance scale and the standard deviations are meaningful rather than dominated by extreme observations. Pareto scaling is useful when full variance equalization would give small-spread features very large relative weights, but the analyst still wants to reduce differences in scale. Median-and-MAD scaling is a reasonable candidate when extreme observations make mean-and-standard-deviation summaries unrepresentative of the central bulk and a positive MAD is available.

These are not interchangeable solutions to every preprocessing problem. Unit-variance and Pareto scaling can both be sensitive to outliers through their mean and standard-deviation estimates. Median-and-MAD scaling is less sensitive in its center and scale estimates, but it does not diagnose why a measurement is extreme, correct a technical artifact, or make the transformed distribution normal. It can also be unstable or undefined for highly tied or low-variation features. A transformation applied separately to different data partitions should use scale estimates appropriate to the analysis design; otherwise, values may no longer be on a common reference scale.

Diagnostics and Common Pitfalls

Before proceeding with a scaled block, check the per-feature centers and scale estimates, the number of zero or near-zero denominators, and the resulting transformed ranges. Compare a small number of features before and after scaling, especially features with extreme values or many ties. A surprisingly large transformed value may reflect a small denominator rather than a large and reliable biological signal. Also record whether robust scaling used raw MAD or the \(1.4826\) factor; silently changing that convention changes the transformed values.

Wrong reasoning: “The MAD is zero only when every observation is identical, so I can always use it for nonconstant features.” Correction: A feature can vary and still have zero MAD when sufficiently many absolute deviations from the median are zero.
Wrong reasoning: “Robust scaling removes outliers.” Correction: It uses center and scale estimates less sensitive to outliers; it does not remove, cap, or explain extreme observations.
Wrong reasoning: “Pareto scaling gives every feature variance one.” Correction: Unit-variance scaling does that for nonconstant features using the same sample standard deviation. Pareto scaling divides by \(\sqrt{s_j}\), so it generally leaves unequal variances.
Wrong reasoning: “The feature with the largest transformed value must be the most important.” Correction: A transformed value describes a measurement relative to a chosen center and scale. Importance requires a separate scientific or modeling argument.

Key takeaway: Unit-variance scaling uses weight \(1/s_j^2\) in squared differences, Pareto scaling uses \(1/s_j\), and median-and-MAD scaling uses \(1/(1.4826\,\mathrm{MAD}_j)^2\) under the normal-consistent convention. Choose the rule for the intended balance of spread and outlier sensitivity, and verify that every denominator is suitable.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

A feature has sample standard deviation \(s_j=4\). What feature weight does unit-variance scaling give in a squared difference, and what weight does Pareto scaling give?
Why does the median of absolute deviations for the four values \(0,1,1,8\) equal \(0.5\), rather than \(0\)?
For the values \(3,3,3,6,10\), find the median and raw MAD. Can median-and-MAD scaling be applied using the stated formula?
State one reason to choose median-and-MAD scaling and one limitation that remains even after using it.

Solutions. (1) Unit-variance weight is \(1/s_j^2=1/4^2=1/16=0.0625\); Pareto weight is \(1/s_j=1/4=0.25\). (2) The median is \(1\); absolute deviations are \(1,0,0,7\), whose ordered middle two values are \(0\) and \(1\). The even-sample median averages them, giving \((0+1)/2=0.5\). (3) The median is \(3\); absolute deviations are \(0,0,0,3,7\), so the raw MAD is \(0\). The formula cannot be applied because its denominator is zero; a prespecified alternative scale or feature-handling rule is needed. (4) Median-and-MAD scaling can be chosen when extreme values would make mean and standard deviation poor summaries of the central observations. It does not remove extreme measurements, and its MAD can be zero.