Why Predictor Scale Matters
Distance-based clustering compares observations using their predictor values. If one predictor is measured in dollars and another is a small count, the dollar differences can dominate a distance calculation simply because their numerical values are larger. That may be appropriate if dollars should dominate the comparison; otherwise, the resulting clusters can reflect units of measurement more than the intended notion of similarity.
Scaling changes the numerical representation of predictors before distances are calculated. The two common operations are centering, which subtracts a reference value such as the mean, and standardization, which also divides by a measure of spread. They have different effects: centering changes coordinates but not pairwise Euclidean distances, while standardization generally changes the relative contributions of predictors.
For a set of numeric predictors, the squared Euclidean distance between observations \(a\) and \(b\) is the sum of their squared coordinate differences. The distance itself is the nonnegative square root of that sum. The squared form is convenient for comparing which observation is nearer, because taking square roots does not change the ordering.
This formula presumes that the observations have values on the same \(p\) numeric predictors and that Euclidean distance is the chosen measure of dissimilarity. It does not, by itself, say that all predictors deserve equal influence. The choice of scaling determines how differences in their original units contribute.
Centering and Standardization Are Not the Same
Centering subtracts the same constant \(c_j\) from every observation on predictor \(j\). It can make values easier to interpret relative to a typical observation, but it does not alter differences between observations. Standardization divides those centered values by \(s_j\), so a one-unit difference on a predictor with small spread can count more than a one-unit difference on a predictor with large spread.
Thus, after standardization, the squared Euclidean distance is the original squared difference on each predictor divided by that predictor's scale squared. With standard deviations as the scales, a difference is judged relative to the typical variation in that predictor.
A short justification follows directly from subtraction. For any two observations, their centered difference on predictor \(j\) is \((x_{aj}-c_j)-(x_{bj}-c_j)=x_{aj}-x_{bj}\). The center cancels, so every squared difference in the distance sum is unchanged. If each centered difference is instead divided by \(s_j\), its square becomes \((x_{aj}-x_{bj})^2/s_j^2\). Unless all scales are equal to one, the contributions and usually the distances change.
Worked Examples
Worked Example: A Change in the Nearest Cluster Center
A hypothetical insurer summarizes two numeric characteristics of each policy: annual premium in dollars and claim count. Consider a policy with values \((52{,}000,1)\), and two existing cluster centers, \(A=(50{,}000,0)\) and \(B=(55{,}000,1)\). Compare the nearer center before and after scaling. For the scaled calculation, use reference standard deviations of \(10{,}000\) dollars and \(0.5\) claims.
Method. Compare squared Euclidean distances, first in the original units and then after dividing each predictor difference by its stated standard deviation. The square root is unnecessary for deciding which center is nearer.
Raw distance to A. The differences are \(52{,}000-50{,}000=2{,}000\) dollars and \(1-0=1\) claim. Therefore, \(d^2(P,A)=2{,}000^2+1^2=4{,}000{,}000+1=4{,}000{,}001\).
Raw distance to B. The differences are \(52{,}000-55{,}000=-3{,}000\) dollars and \(1-1=0\) claims. Therefore, \(d^2(P,B)=(-3{,}000)^2+0^2=9{,}000{,}000\). Since \(4{,}000{,}001<9{,}000{,}000\), the raw calculation selects A.
Scaled distance to A. The standardized differences are \(2{,}000/10{,}000=0.2\) and \(1/0.5=2\). Hence, \(d^2_{\mathrm{scaled}}(P,A)=0.2^2+2^2=0.04+4=4.04\).
Scaled distance to B. The standardized differences are \(-3{,}000/10{,}000=-0.3\) and \(0/0.5=0\). Hence, \(d^2_{\mathrm{scaled}}(P,B)=(-0.3)^2+0^2=0.09\). Since \(0.09<4.04\), scaling selects B instead. The conclusion changes because the claim-count difference, which was numerically small in raw units, becomes large relative to its stated variability.
Check and common mistake. The scaled differences to A are \(0.2\) and \(2\), so their squared contributions must be \(0.04\) and \(4\). Adding them gives \(4.04\), not \(8\). A frequent error is to square or add the differences incorrectly. The method avoids that error by showing each standardized difference before squaring it.
Worked Example: Rechecking a Large-Dollar and Small-Count Difference
For a hypothetical observation, the differences from a reference point are \(2{,}000\) dollars and \(1\) claim. Calculate the raw squared distance and the squared distance after standardizing by reference standard deviations of \(10{,}000\) dollars and \(0.5\) claims.
Method. Square and add the differences in their original units for the raw result. For the standardized result, first divide each difference by its own standard deviation, then square and add.
The raw squared distance is \(2{,}000^2+1^2=4{,}000{,}000+1=4{,}000{,}001\).
For the standardized distance, the dollar difference is \(2{,}000/10{,}000=0.2\), and the claim difference is \(1/0.5=2\). Thus the standardized squared distance is \(0.2^2+2^2=0.04+4=4.04\).
Check and common mistake. The standardized claim difference is \(2\), whose square is \(4\), not \(2\). The final value is \(4.04\). Do not divide the raw squared distance by one standard deviation, or scale the dollar and claim differences using the same denominator; each predictor has its own scale.
Worked Example: Centering Leaves Distances Unchanged
In a hypothetical training sample, annual premiums in thousands of dollars are \(20, 30, 40\), and claim counts are \(0, 1, 2\). Use the sample means and sample standard deviations to compare two records \(R=(40,2)\) and \(S=(30,0)\). Calculate their raw, centered, and standardized squared distances. Treat the premium values as thousands of dollars.
Method and reference values. The sample means are \(30\) for premium and \(1\) for claim count. For premiums, the deviations from the mean are \(-10,0,10\), so the sample variance is \((100+0+100)/(3-1)=100\), and the sample standard deviation is \(10\). For counts, the deviations are \(-1,0,1\), giving sample variance \((1+0+1)/2=1\) and sample standard deviation \(1\).
Raw distance. The record difference is \((40-30,2-0)=(10,2)\). Therefore the raw squared distance is \(10^2+2^2=100+4=104\).
Centered distance. Relative to the means, \(R\) is \((40-30,2-1)=(10,1)\), and \(S\) is \((30-30,0-1)=(0,-1)\). Their centered difference is \((10-0,1-(-1))=(10,2)\), so the centered squared distance is again \(10^2+2^2=104\). The center cancels from the pairwise difference.
Standardized distance. The standardized coordinates are \(R=(10/10,1/1)=(1,1)\) and \(S=(0/10,-1/1)=(0,-1)\). Their difference is \((1,2)\), giving squared distance \(1^2+2^2=5\).
Check and common mistake. The standardized squared distance is \(5\), not \(104\): standardization requires dividing each centered coordinate by its own standard deviation. Centering alone leaves the distance at \(104\). Confusing the two operations incorrectly suggests that subtracting means changes pairwise distances.
Choosing a Scaling Approach
Standardization is often useful when predictors use different units and the analysis should compare deviations relative to their typical spread. It is not automatically correct. If the business question says that a fixed difference in dollars should matter more than a fixed difference in claim counts, standardizing both predictors may remove an intended distinction. Conversely, leaving predictors in raw units can let a large numerical unit dominate even when that was not intended.
A standard deviation is not a neutral definition of importance. A predictor with small variability receives a larger weight after standardization because a given raw difference is large relative to that variability. This can be useful, but it can also amplify an unusual or unreliable predictor. A rare binary indicator, for example, may have a small standard deviation; standardizing it can make a difference between its two levels contribute substantially to distance. Consider whether that is appropriate for the analysis rather than applying a rule mechanically.
Centering can still be useful for interpretation or for methods that depend on a reference point, even though it does not change distances when applied uniformly. Standardization is generally inappropriate for a predictor with zero standard deviation: division by zero is undefined, and a constant predictor has no ability to distinguish observations. Such a predictor should not be included as a varying distance coordinate.
Use the same fitted centers and scales for all observations being compared. If the data are divided into training and assessment partitions as part of a broader predictive workflow, estimate the scaling parameters using the training partition and apply those same values to the assessment partition. Recomputing means or standard deviations on the assessment data changes the transformation and allows information from that partition to influence preprocessing.
Decide whether predictors should contribute in raw units, relative to their variation, or according to another justified weighting.
Confirm that the variables are numeric and that any constant predictor is removed. Use a method suited to categorical predictors rather than treating category labels as measured quantities.
Record the chosen centers and scales, transform every observation consistently, and reuse those parameters whenever new observations are compared.
Explain whether scaling gives equal weight to relative deviations and whether that weighting matches the purpose of the analysis.
This scaling decision is consequential because distance-based methods use the transformed coordinates to define similarity. A cluster assignment can change even when the underlying records do not, as the first example showed. When reporting results, state which predictors were scaled, how the centers and scales were obtained, and why that choice fits the intended comparison. This material is aligned with the SOA Exam PA (Predictive Analytics) syllabus.
Check Your Understanding
Work each problem by showing the predictor differences before squaring them.
- A hypothetical record differs from a reference by \(3{,}000\) dollars and \(0.5\) claims. The respective standard deviations are \(6{,}000\) dollars and \(0.25\) claims. Find the raw and standardized squared distances. (Answer: raw \(9{,}000{,}000.25\); standardized \(4.25\).)
- Two records have raw coordinates \(A=(80,4)\) and \(B=(100,7)\). Center the predictors at \(90\) and \(5\), respectively. Find the raw and centered squared distance, and state whether centering changes it. (Answer: both squared distances are \(409\); centering does not change it.)
- A point is \((8,2)\), and two candidate centers are \(A=(7,1)\) and \(B=(10,2)\). Raw-unit distances are used first. Then standardize using standard deviations \(2\) and \(0.5\). Which center is nearer under each calculation? (Answer: raw selects A, with squared distances \(2\) to A and \(4\) to B; standardized selects B, with squared distances \(4.25\) to A and \(1\) to B.)
- A value \(54\) has mean \(48\) and standard deviation \(3\). Another value \(3\) has mean \(2\) and standard deviation \(0.5\). Calculate the squared standardized distance between the two-value record and the reference record \((48,2)\). (Answer: standardized differences \(2\) and \(2\); squared distance \(8\).)
- Using training means \((60,3)\) and standard deviations \((6,1)\), compare records \(P=(72,5)\) and \(Q=(66,4)\). Find their standardized coordinates and squared distance. Also find the raw squared distance. (Answer: standardized coordinates \(P=(2,2)\), \(Q=(1,1)\); standardized squared distance \(2\); raw squared distance \(37\).)