Standardizing Predictors With Z-Scores

Why Standardize a Predictor?

A numeric predictor may be measured in dollars, years, kilometres, or another unit. A z-score expresses an observation relative to the mean and standard deviation of a reference data set. It replaces the original units with a dimensionless measure of how far the observation lies from that reference mean.

For this tutorial, the reference data are the training set: the observations used to fit a predictive model. Calculate the mean and standard deviation from that set, then use those same values to standardize other observations. This gives the transformed predictor a consistent meaning across the training data and any later data to which the model is applied. This treatment is aligned with the SOA Exam PA (Predictive Analytics) syllabus.

Definition: For a numeric predictor with training observations $x_1,\ldots,x_n$, let $\bar{x}_{\text{train}}$ be their arithmetic mean and $s_{\text{train}}$ their sample standard deviation. The z-score for an observation $x$ is $$ z=\frac{x-\bar{x}_{\text{train}}}{s_{\text{train}}}. $$ The calculation requires $s_{\text{train}}>0$. A z-score is positive when $x$ is above the training mean, negative when it is below, and zero when it equals the mean.

The sample standard deviation used here is $s_{\text{train}}=\sqrt{\sum_{i=1}^{n}(x_i-\bar{x}_{\text{train}})^2/(n-1)}$. In R, scale() uses the sample standard deviation by default when it scales a numeric vector. Using that convention consistently matters: dividing by a population standard deviation instead can give different z-scores for the same finite training data.

What the Standardized Values Mean

The transformation subtracts the training mean and then divides by the training standard deviation. Subtraction changes the zero point: an observation equal to the training mean becomes zero. Division changes the unit: a difference of one training standard deviation becomes one standardized unit.

x=\bar{x}_{\text{train}}+z\,s_{\text{train}}

This reverse expression makes the original-scale interpretation explicit. A z-score of 0 corresponds to the training mean. A z-score of 1 corresponds to one training standard deviation above that mean; a z-score of $-1$ corresponds to one standard deviation below it. For example, if the training mean is 40 and the training standard deviation is 8, then $z=1$ represents an original value of $40+1(8)=48$, while $z=-1$ represents $40-8=32$.

A z-score is a relative distance, not a percentile or a probability. A z-score of 1 does not, by itself, imply that an observation is unusual or that a particular proportion of observations lies below it. Such probability interpretations require distributional assumptions, such as normality, which standardization does not establish.

Worked Example: Standardizing a Training Predictor

A hypothetical insurer records annual driving distance, in thousands of kilometres, for three training policies: 10, 20, and 30. Calculate each training z-score, then standardize a new policy with annual distance 35 thousand kilometres. State the original-scale meaning of $z=0$ and $z=1$.

Method. Calculate the training mean and sample standard deviation, then use those fixed values in the z-score formula. This avoids treating the new observation as if it changed the reference scale.

The training mean is $\bar{x}_{\text{train}}=(10+20+30)/3=60/3=20$ thousand kilometres. The deviations from the mean are $-10,0,10$, and their squared deviations are $100,0,100$. Therefore, $s_{\text{train}}=\sqrt{(100+0+100)/(3-1)}=\sqrt{100}=10$ thousand kilometres.

The standardized training values are $z_1=(10-20)/10=-1$, $z_2=(20-20)/10=0$, and $z_3=(30-20)/10=1$. Thus, the three training observations are one standard deviation below the mean, at the mean, and one standard deviation above the mean.

For the new policy, $z_{\text{new}}=(35-20)/10=15/10=1.5$. Its annual distance is 1.5 training standard deviations above the training mean. In original units, $z=0$ means 20 thousand kilometres, and $z=1$ means $20+10=30$ thousand kilometres.

Check. Reversing the transformation gives $20+1.5(10)=35$ thousand kilometres, the stated new value. A common mistake is to recompute the mean and standard deviation after adding the new policy. That would change the reference scale rather than apply the scale defined by the training data.

Why the Training Z-Scores Have Mean Zero and Standard Deviation One

The properties of the transformed training values follow directly from the formula. Let $z_i=(x_i-\bar{x}_{\text{train}})/s_{\text{train}}$, for $i=1,\ldots,n$, with $s_{\text{train}}>0$. Their arithmetic mean is

\bar{z} =\frac{1}{n}\sum_{i=1}^{n}z_i =\frac{1}{n s_{\text{train}}}\sum_{i=1}^{n}(x_i-\bar{x}_{\text{train}}) =\frac{1}{n s_{\text{train}}} \left(\sum_{i=1}^{n}x_i-n\bar{x}_{\text{train}}\right) =0.

The last equality holds because $\sum_{i=1}^{n}x_i=n\bar{x}_{\text{train}}$, by the definition of the arithmetic mean. For the sample variance of the standardized training values,

s_z^2 =\frac{1}{n-1}\sum_{i=1}^{n}(z_i-\bar{z})^2 =\frac{1}{n-1}\sum_{i=1}^{n} \left(\frac{x_i-\bar{x}_{\text{train}}}{s_{\text{train}}}\right)^2 =\frac{1}{s_{\text{train}}^2} \frac{\sum_{i=1}^{n}(x_i-\bar{x}_{\text{train}})^2}{n-1} =1.

Here, $\bar{z}=0$, and the final fraction is $s_{\text{train}}^2$ by the definition of the training sample variance. Since a standard deviation is the nonnegative square root of variance, the sample standard deviation of the standardized training values is 1.

Result: When a nonconstant numeric predictor is standardized using its training-set mean and sample standard deviation, the standardized training values have arithmetic mean 0 and sample standard deviation 1. This result applies to the training values used to calculate those quantities; it does not require the predictor to be normally distributed.

Calculating and Applying Z-Scores in R

For a numeric vector, R’s scale() function centers by the vector mean and, by default, divides by its sample standard deviation. Because scale() returns a matrix, as.numeric() is useful when a plain numeric vector is wanted.

The following code standardizes three training observations:

R: x.train <- c(100, 120, 140)
R: z.train <- as.numeric(scale(x.train))

The training mean is 120, and the sample standard deviation is $\sqrt{(400+0+400)/2}=\sqrt{400}=20$. The resulting standardized values are $-1,0,1$. To standardize new values using that same training scale, retain the center and scale attributes of the scaled training vector:

R: scaled.train <- scale(x.train)
R: center.train <- attr(scaled.train, "scaled:center")
R: sd.train <- attr(scaled.train, "scaled:scale")
R: x.new <- c(80, 130)
R: z.new <- (x.new - center.train) / sd.train

Here, center.train is 120 and sd.train is 20. Thus, the new values 80 and 130 become $(80-120)/20=-2$ and $(130-120)/20=0.5$, respectively. A new observation is not automatically part of the training distribution: the training mean and standard deviation describe the reference scale, not necessarily the mean and standard deviation of any new batch of observations.

Worked Example: Interpreting a Standardized Value in Original Units

A hypothetical health-insurance model uses a training predictor measuring annual outpatient visits per member. The three training values are 4, 8, and 12 visits. A member has a standardized value of 1.5. Find the training mean, the sample standard deviation, the member’s original visit count, and the original values represented by $z=0$ and $z=1$.

Method. First establish the training scale. Then use the inverse transformation $x=\bar{x}_{\text{train}}+z s_{\text{train}}$. This avoids interpreting a standardized value as though it were measured in visits.

The training mean is $(4+8+12)/3=24/3=8$ visits. The deviations are $-4,0,4$, so the sum of squared deviations is $16+0+16=32$. The sample standard deviation is $s_{\text{train}}=\sqrt{32/(3-1)}=\sqrt{16}=4$ visits.

For $z=1.5$, the original count is $x=8+1.5(4)=8+6=14$ visits. At $z=0$, the original value is 8 visits, the training mean. At $z=1$, it is $8+4=12$ visits, one training standard deviation above the mean.

Check. Standardizing 14 gives $(14-8)/4=6/4=1.5$, as required. A common mistake is to report 1.5 as 1.5 visits. It is 1.5 standard deviations; the inverse calculation is needed to obtain the original unit.

Worked Example: Using R’s Sample Standard Deviation

A hypothetical pricing team has three training observations for a numeric predictor: 5, 5, and 11. Calculate the mean, the sample standard deviation, and the standardized values. Then calculate the z-score for a new observation of 8. Show how using the population standard deviation instead would change the result.

Method. Use the sample standard deviation, matching R’s default scaling convention. Compare it with the population-standard-deviation calculation only to identify the denominator trap.

The mean is $(5+5+11)/3=21/3=7$. The deviations are $-2,-2,4$, and the squared deviations are $4,4,16$, with total 24. The sample variance is $24/(3-1)=12$, so $s_{\text{train}}=\sqrt{12}$, approximately 3.4641.

The training z-scores are $(5-7)/\sqrt{12}$, $(5-7)/\sqrt{12}$, and $(11-7)/\sqrt{12}$. Since $\sqrt{12}=2\sqrt{3}$, these are approximately $-0.5774,-0.5774,$ and $1.1547$. For the new observation, $z_{\text{new}}=(8-7)/\sqrt{12}=1/\sqrt{12}$, approximately 0.2887.

The incorrect population calculation would divide the squared deviations by $n=3$, giving a population standard deviation of $\sqrt{24/3}=\sqrt{8}$, approximately 2.8284. It would produce a new-observation score of $1/\sqrt{8}$, approximately 0.3536, not 0.2887. The sample standard deviation is the appropriate choice here because it is the stated convention and the one used by R’s default scale().

Check. The sample z-scores have mean zero: $(-0.5774-0.5774+1.1547)/3$ is approximately zero, allowing for rounding. Applying the population denominator when the intended procedure is sample standardization changes the scale and therefore every z-score.

Assumptions, Scope, and Common Traps

Z-score standardization is a descriptive linear transformation. It requires numeric values and a positive training standard deviation, but it does not require normality, symmetry, or any particular predictive model. It does not remove skewness or make extreme observations less extreme. If the training predictor is constant, then $s_{\text{train}}=0$, and the formula divides by zero; no z-scores are defined by this calculation.

Keep the reference scale and units clear. If a predictor is recorded in dollars rather than thousands of dollars, both its mean and standard deviation are multiplied by 1,000, so the corresponding z-scores remain unchanged. The units cancel because the numerator and denominator are both measured in the same units. The numerical z-score is therefore dimensionless, while its interpretation in original units still depends on the training mean and standard deviation.

Using the wrong denominator. R’s default scale() uses the sample standard deviation, with denominator $n-1$. Using a population standard deviation instead will generally give different scores.
Misreading the value. A z-score of 1.5 means 1.5 standard deviations above the training mean, not 1.5 original units above it.
Assuming normality. A mean of zero and sample standard deviation of one do not imply a normal distribution or allow a percentile to be read from a normal table.
Changing the reference scale. When standardizing new observations, use the training mean and standard deviation that define the model’s scale; do not silently replace them with statistics from the new observations.

Key takeaway: Calculate a predictor’s training-set mean and sample standard deviation, transform each value as $z=(x-\bar{x}_{\text{train}})/s_{\text{train}}$, and interpret the result as a signed distance measured in training standard deviations. The inverse expression $x=\bar{x}_{\text{train}}+z s_{\text{train}}$ translates that distance back to the original units.

Check Your Understanding

For each problem, use the sample standard deviation unless the question says otherwise. The answers are provided for checking.

A training predictor has values 30, 40, and 50 dollars. Find its training mean and sample standard deviation, then standardize a new value of 60 dollars. (Answer: mean = $40; sample standard deviation = $10; new z-score = 2.)
A training variable has values 2, 4, 6, and 8. Calculate the sample standard deviation and the z-score for a new value of 3. (Answer: sample standard deviation = $\sqrt{20/3}$, approximately 2.5820; z-score = $-2/\sqrt{20/3}$, approximately $-0.7746$.)
A nonconstant training predictor has mean 18 and sample standard deviation 6. What original value corresponds to $z=-1.5$, and how far is it from the training mean? (Answer: 9 original units; 9 units below the training mean.)
A training vector is 12, 18, and 24, and a new observation is 30. Calculate the training mean and sample standard deviation, then standardize the new value using the training quantities. (Answer: mean = 18; sample standard deviation = 6; new z-score = 2.)
A predictor is measured for three training observations as 1, 2, and 3 metres. The same observations are recorded as 100, 200, and 300 centimetres. Calculate the z-scores in each unit system and explain the relationship. (Answer: in either system the z-scores are $-1,0,1$; changing from metres to centimetres multiplies both the mean and sample standard deviation by 100, so the standardized values are unchanged.)

What You'll Learn