Deriving the Z-Score Transformation

Why Standardize a Normal Variable?

Normal variables can describe quantities measured on very different scales. One model might describe a measurement in milligrams per litre, while another describes time in seconds. Standardization converts a normal variable to a common scale: its transformed mean is zero and its transformed standard deviation is one. The transformation preserves the relative position of values within the model, while expressing them in standard-deviation units.

The key is to transform the random variable, not just an observed number. If $X$ is normally distributed with mean $\mu$ and standard deviation $\sigma$, define $Z$ by subtracting $\mu$ from $X$, then dividing by $\sigma$. The derivation below checks the mean and spread algebraically, and also verifies that the transformed variable has the standard normal density.

Definition: If $X\sim N(\mu,\sigma^2)$, where $\sigma>0$, the standardized variable is $$ Z=\frac{X-\mu}{\sigma}. $$ The quantity $\sigma$ is the standard deviation, so $\sigma^2$ is the variance. Standardizing subtracts the model mean and divides by the model standard deviation.

The condition $\sigma>0$ matters: division by zero is undefined, and a normal distribution with positive spread has a positive standard deviation. The notation $X\sim N(\mu,\sigma^2)$ puts the variance, not the standard deviation, in the second position. Thus, for example, $X\sim N(40,36)$ has mean $40$, variance $36$, and standard deviation $6$.

Deriving the Mean and Standard Deviation

First consider the center of $Z$. Expectation is linear: a constant can be taken outside an expectation, and the expectation of a constant is that constant. Since $E[X]=\mu$, these facts give

E[Z] =E\left[\frac{X-\mu}{\sigma}\right] =\frac{E[X]-\mu}{\sigma} =\frac{\mu-\mu}{\sigma} =0.

So the transformed variable has mean zero. To find its spread, use the definition of variance as the expected squared distance from the mean. Since $E[Z]=0$,

\begin{aligned} \operatorname{Var}(Z) &=E[(Z-E[Z])^2]\\ &=E[Z^2]\\ &=E\left[\left(\frac{X-\mu}{\sigma}\right)^2\right]\\ &=\frac{E[(X-\mu)^2]}{\sigma^2}\\ &=\frac{\operatorname{Var}(X)}{\sigma^2}\\ &=\frac{\sigma^2}{\sigma^2}=1. \end{aligned}

The standard deviation is the positive square root of the variance. Therefore, $Z$ has standard deviation $\sqrt{1}=1$. Subtracting the mean centers the variable; dividing by the original standard deviation sets its variance to one.

Result: For $X\sim N(\mu,\sigma^2)$ with $\sigma>0$, the variable $Z=(X-\mu)/\sigma$ has mean $0$ and standard deviation $1$. These two calculations establish its center and spread. Because $X$ is normal, its density can also be transformed directly to show that $Z\sim N(0,1)$.

Why the Transformed Variable Is Standard Normal

Having mean zero and standard deviation one does not, by itself, establish that a variable is normally distributed. A different kind of variable could have the same mean and standard deviation. To verify the full distribution here, transform the density of $X$.

For $X\sim N(\mu,\sigma^2)$, its density at $x$ is

f_X(x)=\frac{1}{\sigma\sqrt{2\pi}} \exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right).

From $Z=(X-\mu)/\sigma$, solve for $X$: $x=\mu+\sigma z$. As $z$ changes by a small amount, the corresponding change in $x$ is scaled by $dx/dz=\sigma$. The density transformation therefore multiplies the original density by $\sigma$:

\begin{aligned} f_Z(z) &=f_X(\mu+\sigma z)\left|\frac{dx}{dz}\right|\\ &=\frac{1}{\sigma\sqrt{2\pi}} \exp\left(-\frac{((\mu+\sigma z)-\mu)^2}{2\sigma^2}\right)\sigma\\ &=\frac{1}{\sqrt{2\pi}}\exp\left(-\frac{z^2}{2}\right). \end{aligned}

The final expression is the standard normal density, the density of $N(0,1)$. Thus standardizing a normal variable yields a standard normal variable—not merely a variable with matching mean and standard deviation. The factor $\sigma$ in the density transformation cancels the $1/\sigma$ in the original density, leaving the same density for every normal model after standardization.

Worked Example: Standardizing a Measurement

Suppose a hypothetical laboratory measurement $X$, in milligrams per litre, follows $N(100,15^2)$. Find the standardized value corresponding to a measurement of $130$ milligrams per litre.

Here the model mean is $\mu=100$ milligrams per litre and the model standard deviation is $\sigma=15$ milligrams per litre. Substituting the observed value $x=130$ into the standardization formula gives

z=\frac{x-\mu}{\sigma} =\frac{130-100}{15} =\frac{30}{15} =2.

The subtraction gives $30$ milligrams per litre, and division by $15$ milligrams per litre leaves the dimensionless value $2$. The calculation applies the transformation to one observed value; the corresponding random variable $Z=(X-100)/15$ has distribution $N(0,1)$.

Worked Examples With Different Parameters

The same formula applies whether the original variable is measured in time, mass, or another unit. In each case, use the standard deviation in the denominator and the mean in the subtraction. The parameters belong to the specified model; they are not automatically the mean and standard deviation of a particular sample.

Worked Example: A Value Below the Model Mean

Let the completion time $X$ for a hypothetical process follow $N(6.4,0.8^2)$, with time measured in minutes. Standardize a completion time of $5.2$ minutes.

The model has $\mu=6.4$ minutes and $\sigma=0.8$ minute. Therefore,

z=\frac{x-\mu}{\sigma} =\frac{5.2-6.4}{0.8} =\frac{-1.2}{0.8} =-1.5.

The negative result follows from $5.2<6.4$: subtracting the model mean produces a negative numerator, and the standard deviation is positive. As a check on the scale, the time differs from the mean by $1.2$ minutes, which is $1.2/0.8=1.5$ standard deviations.

Worked Example: Reading the Variance Correctly

Suppose a hypothetical measurement $X$ follows $N(40,36)$, and consider the value $49$. Find its standardized value.

In this notation, $36$ is the variance. The standard deviation is its positive square root:

\sigma=\sqrt{36}=6.

Now substitute $x=49$, $\mu=40$, and $\sigma=6$:

z=\frac{x-\mu}{\sigma} =\frac{49-40}{6} =\frac{9}{6} =1.5.

The difference between the value and the mean is $9$ units, and the model standard deviation is $6$ units, so the standardized value is $1.5$. Dividing by the variance instead would give $9/36=0.25$, which is not the standardization formula and would leave the units inconsistent: units divided by squared units is not dimensionless.

What the Algebra Does—and Does Not—Change

Standardization changes the coordinate system used to describe values. Subtracting $\mu$ shifts the center of the distribution to zero. Dividing by the positive quantity $\sigma$ rescales distances so that the standard deviation becomes one. The density derivation confirms that the entire transformed distribution is standard normal when the original model is normal.

It does not change which observations are larger than others. If $x_1<x_2$, then, because $\sigma>0$,

\frac{x_1-\mu}{\sigma}<\frac{x_2-\mu}{\sigma}.

Thus the transformation preserves order. It also removes the original measurement units: both the numerator and denominator are expressed in the same units, so their ratio has no units. Detailed interpretations of standardized values and their use in finding normal probabilities are separate tasks; the algebra here establishes the transformation and its distribution.

Start with the model.
Identify $X\sim N(\mu,\sigma^2)$, and note that $\sigma=\sqrt{\sigma^2}>0$.

Center the variable.
Subtract $\mu$, giving $X-\mu$, whose expectation is $E[X]-\mu=0$.

Rescale the variable.
Divide by $\sigma$, giving $Z=(X-\mu)/\sigma$, with variance $\operatorname{Var}(X)/\sigma^2=1$.

Verify the distribution.
Transform the normal density to obtain $f_Z(z)=(1/\sqrt{2\pi})\exp(-z^2/2)$, the standard normal density.

Common Mistakes

Dividing by the variance. The formula uses $\sigma$, the standard deviation, not $\sigma^2$. If a model is written $N(\mu,\sigma^2)$, take the square root of its second parameter before standardizing.
Subtracting the value from the mean. The numerator is $x-\mu$, not $\mu-x$. Reversing the order reverses the sign.
Treating the result as a measurement in the original units. The standardized value is a ratio, so its units cancel.
Assuming mean zero and standard deviation one alone prove normality. Those two summaries do not determine a distribution. The conclusion $Z\sim N(0,1)$ here also relies on the assumption that $X$ is normally distributed, as the density calculation shows.
Using sample summaries without checking the model. The definition uses the mean and standard deviation of the specified normal model. A sample mean and sample standard deviation are different quantities and should not be substituted without a separate reason.

Key takeaway: For $X\sim N(\mu,\sigma^2)$ with $\sigma>0$, standardization gives $Z=(X-\mu)/\sigma$. Algebra shows $E[Z]=0$ and $\operatorname{SD}(Z)=1$, while transforming the density establishes $Z\sim N(0,1)$.

Check Your Understanding

Use the model parameters, not just the observed value, in each standardization.

If $X\sim N(20,4^2)$, what is the standard deviation, and what standardized value corresponds to $x=26$?
For $X\sim N(8,9)$, explain why the standard deviation is $3$, not $9$.
Show algebraically that $E[(X-\mu)/\sigma]=0$ when $E[X]=\mu$.
Why does a negative standardized value occur when the observed value is below the model mean?
Why does finding mean zero and standard deviation one not by itself prove that a variable is standard normal?