Standardized Effect Sizes for Power Calculations

Why Standardize a Mean Departure?

A mean difference of 5 units may be large for one measurement and small for another. The scale alone does not tell us how far an alternative mean lies from a null mean relative to the natural variability of the observations. Dividing that departure by the population standard deviation gives a dimensionless quantity that supports meaningful comparisons and enters directly into power calculations.

This tutorial focuses on the standardized mean difference for a one-sample Z test with known population standard deviation. The same raw mean difference can correspond to different standardized effects when variability differs, and the same standardized effect can arise on very different measurement scales. A standardized effect is useful for comparing statistical separation, but it does not by itself establish whether a difference is important in practice.

Definition: For a null mean $\mu_0$, a specified alternative mean $\mu_1$, and a positive population standard deviation $\sigma$, the signed standardized mean difference is $$ \delta=\frac{\mu_1-\mu_0}{\sigma}. $$ It measures the alternative mean's departure from the null in population-standard-deviation units. Its sign gives the direction of the departure; its absolute value gives the size of the departure in those units.

The definition requires a specified alternative mean and a population standard deviation that is treated as known. It is not a formula for standardizing by the sample standard deviation. Tests with unknown variance use a different reference distribution and are outside the model used here. The standardized effect is also not a significance level or a probability: it describes a parameter departure, not the chance of an event.

How the Standardized Effect Enters Z-Test Power

Assume $X_1,\ldots,X_n$ are independent observations from a normal population with mean $\mu$ and known standard deviation $\sigma>0$. To test $H_0:\mu=\mu_0$, the one-sample Z statistic is

Z=\frac{\bar X-\mu_0}{\sigma/\sqrt{n}}.

For a two-sided test at significance level $\alpha$, reject $H_0$ when $|Z|>z_{1-\alpha/2}$, where $z_q$ is the $q$-quantile of the standard normal distribution. Let $\Phi$ denote the standard normal cumulative distribution function. Under a fixed alternative mean $\mu_1$, the statistic has variance 1 and mean $\sqrt n\,\delta$. That mean, $\sqrt n\,\delta$, is the noncentrality of the statistic under the alternative.

Standardize the sample mean.
Under the model, $\bar X$ has mean $\mu_1$ and standard deviation $\sigma/\sqrt n$ when the alternative is true.

Find the statistic's alternative mean.
Therefore, $E[Z]=(\mu_1-\mu_0)/(\sigma/\sqrt n)=\sqrt n(\mu_1-\mu_0)/\sigma=\sqrt n\,\delta$.

Calculate rejection probability.
Under the alternative, $Z\sim N(\sqrt n\,\delta,1)$. Power is the probability that this shifted normal statistic falls outside the null test's acceptance interval.

This derivation explains why the effect is divided by $\sigma$, and why sample size multiplies its contribution by $\sqrt n$. The mean departure is first measured in standard-deviation units, then scaled by the standard error of the sample mean. Thus, at a fixed significance level, the power of this test depends on $n$ and $\delta$, not on the outcome's units separately.

Theorem: For the two-sided one-sample Z test described above, define $c=z_{1-\alpha/2}$ and $\lambda=\sqrt n\,\delta$. Its power at the fixed alternative $\mu_1$ is $$ P_{\mu_1}(|Z|>c)=\Phi(-c-\lambda)+1-\Phi(c-\lambda). $$ Because the rejection region is symmetric, the power is unchanged when $\delta$ is replaced by $-\delta$. Equivalently, it can be calculated using $|\lambda|$.

The first term is the probability of rejecting in the lower tail, and the second is the probability of rejecting in the upper tail. For a one-sided upper-tail test, the formula is different: if the rejection rule is $Z>z_{1-\alpha}$, power is $1-\Phi(z_{1-\alpha}-\sqrt n\,\delta)$. In that setting, the sign of $\delta$ matters directly; replacing it by its absolute value would change the alternative.

Worked Examples

Worked Example: A Two-Sided Test with a Half-Standard-Deviation Effect

In a hypothetical calibration study, independent readings are normally distributed with known population standard deviation 8 units. A test examines $H_0:\mu=100$ against a two-sided alternative, using $n=16$ readings and significance level $\alpha=0.05$. Calculate the power when the true mean is 104 units.

Method and model. Use the known-variance one-sample Z test. The model is $X_i\sim N(\mu,8^2)$, independently, with $\mu_0=100$ and fixed alternative $\mu_1=104$. The critical value is $c=z_{0.975}\approx1.96$.

Step 1: Standardize the departure.

\delta=\frac{104-100}{8}=\frac{4}{8}=0.5.

The alternative mean is one-half of a population standard deviation above the null mean. The noncentrality is $\lambda=\sqrt{16}(0.5)=4(0.5)=2$.

Step 2: Add both rejection tails. Using $c=1.96$,

\begin{aligned} \text{Power} &=\Phi(-1.96-2)+1-\Phi(1.96-2)\\ &=\Phi(-3.96)+1-\Phi(-0.04)\\ &\approx 0.00003747+0.51595344\\ &=0.51599091\\ &\approx 0.5160. \end{aligned}

The power is approximately 0.5160, or 51.60%. The lower-tail probability is $3.747\times10^{-5}$; nearly all the power here comes from the upper tail.

Check. The alternative mean is above the null, so the upper-tail rejection probability should be substantial. The computed power is just above one-half, consistent with the statistic's alternative mean of 2 being close to the upper critical value 1.96.

Common mistake avoided. Do not omit the lower-tail probability merely because the alternative is above the null: the specified test is two-sided. Also, $\delta=0.5$, not 4; 4 is the raw mean departure in the measurement units.

Worked Example: Comparing Power Across Different Measurement Scales

Two hypothetical studies each use a two-sided one-sample Z test at $\alpha=0.05$ with $n=36$ independent observations and known population standard deviation. Study A measures a response score: its null mean is 50, its alternative mean is 56, and its standard deviation is 12 points. Study B measures a laboratory marker: its null mean is 2.0, its alternative mean is 2.30, and its standard deviation is 0.75 units. Compare their standardized effects and powers.

Method. Calculate each alternative's mean departure in standard-deviation units, then use the common sample size and significance level to compare power. In both studies, the observations are assumed normal with the stated known standard deviation.

Study A. The raw departure is $56-50=6$ points, so

\delta_A=\frac{6}{12}=0.5,\qquad \lambda_A=\sqrt{36}(0.5)=6(0.5)=3.

Study B. The raw departure is $2.30-2.0=0.30$ units, so

\delta_B=\frac{0.30}{0.75}=0.4,\qquad \lambda_B=\sqrt{36}(0.4)=6(0.4)=2.4.

For Study A, the two-sided power is

\begin{aligned} \text{Power}_A &=\Phi(-1.96-3)+1-\Phi(1.96-3)\\ &=\Phi(-4.96)+1-\Phi(-1.04)\\ &\approx 0.00000035+0.850830\\ &\approx 0.8508. \end{aligned}

For Study B, it is

\begin{aligned} \text{Power}_B &=\Phi(-1.96-2.4)+1-\Phi(1.96-2.4)\\ &=\Phi(-4.36)+1-\Phi(-0.44)\\ &\approx 0.0000065+0.670031\\ &\approx 0.6700. \end{aligned}

Study A has the larger standardized effect, 0.5 rather than 0.4, and the higher power. Its raw difference is also larger, but raw differences cannot in general be compared across these different units. The standardized effects make the relative separation clear.

Check. Both studies have the same $n$ and $\alpha$, so their power ordering should match the ordering of the absolute standardized effects. The results, 0.8508 versus 0.6700, satisfy that check.

Common mistake avoided. Comparing 6 points directly with 0.30 marker units would be meaningless. Divide each departure by its own population standard deviation before comparing.

Worked Example: Direction Matters for a One-Sided Test

A hypothetical process has normally distributed readings with known $\sigma=4$ units. A study tests $H_0:\mu=40$ against the upper-tail alternative $H_1:\mu>40$, using $n=25$ and $\alpha=0.05$. Calculate the power at each of two possible true means, 40.8 and 39.2.

Method and model. Use the upper-tail one-sample Z test with critical value $z_{0.95}\approx1.645$. The model is $X_i\sim N(\mu,4^2)$, independently. For this test, power is $1-\Phi(1.645-\sqrt n\,\delta)$.

Alternative mean 40.8. The signed effect and noncentrality are

\delta_+=\frac{40.8-40}{4}=\frac{0.8}{4}=0.2,\qquad \lambda_+=\sqrt{25}(0.2)=5(0.2)=1.

Thus,

\text{Power}_+=1-\Phi(1.645-1) =1-\Phi(0.645) \approx 1-0.7405 =0.2595.

Alternative mean 39.2. Here the departure points in the opposite direction:

\delta_-=\frac{39.2-40}{4}=\frac{-0.8}{4}=-0.2,\qquad \lambda_-=\sqrt{25}(-0.2)=-1.

Therefore,

\text{Power}_-=1-\Phi(1.645-(-1)) =1-\Phi(2.645) \approx 1-0.9959 =0.0041.

The two effects have the same absolute value, but the upper-tail test has very different power for them: approximately 0.2595 at 40.8 and 0.0041 at 39.2. This is expected because the second alternative shifts the statistic away from the rejection region.

Check and common mistake avoided. A one-sided test does not treat effects of equal magnitude in opposite directions alike. Using $|\delta|=0.2$ for both alternatives would incorrectly assign them the same power; retain the sign when the test has a directional rejection rule.

Interpreting Standardized Effects Correctly

A standardized mean difference is invariant to a positive change of measurement units. If every observation is converted by multiplying by a positive constant $b$, then both the mean departure and the standard deviation are multiplied by $b$, leaving their ratio unchanged. For example, converting a length from one unit to another does not change its standardized effect. Adding a fixed constant to all measurements also leaves the departure and standard deviation unchanged.

This invariance explains why standardized effects are useful when studies use different scales. It does not mean the effects measure practical importance universally. A difference of a fraction of a standard deviation may be consequential in one application and not in another; interpretation requires subject-matter context. Nor does $\delta$ alone determine power. For the two-sided Z test, the significance level and sample size also matter, through the critical value and $\sqrt n\,\delta$.

Use the population standard deviation. In the known-variance Z model, the denominator is $\sigma$, not the standard error $\sigma/\sqrt n$. The latter appears in the test statistic, not in the definition of the standardized effect.
Keep the sign when direction matters. A signed effect distinguishes alternatives above and below the null. Only the symmetric two-sided power calculation is unchanged by reversing that sign.
Do not equate effect size with power. The same $\delta$ can yield different power at different sample sizes or significance levels.
Check the variance assumption. The formulas here require a known population standard deviation and the stated one-sample Z-test model. If variance is estimated, the reference distribution and power calculation change.

Standardized mean differences provide a compact link between a parameter departure and test power. This treatment is aligned with the SOA VEE Mathematical Statistics syllabus.

Key takeaway: For a one-sample known-variance Z test, first compute $\delta=(\mu_1-\mu_0)/\sigma$, retaining its sign. Then combine it with sample size as $\sqrt n\,\delta$ to calculate power under the specified test. The standardized effect permits scale-aware comparisons, but direction, design assumptions, and practical context still matter.

Check Your Understanding

For each problem, use the stated known-variance normal model. Round probabilities to four decimal places where a numerical power is requested.

A two-sided test has $\mu_0=30$, known $\sigma=6$, $n=9$, and $\alpha=0.05$. Find the standardized effect and power at $\mu_1=33$. (Answer: $\delta=0.5000$, power $\approx0.3230$)
A sensor study has null mean 8.0, alternative mean 9.2, known $\sigma=2.4$, and $n=25$. For a two-sided test at $\alpha=0.05$, find $\delta$ and power. (Answer: $\delta=0.5000$, power $\approx0.7054$)
An upper-tail test has $\mu_0=100$, known $\sigma=5$, $n=16$, and $\alpha=0.05$. Find its power at $\mu_1=102.5$, using $z_{0.95}=1.645$. (Answer: $\delta=0.5000$, power $=1-\Phi(-0.355)\approx0.6387$)
Two-sided tests use $\alpha=0.05$ and the same standardized effect $\delta=0.4$. Compare their noncentralities and approximate powers for $n=25$ and $n=100$, using $c=1.96$. (Answer: $\lambda=2$, power $\approx0.5160$ for $n=25$; $\lambda=4$, power $\approx0.9793$ for $n=100$)
Study A has a mean departure of 3 units and known population standard deviation 12 units. Study B has a departure of 0.6 units and known population standard deviation 1.5 units. Which has the larger standardized effect, and what are the effects? (Answer: Study B; $\delta_A=3/12=0.2500$ and $\delta_B=0.6/1.5=0.4000$)

What You'll Learn