Power for Pooled Two-Sample T Tests

Why Power for a Pooled t Test Uses a Noncentral Distribution

A pooled two-sample t test is used to compare the means of two independent normal populations when their variances are unknown but assumed equal. Unlike a two-sample z test, the standard error is estimated from the samples. That estimate is random, so under a specified alternative the test statistic does not simply have a normal distribution shifted by a fixed amount. Its exact distribution is a noncentral t distribution.

Power is the probability that a test rejects its null hypothesis when a particular alternative is true. To calculate it, specify the population difference at which power is desired, along with the common population standard deviation, the two sample sizes, the test direction, and the significance level. The resulting calculation is exact under the model assumptions below.

Definition: Suppose $X_{11},\ldots,X_{1n_1}$ are independent observations from $N(\mu_1,\sigma^2)$, and $X_{21},\ldots,X_{2n_2}$ are independent observations from $N(\mu_2,\sigma^2)$. Assume the two samples are independent and share the same unknown variance $\sigma^2$. The pooled variance estimate and pooled two-sample t statistic for testing $H_0:\mu_1-\mu_2=0$ are $$ S_p^2=\frac{(n_1-1)S_1^2+(n_2-1)S_2^2}{n_1+n_2-2}, \qquad T=\frac{(\bar X_1-\bar X_2)-0}{S_p\sqrt{1/n_1+1/n_2}}. $$ The degrees of freedom are $\nu=n_1+n_2-2$. Under a true difference $\Delta=\mu_1-\mu_2$, $T$ has a noncentral t distribution with degrees of freedom $\nu$ and noncentrality parameter $$ \lambda=\frac{\Delta}{\sigma\sqrt{1/n_1+1/n_2}}. $$

Write $F_{\nu,\lambda}(x)$ for the cumulative distribution function of a noncentral t random variable with degrees of freedom $\nu$ and noncentrality parameter $\lambda$. The value of $\lambda$ is signed: a positive planned difference shifts the statistic toward the upper tail, while a negative planned difference shifts it toward the lower tail. For a two-sided test, power includes the rejection probability from both tails.

Rejection Regions and Power Formulas

Let $t_{p,\nu}$ denote the $p$-quantile of the central t distribution with $\nu$ degrees of freedom. The significance level determines the cutoff or cutoffs under the null hypothesis. To calculate power at a particular alternative, keep those null-calibrated cutoffs and evaluate the rejection probability using $F_{\nu,\lambda}$, not the central t distribution.

Test alternative	Rejection region	Power at difference $\Delta$
$H_a:\mu_1-\mu_2\ne 0$, size $\alpha$	$T<-t_{1-\alpha/2,\nu}$ or $T>t_{1-\alpha/2,\nu}$	$F_{\nu,\lambda}(-t_{1-\alpha/2,\nu})+1-F_{\nu,\lambda}(t_{1-\alpha/2,\nu})$
$H_a:\mu_1-\mu_2>0$, size $\alpha$	$T>t_{1-\alpha,\nu}$	$1-F_{\nu,\lambda}(t_{1-\alpha,\nu})$
$H_a:\mu_1-\mu_2<0$, size $\alpha$	$T<-t_{1-\alpha,\nu}$	$F_{\nu,\lambda}(-t_{1-\alpha,\nu})$

These expressions are probabilities under the specified alternative, not under the null. A noncentral t CDF can be evaluated with statistical software or a suitable table. Reporting the CDF expression precisely identifies the exact power even when a numerical evaluation is not available.

Derivation: Under the stated normal model, $(\bar X_1-\bar X_2-\Delta)/[\sigma\sqrt{1/n_1+1/n_2}]$ has a standard normal distribution. Also, $\nu S_p^2/\sigma^2$ has a chi-square distribution with $\nu$ degrees of freedom and is independent of the difference of sample means. Thus the test statistic can be written as $$ T=\frac{Z+\lambda}{\sqrt{V/\nu}}, $$ where $Z\sim N(0,1)$, $V\sim\chi^2_\nu$, $Z$ and $V$ are independent, and $\lambda=\Delta/[\sigma\sqrt{1/n_1+1/n_2}]$. This is the defining representation of a noncentral t distribution. When $\Delta=0$, $\lambda=0$, giving the central t distribution used to set the test's cutoffs.

Worked Power Calculations

Worked Example: Two-Sided Test With a Positive Planning Difference

A hypothetical laboratory compares mean calibration readings from two independent processes. Assume normal readings with a common population standard deviation of $\sigma=2$ units. The planned sample sizes are $n_1=8$ and $n_2=8$, and the planning means are $\mu_1=52$ and $\mu_2=50$. Find the exact power expression for a two-sided pooled t test at $\alpha=0.05$.

Method. Calculate the degrees of freedom and the noncentrality parameter, find the two-sided central-t cutoff, and then add the two noncentral-tail probabilities.

The planning difference, using the test's stated order, is $\Delta=\mu_1-\mu_2=52-50=2$. The degrees of freedom are $$ \nu=n_1+n_2-2=8+8-2=14. $$ The standard-error factor using the true common standard deviation is $$ \sigma\sqrt{1/n_1+1/n_2} =2\sqrt{1/8+1/8} =2\sqrt{2/8} =2\sqrt{1/4} =1. $$ Therefore $\lambda=2/1=2$. For a two-sided test at level $0.05$, each tail has probability $0.025$ under the null, and the critical value is $t_{0.975,14}$, approximately $2.1448$. Hence the power is $$ F_{14,2}(-2.1448)+1-F_{14,2}(2.1448). $$ This is the exact power written in terms of the noncentral t CDF. The calculation uses both tails because the alternative is two-sided.

Check. The degrees of freedom are $14$, the positive mean difference gives a positive $\lambda$, and both rejection tails appear in the result. A common mistake is to use $t_{0.95,14}$ as the two-sided cutoff; that would put $0.05$, rather than $0.025$, in each null tail.

Worked Example: Lower-Tailed Test and a Negative Difference

A hypothetical quality study compares independent normal measurements from two production settings. The common standard deviation is $\sigma=5$ units, with $n_1=9$ and $n_2=16$. The planned means are $\mu_1=97.9167$ and $\mu_2=100$, so use the exact planning difference $\Delta=-25/12$ units. Find the power expression for testing $H_a:\mu_1-\mu_2<0$ at $\alpha=0.05$.

Method. Preserve the order $\mu_1-\mu_2$ when calculating the signed noncentrality parameter, then use the lower-tail rejection probability.

The degrees of freedom are $$ \nu=9+16-2=23. $$ The standard-error factor under the planned common standard deviation is $$ 5\sqrt{1/9+1/16} =5\sqrt{16/144+9/144} =5\sqrt{25/144} =5(5/12) =25/12. $$ Consequently, $$ \lambda=\frac{-25/12}{25/12}=-1. $$ The lower-tailed test rejects when $T<-t_{0.95,23}$. The critical value is approximately $1.7139$, so the rejection boundary is $-1.7139$. The exact power is therefore $$ F_{23,-1}(-1.7139). $$

Check. The planned difference is negative and $\lambda=-1$, consistent with an alternative that shifts the statistic toward the lower tail. A common mistake is to report $1-F_{23,-1}(-1.7139)$; that is the probability of not being in the lower rejection tail, not the power of this test.

Worked Example: Choosing Both Tails at a Different Significance Level

A hypothetical assessment compares average completion times from two independent normal groups. Assume a common standard deviation of $\sigma=4$ minutes, $n_1=n_2=10$, and a planning difference $\mu_1-\mu_2=2$ minutes. Find the power expression for a two-sided pooled t test at $\alpha=0.10$.

Method. Use the two-sided cutoff with $\alpha/2$ in each tail, even though the overall test size is $0.10$.

The degrees of freedom are $$ \nu=10+10-2=18. $$ The standard-error factor is $$ 4\sqrt{1/10+1/10} =4\sqrt{2/10} =4\sqrt{1/5} =\frac{4}{\sqrt{5}}. $$ The noncentrality parameter is $$ \lambda=\frac{2}{4/\sqrt{5}} =\frac{\sqrt{5}}{2} \approx 1.1180. $$ For a two-sided test of size $0.10$, the cutoff is $t_{0.95,18}$, approximately $1.7341$. The exact power is $$ F_{18,1.1180}(-1.7341)+1-F_{18,1.1180}(1.7341). $$

Check. The cutoff uses the $0.95$ quantile because $0.10/2=0.05$ lies in each tail. The positive $\lambda$ favors upper-tail rejection, but the lower-tail rejection probability remains part of the two-sided power.

How to Avoid the Main Power-Calculation Traps

The central t distribution sets the critical values; the noncentral t distribution calculates rejection probability at the alternative. Mixing those roles is a frequent source of errors. Before evaluating any CDF, write down the rejection region and check that it matches the verbal alternative.

Keep the difference order fixed. If the statistic uses $\bar X_1-\bar X_2$, calculate $\Delta=\mu_1-\mu_2$. Reversing the order changes the sign of $\lambda$.
Use the correct tail. An upper-tailed test uses $1-F_{\nu,\lambda}(c)$; a lower-tailed test uses $F_{\nu,\lambda}(-c)$. A two-sided test adds both tail probabilities.
Do not split a one-sided significance level. Its cutoff is $t_{1-\alpha,\nu}$. Only a two-sided test splits $\alpha$ into $\alpha/2$ per tail.
Do not substitute the estimated pooled standard deviation into the planning parameter. The exact noncentrality parameter uses the assumed true common $\sigma$; randomness in $S_p$ is already represented by the noncentral t distribution.
Check the model. The exact pooled-test result requires independent samples, normal populations, and equal population variances. With unequal variances, this pooled noncentral t calculation is not exact.

For a two-sided test, changing the planned difference from $\Delta$ to $-\Delta$ changes $\lambda$ to $-\lambda$ but leaves the total power unchanged: the two rejection tails exchange roles. For a one-sided test, changing the sign can sharply reduce power because the effect shifts the statistic away from the rejection tail. Thus the sign is not a cosmetic detail, even when a two-sided power calculation is ultimately symmetric.

Key takeaway: Under independent normal samples with a common variance, calculate $\nu=n_1+n_2-2$ and $\lambda=(\mu_1-\mu_2)/[\sigma\sqrt{1/n_1+1/n_2}]$. Keep the central-t cutoff fixed by the test size, then evaluate the correct rejection-tail probability using the noncentral t CDF.

Check Your Understanding

For each problem, determine the degrees of freedom and noncentrality parameter, then write the power using the specified noncentral t CDF.

A two-sided pooled test uses $n_1=n_2=6$, common standard deviation $\sigma=3$, planned means $31.5$ and $30$, and $\alpha=0.05$. Find the power expression. (Answer: $\nu=10$, $\lambda=(1.5)/(3\sqrt{1/6+1/6})=\sqrt{3}/2\approx0.8660$, and power $F_{10,0.8660}(-t_{0.975,10})+1-F_{10,0.8660}(t_{0.975,10})$.)
A lower-tailed test has $n_1=n_2=8$, $\sigma=2$, and planned difference $\mu_1-\mu_2=-1$. At $\alpha=0.10$, give the power expression. (Answer: $\nu=14$, $\lambda=-1/[2\sqrt{1/8+1/8}]=-1$, and power $F_{14,-1}(-t_{0.90,14})$.)
An upper-tailed test uses $n_1=n_2=12$, $\sigma=6$, and a planned difference of $3$. At $\alpha=0.01$, find its power expression. (Answer: $\nu=22$, $\lambda=3/[6\sqrt{1/12+1/12}]=3/\sqrt{6}=\sqrt{3/2}\approx1.2247$, and power $1-F_{22,1.2247}(t_{0.99,22})$.)
A two-sided test uses $n_1=n_2=10$, $\sigma=1$, and planned difference $\mu_1-\mu_2=-0.5$, at $\alpha=0.05$. Give $\nu$, $\lambda$, and the power expression. (Answer: $\nu=18$, $\lambda=-0.5/\sqrt{1/10+1/10}=-\sqrt{5/8}\approx-0.7906$, and power $F_{18,-0.7906}(-t_{0.975,18})+1-F_{18,-0.7906}(t_{0.975,18})$.)
For the preceding problem, suppose the alternative were instead $H_a:\mu_1-\mu_2<0$, still at $\alpha=0.05$. What is the power expression? (Answer: $\nu=18$, $\lambda=-\sqrt{5/8}\approx-0.7906$, and power $F_{18,-0.7906}(-t_{0.95,18})$.)

Test alternative	Rejection region	Power at difference \(\Delta\)
\(H_a:\mu_1-\mu_2\ne 0\), size \(\alpha\)	\(T<-t_{1-\alpha/2,\nu}\) or \(T>t_{1-\alpha/2,\nu}\)	\(F_{\nu,\lambda}(-t_{1-\alpha/2,\nu})+1-F_{\nu,\lambda}(t_{1-\alpha/2,\nu})\)
\(H_a:\mu_1-\mu_2>0\), size \(\alpha\)	\(T>t_{1-\alpha,\nu}\)	\(1-F_{\nu,\lambda}(t_{1-\alpha,\nu})\)
\(H_a:\mu_1-\mu_2<0\), size \(\alpha\)	\(T<-t_{1-\alpha,\nu}\)	\(F_{\nu,\lambda}(-t_{1-\alpha,\nu})\)

What You'll Learn