The Central Limit Theorem for Standardized Sums

Why Centering and Scaling Matter

For independent observations with the same distribution, a sum usually grows in size with the number of terms. Its fluctuations around its mean also grow, but on a different scale. The central limit theorem identifies that fluctuation scale and the distribution seen after centering and scaling. The resulting limit is useful not because every finite sum is exactly normal, but because the normalized sums converge in distribution to a normal random variable.

Work on a probability space $(\Omega,\mathcal{F},\mathbb{P})$. Let $(X_k)_{k\ge1}$ be independent and identically distributed real-valued random variables, with finite mean $\mu=\mathbb{E}[X_1]$ and finite, strictly positive variance $\sigma^2=\operatorname{Var}(X_1)$. Define the partial sum $S_n=\sum_{k=1}^n X_k$ and the sample mean $\bar X_n=S_n/n$. The condition $\sigma^2>0$ ensures that the standardizing denominator is nonzero.

Theorem (iid central limit theorem, or Lindeberg–Lévy theorem): Under the assumptions above, $$ \frac{S_n-n\mu}{\sigma\sqrt{n}}\ \xrightarrow{\ d\ }\ N(0,1). $$ Here $\xrightarrow{\ d\ }$ denotes convergence in distribution, and $N(0,1)$ is the normal distribution with mean $0$ and variance $1$. Equivalently, at every real $x$, the distribution functions of the standardized sums converge to $\Phi(x)$, where $\Phi$ is the standard normal distribution function. This classical theorem is a standard result; its proof, which uses distributional limit arguments beyond the algebraic derivations here, is not attempted in this tutorial.

The theorem’s assumptions matter: the variables must share a distribution and be independent, and their common distribution must have finite positive variance. The result describes convergence in distribution, not an assertion that $S_n$ is exactly normal for finite $n$, nor that the standardized sums converge almost surely or in probability to a particular random variable.

From Sums to Standardized Sums and Means

Corollary (standardized sum and sample mean forms): Under the hypotheses of the iid central limit theorem, $$ Z_n:=\frac{S_n-n\mu}{\sigma\sqrt{n}} =\frac{\sqrt{n}(\bar X_n-\mu)}{\sigma} \ \xrightarrow{\ d\ }\ N(0,1). $$

Proof. By definition, $\bar X_n=S_n/n$, so $\sqrt{n}(\bar X_n-\mu)/\sigma=\sqrt{n}(S_n/n-\mu)/\sigma=(S_n-n\mu)/(\sigma\sqrt{n})$. The latter sequence converges in distribution to $N(0,1)$ by the stated iid central limit theorem. Thus both displayed expressions are the same random variable for each $n$, and have the same limit in distribution. $\square$

This algebra also identifies the roles of the two adjustments. Since $\mathbb{E}[S_n]=n\mu$, subtracting $n\mu$ centers the sum. Independence gives $\operatorname{Var}(S_n)=n\sigma^2$, so its standard deviation is $\sigma\sqrt{n}$. Dividing by this quantity makes the standardized sum have mean zero and variance one for every $n$, not only in the limit.

Result (exact moments of the standardized sum): For every $n\ge1$, $\mathbb{E}[Z_n]=0$ and $\operatorname{Var}(Z_n)=1$.

Proof. Linearity of expectation gives $\mathbb{E}[S_n]=\sum_{k=1}^n\mathbb{E}[X_k]=n\mu$. Independence gives $\operatorname{Var}(S_n)=\sum_{k=1}^n\operatorname{Var}(X_k)=n\sigma^2$. Therefore $\mathbb{E}[Z_n]=(n\mu-n\mu)/(\sigma\sqrt{n})=0$, and $\operatorname{Var}(Z_n)=n\sigma^2/(\sigma^2 n)=1$. The denominators are nonzero because $n\ge1$ and $\sigma^2>0$. $\square$

Worked Example: Bernoulli Observations

Let $X_1,X_2,\ldots$ be iid Bernoulli random variables with $\mathbb{P}(X_k=1)=\tfrac13$ and $\mathbb{P}(X_k=0)=\tfrac23$. Find the standardized sum and check its exact mean and variance.

For one observation, $\mathbb{E}[X_k]=1\cdot\tfrac13+0\cdot\tfrac23=\tfrac13$, and $\mathbb{E}[X_k^2]=1^2\cdot\tfrac13+0^2\cdot\tfrac23=\tfrac13$. Hence $\sigma^2=\tfrac13-(\tfrac13)^2=\tfrac29$, which is finite and positive. The sum $S_n$ counts the number of successes, so the standardized sum is

Z_n=\frac{S_n-\tfrac n3}{\sqrt{\tfrac{2n}{9}}} =\frac{3S_n-n}{\sqrt{2n}}.

Check: $\mathbb{E}[S_n]=n/3$ and $\operatorname{Var}(S_n)=n(2/9)=2n/9$. Thus $\mathbb{E}[Z_n]=0$ and $\operatorname{Var}(Z_n)=(2n/9)/(2n/9)=1$. The iid central limit theorem gives $Z_n\xrightarrow{\ d\ }N(0,1)$. The exact moment check is not itself the limit theorem: many different distributions have mean zero and variance one.

Worked Example: A Nonzero Mean and Unit Variance

Let the iid variables take values $0$ and $2$, each with probability $\tfrac12$. Write the standardized sum explicitly and verify its centering and scale.

For one variable, $\mu=0\cdot\tfrac12+2\cdot\tfrac12=1$. Its second moment is $0^2\cdot\tfrac12+2^2\cdot\tfrac12=2$, so $\sigma^2=2-1^2=1$. Therefore \[ Z_n=\frac{S_n-n}{\sqrt{n}}. \] The subtraction is $n$, not zero: the sum has mean $n$. Independence gives $\mathbb{E}[S_n]=n$ and $\operatorname{Var}(S_n)=n$. Consequently $\mathbb{E}[Z_n]=(n-n)/\sqrt n=0$ and $\operatorname{Var}(Z_n)=n/n=1$. By the theorem, $Z_n\xrightarrow{\ d\ }N(0,1)$.

This example illustrates why “standardized” does not mean simply dividing a sum by $\sqrt n$. The mean must first be removed, unless it is already zero.

What Changes When the Center or Scale Changes?

The theorem’s normalization is not the only possible one. A fixed positive change in scale changes the variance of the limiting normal distribution; a center shifted by an amount comparable to $\sqrt n$ changes its mean. The following result makes this precise for positive scales.

Theorem (affine transformation of a standard-normal limit): Suppose $Z_n\xrightarrow{\ d\ }N(0,1)$, and let $a_n$ and $b_n$ be deterministic real sequences such that $a_n\to a$, $b_n\to b$, and $b>0$. Then $a_n+b_nZ_n\xrightarrow{\ d\ }N(a,b^2)$.

Proof. Let $F_n$ be the distribution function of $Z_n$, and fix $x\in\mathbb{R}$. Since $b_n\to b>0$, eventually $b_n>0$, and therefore $\mathbb{P}(a_n+b_nZ_n\le x)=F_n((x-a_n)/b_n)$. Set $t=(x-a)/b$. The convergence of $a_n,b_n$ and the positivity of $b$ imply $(x-a_n)/b_n\to t$. For any $\varepsilon>0$, eventually $t-\varepsilon\le (x-a_n)/b_n\le t+\varepsilon$. Monotonicity of distribution functions then gives $F_n(t-\varepsilon)\le F_n((x-a_n)/b_n)\le F_n(t+\varepsilon)$. At each fixed endpoint, convergence in distribution gives $F_n(t-\varepsilon)\to\Phi(t-\varepsilon)$ and $F_n(t+\varepsilon)\to\Phi(t+\varepsilon)$. Taking lower and upper limits in the inequality, and then letting $\varepsilon$ decrease to zero, gives the limit $\Phi(t)$, because $\Phi$ is continuous. This is the distribution function at $x$ of $N(a,b^2)$. The argument holds for every real $x$, proving the result. $\square$

In particular, suppose $d_n>0$ and $c_n$ are alternative deterministic scales and centers for the sum. The identity \[ \frac{S_n-c_n}{d_n} =\frac{n\mu-c_n}{d_n}+\frac{\sigma\sqrt n}{d_n}Z_n \] shows exactly what must be checked. If $(n\mu-c_n)/d_n\to a$ and $\sigma\sqrt n/d_n\to b>0$, the limit is $N(a,b^2)$. A center that differs from $n\mu$ by a non-negligible amount on the scale $d_n$ can thus shift the limiting mean. A scale that differs from $\sigma\sqrt n$ by a fixed factor can change the limiting variance.

Worked Example: A Shifted Center and a Different Scale

Let the iid variables take values $1$ and $7$, each with probability $\tfrac12$. Consider the sum centered at $4n-3\sqrt n$ and divided by $6\sqrt n$. Find the limiting distribution.

The mean of one observation is $1\cdot\tfrac12+7\cdot\tfrac12=4$. Its second moment is $1^2\cdot\tfrac12+7^2\cdot\tfrac12=25$, so its variance is $25-4^2=9$, and its standard deviation is $3$. The theorem gives $Z_n=(S_n-4n)/(3\sqrt n)\xrightarrow{\ d\ }N(0,1)$. Now substitute this expression into the alternative normalization: \[ \frac{S_n-(4n-3\sqrt n)}{6\sqrt n} =\frac{S_n-4n}{6\sqrt n}+\frac{3\sqrt n}{6\sqrt n} =\frac12 Z_n+\frac12. \] The affine transformation result applies with $a_n=a=\tfrac12$ and $b_n=b=\tfrac12>0$. Hence the limit is $N(\tfrac12,\tfrac14)$.

Check: the alternative statistic has mean $(4n-(4n-3\sqrt n))/(6\sqrt n)=3\sqrt n/(6\sqrt n)=\tfrac12$. Its variance is $9n/(36n)=\tfrac14$. These are the mean and variance of the claimed limiting normal distribution.

When the Theorem Does Not Apply

Independence cannot simply be dropped. For a concrete example, let $Y$ take values $-1$ and $1$, each with probability $\tfrac12$, and set $X_k=Y$ for every $k$. Each $X_k$ has mean zero and variance one, but the sequence is dependent: all its terms are identical. Its standardized sum is $S_n/\sqrt n=\sqrt n\,Y$. For every fixed $M>0$, once $n>M^2$, $\mathbb{P}(|\sqrt n\,Y|>M)=1$. These distributions do not converge to $N(0,1)$, whose probability of lying in $[-M,M]$ is positive. Thus matching the one-variable mean and variance is not enough to obtain the iid central limit theorem.

Strict positivity of the variance is also necessary for this particular standardization. If $X_k=c$ with probability one for a fixed constant $c$, then $\sigma^2=0$ and $(S_n-n\mu)/(\sigma\sqrt n)$ is undefined. The theorem does not assert a standard-normal limit in this degenerate case. Other central limit theorems have different hypotheses, but they are not needed for the iid finite-positive-variance result stated here.

Common mistake. For the Bernoulli variables with success probability $\tfrac13$, one might divide $S_n-n/3$ by $\sqrt{n/3}$, treating $n/3$ as the variance. But the variance of $S_n$ is $n(2/9)=2n/9$, not $n/3$. With the incorrect denominator, the variance of the resulting statistic is \[ \frac{2n/9}{n/3}=\frac23, \] so the standard-normal conclusion does not follow; the corresponding limit is $N(0,\tfrac23)$, by the positive-scale transformation result. The correct denominator is $\sqrt{2n/9}$, the standard deviation of the sum. Always compute the variance before choosing the scale.

Check Your Understanding

For each question, identify the centering and scale before deciding what limit is justified.

If iid variables have mean $5$ and variance $4$, write the standardized-sum and sample-mean forms of the central limit theorem.
For iid Bernoulli variables with success probability $\tfrac14$, calculate the mean and variance of one observation, then write the standardized sum.
If $Z_n\xrightarrow{\ d\ }N(0,1)$, what is the limit of $2-3Z_n$? Explain how its mean and variance follow from the affine transformation result.
Suppose $S_n$ has mean $n\mu$ and variance $n\sigma^2$, with $\sigma^2>0$. Verify the mean and variance of $(S_n-n\mu)/(\sigma\sqrt n)$.
In the repeated-variable example, why does $\sqrt n\,Y$ fail to converge in distribution to a standard normal variable?