Centering and Scaling Sums for Normal Limits

Why Centering and Scaling Come First

A normal-limit calculation starts by identifying the location and spread of the quantity whose distribution is being studied. For a sum of independent variables, its expectation sets the natural center, while its total standard deviation sets the natural scale. These are determined by the summands’ distributions and the way they enter the sum; they are not chosen simply because a familiar formula uses \(\sqrt{n}\).

This tutorial develops that normalization and calculates it in several settings. The result is a random variable with mean zero and variance one, provided the sum has finite, positive variance. That property is an exact finite-sample calculation. It does not, by itself, say that the resulting distribution is normal or that it converges to a normal distribution. A central limit theorem supplies such a convergence conclusion only under additional conditions.

The Exact Normalization for an Independent Sum

The numerator subtracts the sum’s expectation. The denominator is the square root of the sum’s variance, which is the sum of the individual variances under independence. A standard deviation is a square root of a variance, so variances—not standard deviations—are added before taking the square root.

Proof. Linearity of expectation gives \(E[S_n]=\sum_{i=1}^n\mu_i\), so the numerator has expectation zero. To calculate its variance, center each summand and expand:

For \(i\ne j\), independence implies that the centered variables are independent. Their product is integrable, since \(E[|(X_i-\mu_i)(X_j-\mu_j)|]=E[|X_i-\mu_i|]E[|X_j-\mu_j|]<\infty\). Thus the cross-term expectation factors into two zero expectations. It follows that \(\operatorname{Var}(S_n)=\sum_{i=1}^n\sigma_i^2\). Dividing the centered sum by the square root of this positive variance gives \(E[Z_n]=0\) and \(\operatorname{Var}(Z_n)=1\). \(\square\)

Independence is a sufficient condition for the variance sum above; without it, covariances may contribute. In that case, the variance of a finite sum is \(\sum_i\operatorname{Var}(X_i)+2\sum_{i<j}\operatorname{Cov}(X_i,X_j)\), whenever the second moments exist.

Identically Distributed Summands

If \(X_1,\ldots,X_n\) are iid with \(E[X_i]=\mu\) and \(\operatorname{Var}(X_i)=\sigma^2\), where \(0<\sigma^2<\infty\), then the sum has mean \(n\mu\) and variance \(n\sigma^2\). Its standard deviation is \(\sqrt{n}\sigma\), so the normalization becomes

The two forms are equal because \(\sum_i(X_i-\mu)=n(\bar X_n-\mu)\). This algebra explains the \(\sqrt{n}\) in the normalization of an iid sum or sample mean: the total variance grows as \(n\), and the total standard deviation therefore grows as \(\sqrt{n}\). The formula uses the population standard deviation \(\sigma\), not a sample estimate.

Weighted Sums: Coefficients Affect Both Center and Spread

For a weighted sum \(T=\sum_{i=1}^n a_iX_i\) of independent variables with finite means and variances, its expectation is \(\sum_i a_i\mu_i\), and its variance is \(\sum_i a_i^2\sigma_i^2\). The square on each coefficient matters: a negative weight reverses the contribution to the mean but still contributes nonnegatively to the variance.

To verify the variance formula, center the weighted summands and expand the square. Each diagonal term has expectation \(a_i^2E[(X_i-\mu_i)^2]=a_i^2\sigma_i^2\). Each cross-term has expectation zero by independence and centering, just as in the theorem above. The weighted sum’s expectation follows directly from linearity. Hence the displayed quotient has mean zero and variance one.

For iid \(X_i\) with mean \(\mu\) and variance \(\sigma^2\), this specializes to

as long as \(\sigma^2\sum_i a_i^2>0\). In general this is not the same as the equal-weight normalization: the sum of squared weights, not the number of terms alone, determines the variance.

Triangular Arrays: Normalize Each Row Separately

In a triangular array, the variables in row \(n\) are written \(X_{n,1},\ldots,X_{n,k_n}\). The row length \(k_n\) may change with \(n\), and the distributions may vary across both indices. Suppose the variables within each row are independent, each has a finite mean \(\mu_{n,k}\) and finite variance \(\sigma_{n,k}^2\), and define

When \(s_n^2>0\), the row’s normalization is \((S_n-m_n)/s_n\). Both center and scale are indexed by \(n\), because they can change from row to row. Independence is needed only among the variables within each row for this variance calculation; dependence between different rows does not change the normalization of any single row.

What the Normalization Does—and Does Not Do

Subtracting the exact expectation and dividing by the exact standard deviation creates a common scale for sums whose raw magnitudes may differ. It is a bookkeeping step essential to studying normal limits: without it, changing the mean or variance can obscure the behavior of the distribution. But standardizing does not make an arbitrary variable normal, and mean zero with variance one does not uniquely determine a distribution.

For instance, if \(X\) is Bernoulli with success probability \(1/2\), then \(E[X]=1/2\) and \(\operatorname{Var}(X)=1/4\). Its standardized version is \((X-1/2)/(1/2)\), which takes only the values \(-1\) and \(1\). It has mean zero and variance one, but it is not a normal variable. Normal approximation requires a limit result and its hypotheses, not merely this moment calculation.

• Common mistake: adding standard deviations. Independence gives \(\operatorname{Var}(S_n)=\sum_i\sigma_i^2\), so the denominator is \(\sqrt{\sum_i\sigma_i^2}\), not \(\sum_i\sigma_i\).
• Common mistake: using \(n\) as the scale in every setting. For iid terms the standard deviation is \(\sigma\sqrt{n}\); for weighted terms it is \(\sigma\sqrt{\sum_i a_i^2}\); for a triangular array it is the square root of that row’s total variance.
• Common mistake: treating a negative weight as a negative variance contribution. The mean uses \(a_i\), while the variance uses \(a_i^2\). The latter cannot be negative.
• Common mistake: confusing exact standardization with a normal limit. Unit variance is an identity under the stated assumptions. A normal-limit claim needs additional distributional conditions.