Normal Approximation for Independent Nonidentical Sums

Why Nonidentical Sums Need Their Own Standardization

A sum need not be made up of identically distributed terms to have a useful normal approximation. For example, a total might combine measurements from several groups, where each group has a different distribution or spread. The key calculations use the mean and variance of each term, rather than multiplying one common mean and variance by the number of terms.

Suppose \(X_1,\ldots,X_n\) are independent random variables with finite means \(\mu_i=E[X_i]\) and finite variances \(\sigma_i^2=\operatorname{Var}(X_i)\). Write their sum as \(S_n=\sum_{i=1}^n X_i\). Independence gives the aggregate mean and variance:

\mu_S=E[S_n]=\sum_{i=1}^n\mu_i, \qquad \sigma_S^2=\operatorname{Var}(S_n)=\sum_{i=1}^n\sigma_i^2.

If a central limit approximation is appropriate, \(S_n\) is approximated by a normal variable with mean \(\mu_S\) and variance \(\sigma_S^2\). Thus, for a threshold \(c\), the standardized value is \(z=(c-\mu_S)/\sigma_S\), where \(\sigma_S=\sqrt{\sigma_S^2}\). This is the same standardization idea used for sums of identically distributed variables, with the aggregate quantities now calculated term by term.

Definition: A sum is independent, nonidentical when its terms are mutually independent but are not required to have the same probability distribution. Its exact mean is the sum of the individual means, and its exact variance is the sum of the individual variances. The normal approximation replaces the distribution of the standardized sum by the standard normal distribution when suitable central limit conditions hold.

Why the Mean and Variance Add

Linearity of expectation gives \(E[\sum_i X_i]=\sum_iE[X_i]\), whether or not the terms are independent. For the variance, expand the squared deviation from the aggregate mean:

\operatorname{Var}(S_n) =E\left[\left(\sum_{i=1}^n(X_i-\mu_i)\right)^2\right] =\sum_{i=1}^n\operatorname{Var}(X_i) +2\sum_{i<j}\operatorname{Cov}(X_i,X_j).

Independence makes each covariance zero, so only the individual variances remain. This also explains an important restriction: if the terms are dependent, the variance generally includes covariance terms and cannot be found by simply adding individual variances.

For a sequence of sums, a standard sufficient condition for the normal limit is the Lyapunov condition. Let \(\mu_{ni}=E[X_{ni}]\), let \(s_n^2=\sum_i\operatorname{Var}(X_{ni})\), and suppose that for some \(\delta>0\),

\frac{\sum_i E\left[|X_{ni}-\mu_{ni}|^{2+\delta}\right]} {s_n^{2+\delta}}\longrightarrow 0.

When the terms in each row are independent and this condition holds, the standardized row sum converges in distribution to \(N(0,1)\). In practice, this result justifies using the normal distribution for sufficiently large sums under the stated conditions; it does not say that every finite sum is exactly normal. For repeated collections of bounded terms with no single term growing on the scale of the total standard deviation, the condition is often readily satisfied.

Checking Whether One Term Dominates

A useful first diagnostic is the largest individual variance as a fraction of total variance:

r_{\max}=\frac{\max_i \sigma_i^2}{\sum_i\sigma_i^2}.

If this fraction is large, much of the total spread comes from one term, so the sum is not clearly being smoothed by many small contributions. A small fraction is reassuring, but it is not by itself a proof of a good normal approximation: distributional shape and tail behavior also matter. The Lyapunov condition above is one sufficient condition that accounts for more than variance shares alone.

This diagnostic is especially useful when the terms have very different scales. Counting terms is not enough: a sum with many low-variance terms may still be driven by one high-variance term. Always compare individual variances with their total, rather than assuming that the number of terms settles the question.

Worked Example: Different Uniform Distributions for Task Durations

In a hypothetical scheduling model, 20 independent task durations are uniformly distributed on \([0,2]\) hours, 30 are uniformly distributed on \([0,4]\) hours, and 10 are uniformly distributed on \([0,6]\) hours. The groups are independent, and durations within each group are independent. Approximate the probability that the total duration is at most 120 hours, and check whether an individual term dominates the variance.

Method. For a uniform variable on \([a,b]\), the mean is \((a+b)/2\) and the variance is \((b-a)^2/12\). For the first group, each mean is \(1\) hour and each variance is \(4/12=1/3\) hour squared. For the second group, these are \(2\) hours and \(16/12=4/3\) hour squared. For the third group, they are \(3\) hours and \(36/12=3\) hour squared.

The aggregate mean is

\mu_S=20(1)+30(2)+10(3)=20+60+30=110\text{ hours}.

By independence, the aggregate variance is

\sigma_S^2=20\left(\frac13\right)+30\left(\frac43\right)+10(3) =\frac{20}{3}+40+30 =\frac{220}{3}\approx73.3333\text{ hours}^2.

Therefore, \(\sigma_S=\sqrt{220/3}\approx8.5635\) hours. The standardized threshold is \(z=(120-110)/8.5635\approx1.1677\). The normal approximation gives \(P(S\le120)\approx\Phi(1.1677)=0.8785\), to four decimal places.

The largest individual variance is \(3\), so \(r_{\max}=3/(220/3)=9/220\approx0.0409\). No single task accounts for a large share of total variance. The bounded task durations and increasing numbers of independent terms also fit the usual setting in which a central limit approximation is appropriate. As a check, the threshold is above the aggregate mean, giving a probability above \(0.5\), as the calculated value does.

Common mistake avoided: using the variance of one group for every task. The three groups have different variances, so each group’s variance must be multiplied by its own count and then added.

Worked Example: Independent Exponential Waiting Times

A hypothetical system has 25 independent waiting times with exponential distributions of mean 2 minutes and 15 independent waiting times with exponential distributions of mean 1 minute. All waiting times are mutually independent. Approximate the probability that their total exceeds 80 minutes, and assess the largest variance share.

Model and moments. An exponential variable with mean \(\theta\) has variance \(\theta^2\). Thus the first group has mean 2 and variance 4 per term; the second has mean 1 and variance 1 per term. The total mean is

\mu_S=25(2)+15(1)=50+15=65\text{ minutes}.

The total variance and standard deviation are

\sigma_S^2=25(4)+15(1)=100+15=115\text{ minutes}^2, \qquad \sigma_S=\sqrt{115}\approx10.7238\text{ minutes}.

Standardizing 80 minutes gives \(z=(80-65)/10.7238\approx1.3988\). Therefore,

P(S>80)\approx 1-\Phi(1.3988)=1-0.9191=0.0809.

The largest individual variance is 4, so \(r_{\max}=4/115\approx0.0348\). No single term dominates by this variance-share diagnostic. Exponential variables have finite moments of every positive order; for growing collections with these two types retained in fixed proportions, the Lyapunov ratio tends to zero. The computed approximation is therefore supported by the standard independent-sum limit argument. The result is about 8.09%, to four decimal places. Since 80 minutes is above the total mean, an upper-tail probability below \(0.5\) is consistent with the result.

Common mistake avoided: treating an exponential variable’s mean as its variance. For an exponential variable with mean \(\theta\), the variance is \(\theta^2\), not \(\theta\).

Worked Example: A Large Variance Share

A portfolio of independent measurement errors consists of one variable uniformly distributed on \([-10,10]\) and 99 variables uniformly distributed on \([-1,1]\). All have mean zero, and all are independent. Use the normal-approximation diagnostic to decide whether the usual many-small-terms rationale is persuasive.

Method. A uniform variable on \([-a,a]\) has mean zero and variance \((2a)^2/12=a^2/3\). The wide variable therefore has variance \(100/3\). Each of the other 99 variables has variance \(1/3\), so together they contribute \(99/3=33\).

The total variance is

\sigma_S^2=\frac{100}{3}+\frac{99}{3}=\frac{199}{3}.

The largest variance share is

r_{\max} =\frac{100/3}{199/3} =\frac{100}{199} \approx0.5025.

One term accounts for about 50.25% of the total variance. Although there are 100 independent terms, the sum is not evidently built from many comparable small contributions. The variance-share check does not support relying on the usual many-small-terms rationale for a normal approximation. It does not prove that the sum cannot be approximated by a normal distribution; it shows that this diagnostic alone gives no such justification. The calculation is checked by noting that \(100/3+99/3=199/3\), so the stated share is less than, but close to, one half.

Common mistake avoided: concluding that 100 terms automatically make a sum approximately normal. The scale of each term matters; here one term supplies about half the total variance.

Common Calculation Traps

The normal approximation is straightforward once the model is clear, but several errors can produce a plausible-looking yet incorrect answer.

Adding standard deviations instead of variances. Independent variances add; the aggregate standard deviation is the square root of their sum.
Using one mean or variance for all terms. In a nonidentical sum, calculate each group’s moments from its own distribution and parameters.
Using independence without checking it. If terms are dependent, covariance terms may change the variance.
Confusing a diagnostic with a theorem. A small largest variance share is helpful evidence, not a complete condition for normality. A large share warns that the ordinary many-small-terms justification is weak.
Standardizing with the wrong center or scale. Use the sum of means and the square root of the sum of variances, not the mean and standard deviation of a typical term.

For a probability involving a threshold \(c\), first calculate \(\mu_S\) and \(\sigma_S\), then form \((c-\mu_S)/\sigma_S\). The direction of the event determines whether the normal probability is a lower tail, an upper tail, or a difference of two tails. Keep the problem’s units consistent throughout.

Key takeaway: For independent, nonidentically distributed terms, add their individual means and variances, then standardize the sum using the resulting aggregate mean and standard deviation. Before relying on the normal approximation, check that no single term supplies a substantial share of total variance and remember that this diagnostic alone does not establish the required limit conditions.

Check Your Understanding

For each problem, state the aggregate moments, standardize using the normal approximation where requested, and check the variance share when asked. Report probabilities to four decimal places.

A total is the sum of 12 independent variables uniform on \([0,2]\) and 18 independent variables uniform on \([0,4]\); all terms are mutually independent. Approximate the probability that the total is at most 55, and find the largest individual variance share. (Answer: \(\mu_S=48\), \(\sigma_S^2=28\), \(z=1.3229\), probability \(0.9071\), largest share \(1/7\approx0.1429\).)
A sum consists of 20 independent exponential variables with mean 1 and 10 independent exponential variables with mean 3; all terms are mutually independent. Approximate the probability that the total exceeds 65, and find the largest individual variance share. (Answer: \(\mu_S=50\), \(\sigma_S^2=110\), \(z=1.4304\), upper-tail probability \(0.0763\), largest share \(9/110\approx0.0818\).)
Let \(S\) be the sum of 40 independent variables uniform on \([-1,1]\) and 10 independent variables uniform on \([-2,2]\), with all terms mutually independent. Approximate \(P(|S|<6)\). (Answer: \(\mu_S=0\), \(\sigma_S^2=80/3\), standardized boundary \(6/\sqrt{80/3}\approx1.1619\), probability \(0.7546\).)
A sum contains one independent variable uniform on \([-12,12]\) and 48 independent variables uniform on \([-1,1]\). Find the total variance and the largest variance share. Does the variance-share diagnostic support a many-small-terms rationale? (Answer: total variance \(64\), largest share \(48/64=0.7500\); no, one term supplies 75% of total variance.)
A total is formed from 25 independent variables uniform on \([0,1]\) and 25 independent variables uniform on \([0,3]\), with all terms mutually independent. Approximate the probability that the total is below 54, and find the largest individual variance share. (Answer: \(\mu_S=50\), \(\sigma_S^2=125/6\), \(z\approx0.8764\), probability \(0.8095\), largest share \(9/50=0.1800\).)

What You'll Learn