Tail Probabilities for Sums and Averages

From a Sum or Average to a Tail Probability

A probability about a sum or sample mean is often easier to evaluate after expressing the event in standard-deviation units. The central limit theorem (CLT) supplies the approximation: under suitable conditions, a standardized sum or average is approximately standard normal. The remaining work is to translate the event correctly, use the appropriate lower tail, upper tail, or interval, and interpret the result in context.

This tutorial concerns probabilities for sums and averages, not the reverse task of finding a cutoff that gives a specified probability. The normal approximation is not an exact distributional identity unless the variable in question is itself normal. Its accuracy depends on the model and the conditions for the CLT.

Definition: Let \(Z\sim N(0,1)\). Its cumulative distribution function is \(\Phi(z)=P(Z\le z)\). For a normal approximation, a standardized cutoff \(z\) gives a lower-tail probability \(\Phi(z)\), an upper-tail probability \(1-\Phi(z)\), and an interval probability \(\Phi(z_U)-\Phi(z_L)\), where \(z_L\) and \(z_U\) are the standardized lower and upper cutoffs.

Standardizing Sums and Sample Means

For independent and identically distributed random variables \(X_1,\ldots,X_n\) with finite mean \(\mu\) and finite, positive variance \(\sigma^2\), define the sum \(S_n=\sum_{j=1}^n X_j\) and sample mean \(\bar X=S_n/n\). Their means and variances are

E[S_n]=n\mu,\qquad \operatorname{Var}(S_n)=n\sigma^2,\qquad E[\bar X]=\mu,\qquad \operatorname{Var}(\bar X)=\frac{\sigma^2}{n}.

When the CLT approximation is appropriate, \(S_n\) is approximated by a normal distribution with mean \(n\mu\) and standard deviation \(\sigma\sqrt n\). Equivalently, \(\bar X\) is approximated by a normal distribution with mean \(\mu\) and standard error \(\sigma/\sqrt n\). “Standard error” here means the standard deviation of the sampling distribution of the sample mean.

Z_S=\frac{S_n-n\mu}{\sigma\sqrt n}\ \approx\ N(0,1), \qquad Z_{\bar X}=\frac{\bar X-\mu}{\sigma/\sqrt n}\ \approx\ N(0,1).

For independent terms that are not identically distributed, let their means be \(\mu_j\) and variances be \(\sigma_j^2\). Independence gives the sum mean \(\sum_j\mu_j\) and variance \(\sum_j\sigma_j^2\). If a normal approximation for this sum is appropriate, standardize using those totals. The approximation requires suitable CLT conditions; independence alone does not guarantee a good normal approximation.

Key result: For a sum with mean \(M\) and variance \(V>0\), use \(z=(c-M)/\sqrt V\) to standardize a cutoff \(c\). For a sample mean with population mean \(\mu\), variance \(\sigma^2\), and sample size \(n\), use \(z=(c-\mu)/(\sigma/\sqrt n)\). A sum and its sample mean have different scales, so their denominators are different.

Why Standardization Gives the Probability

The probability calculation follows from an algebraic change of scale and the CLT. For an iid sample with finite mean \(\mu\) and positive finite variance \(\sigma^2\), the CLT states that \((S_n-n\mu)/(\sigma\sqrt n)\) converges in distribution to \(N(0,1)\) as \(n\) grows. Thus, when the approximation is suitable, the probability that this standardized quantity is at most \(z\) is approximately \(\Phi(z)\).

For example, if the event is \(S_n\le c\), subtracting \(n\mu\) from both sides and dividing by the positive number \(\sigma\sqrt n\) gives the standardized cutoff \((c-n\mu)/(\sigma\sqrt n)\). The inequality direction remains unchanged because the denominator is positive. The CLT then gives the lower-tail approximation:

P(S_n\le c) = P\left(\frac{S_n-n\mu}{\sigma\sqrt n} \le \frac{c-n\mu}{\sigma\sqrt n}\right) \approx \Phi\left(\frac{c-n\mu}{\sigma\sqrt n}\right).

The same reasoning applies to other event boundaries. For a sample mean, replace the sum’s center and standard deviation by \(\mu\) and \(\sigma/\sqrt n\). For an interval, standardize each endpoint separately and subtract the lower cumulative probability from the upper one.

Identify the statistic and its scale.
Determine whether the event concerns a sum or a sample mean, and find its mean and standard deviation.

Translate the event boundaries.
Keep track of words such as “at most,” “at least,” and “between.” For a discrete sum, apply the continuity correction when using a normal approximation.

Standardize each boundary.
Subtract the appropriate mean and divide by the appropriate standard deviation or standard error.

Evaluate and interpret.
Use \(\Phi\) for a lower tail, \(1-\Phi\) for an upper tail, or the difference of two cumulative probabilities for an interval. State what the probability means in the original setting.

Worked Probability Calculations

Worked Example: A Lower Tail for a Discrete Count

In a hypothetical survey, each of 100 independently selected participants is equally likely to choose either of two response options. Let \(N\) be the number who choose option A. Use a normal approximation with a continuity correction to estimate \(P(N\le42)\).

Method and model. Under the stated assumptions, \(N\sim\operatorname{Binomial}(100,0.5)\). Its mean and variance are \(np=100(0.5)=50\) and \(np(1-p)=100(0.5)(0.5)=25\), so its standard deviation is \(\sqrt{25}=5\). We use a normal approximation, as requested.

Continuity correction. The integer event \(N\le42\) is approximated by the continuous event \(Y\le42.5\), where \(Y\) has the matching normal mean and standard deviation. Thus

z=\frac{42.5-50}{5}=-1.5,\qquad P(N\le42)\approx\Phi(-1.5)=0.0668.

The estimate says that about 6.68% of repeated samples under this hypothetical model would have at most 42 participants choosing option A. The common mistake this method avoids is using 42 as the normal cutoff without adjusting for the discrete boundary; that would standardize a different event.

Check. The cutoff is below the mean, so the lower-tail probability must be less than 0.5. The estimate \(0.0668\) has the correct direction and scale.

Worked Example: An Interval for a Sample Mean

Suppose a hypothetical population of repair times has mean \(\mu=100\) minutes and standard deviation \(\sigma=16\) minutes. A random sample of \(n=64\) repair times is taken. Assuming the CLT approximation is appropriate, estimate the probability that the sample mean is between 97 and 103 minutes.

Method. The standard deviation of the sample mean is the standard error, not the population standard deviation:

\operatorname{SE}(\bar X)=\frac{\sigma}{\sqrt n} =\frac{16}{\sqrt{64}}=\frac{16}{8}=2\text{ minutes}.

Standardize both endpoints using this standard error. The lower endpoint gives \(z_L=(97-100)/2=-1.5\), and the upper endpoint gives \(z_U=(103-100)/2=1.5\). Therefore,

P(97\le\bar X\le103) \approx\Phi(1.5)-\Phi(-1.5) =0.9332-0.0668 =0.8664.

The estimated probability is 0.8664, or 86.64%. The event is centered on the population mean, which explains the symmetric standardized endpoints. The main mistake to avoid is dividing by 16 instead of by the standard error of 2. That error would use \(z_L=-3/16=-0.1875\) and \(z_U=3/16=0.1875\), producing an interval probability of about 0.1488 rather than 0.8664.

Check. The interval extends 1.5 standard errors on either side of the center. A probability substantially greater than one-half is therefore reasonable, and the calculated value is below 1.

Worked Example: An Upper Tail for a Sum of Independent Terms

A hypothetical portfolio model has two groups of independent, continuous-valued payment amounts, all measured in thousands of dollars. There are 16 payments in the first group, each with mean 8 and variance 9, and 16 in the second group, each with mean 12 and variance 16. Assume the normal approximation for the total \(T\) is appropriate. Estimate \(P(T\ge350)\), where \(T\) is in thousands of dollars.

Method. Independence means the total mean is the sum of the component means, and the total variance is the sum of their variances:

E[T]=16(8)+16(12)=128+192=320, \qquad \operatorname{Var}(T)=16(9)+16(16)=144+256=400.

The total standard deviation is \(\sqrt{400}=20\) thousand dollars. Because the payment amounts are continuous-valued, no continuity correction is used. The standardized cutoff is

z=\frac{350-320}{20}=\frac{30}{20}=1.5.

The requested event is an upper tail, not a lower tail:

P(T\ge350)\approx1-\Phi(1.5)=1-0.9332=0.0668.

The estimated probability that the total exceeds 350 thousand dollars is 0.0668. A common mistake is to report \(\Phi(1.5)=0.9332\), which is the probability of being below the cutoff. Since 350 is above the total mean of 320, the requested upper-tail probability should be less than one-half. The answer passes this check.

Reading the Standardized Cutoff

A \(z\)-score describes a cutoff’s location relative to the center in standard-deviation units. A negative score is below the mean, a positive score is above it, and a score near zero is close to the mean. The sign helps identify which side of the distribution the cutoff occupies, but it does not by itself determine whether the requested answer is \(\Phi(z)\) or \(1-\Phi(z)\). That depends on the event.

Event under a normal approximation	Probability calculation
At most a cutoff \(c\)	\(\Phi(z_c)\)
At least a cutoff \(c\)	\(1-\Phi(z_c)\)
Between lower cutoff \(a\) and upper cutoff \(b\)	\(\Phi(z_b)-\Phi(z_a)\)

When the statistic is discrete and a normal approximation is used, a continuity correction adjusts the boundary to better match the integer event. For example, “at most \(k\)” uses \(k+0.5\), “at least \(k\)” uses \(k-0.5\), and “from \(a\) through \(b\), inclusive” uses \(a-0.5\) and \(b+0.5\). A continuous statistic does not need this adjustment. The correction changes the cutoff before standardization; it does not change the mean or standard deviation.

Common Errors and Practical Checks

Using the wrong scale. A sum’s standard deviation is \(\sigma\sqrt n\), while a sample mean’s standard error is \(\sigma/\sqrt n\). Check whether the event is stated in total units or average units before choosing the denominator.
Choosing the wrong tail. “At most” calls for a lower-tail probability; “at least” calls for an upper-tail probability. Draw or imagine the normal curve if the direction is unclear.
Subtracting interval probabilities in the wrong order. The upper cumulative probability must come first: \(\Phi(z_U)-\Phi(z_L)\). Since \(z_U>z_L\), the result should be nonnegative.
Forgetting discreteness. Apply the continuity correction when approximating an integer-valued sum. Do not apply it to a continuous-valued sum or sample mean.
Rounding too early. Carry the standardized cutoff at full calculator precision into \(\Phi\), then round the final probability. Early rounding can change the reported tail probability.
Treating an approximation as exact. The CLT provides a useful approximation under suitable conditions; it does not make a finite-sample distribution exactly normal. If the conditions are not established or the approximation is explicitly assumed, make that status clear.

A final reasonableness check should use both the direction and the size of the cutoff. For instance, an upper-tail event with a cutoff well above the mean should not produce a probability greater than one-half. An interval probability should lie between zero and one, and moving either endpoint outward cannot reduce it. These checks catch many sign and tail-selection errors before they reach a final answer.

Key takeaway: Find the correct mean and standard deviation for the statistic, translate each event boundary (including a continuity correction when needed), standardize without changing the inequality direction, and match the resulting \(z\)-score to the event’s tail. Keep full precision until the final probability.

Check Your Understanding

Use a normal approximation where stated, and retain full precision in intermediate calculations. Probabilities are rounded to four decimal places.

A hypothetical count \(N\) is binomial with \(n=100\) and \(p=0.5\). Using a continuity correction, estimate \(P(N\ge58)\). (Answer: 0.0668)
A population has mean 20 and standard deviation 6. For a random sample of 36 observations, assume the CLT approximation is appropriate. Estimate the probability that the sample mean is below 18.5. (Answer: 0.0668)
A sum \(S\) consists of 25 iid continuous random variables, each with mean 4 and variance 9. Using a normal approximation, estimate \(P(85\le S\le130)\). (Answer: 0.8186)
A total \(T\) consists of 10 independent terms, each with mean 6 and variance 4, and 15 independent terms, each with mean 8 and variance 4. Assume a normal approximation is appropriate. Estimate \(P(170\le T\le200)\). (Answer: 0.8186)

What You'll Learn