CLT Accuracy in Central and Extreme Tails

Why Accuracy Depends on Where You Look

The central limit theorem (CLT) says that, under suitable conditions, the standardized sample mean becomes approximately normal as the sample size grows. But “approximately normal” does not mean that every probability is equally well approximated. A normal curve may give a reasonable estimate for an interval near the center and a noticeably poor estimate for a rare event far out in a tail.

The location of an event is described by its standardized cutoff: how many standard errors the boundary lies from the mean. A cutoff near zero concerns the center; a cutoff several standard errors away concerns a tail. Skewness also matters. When the population is asymmetric, the sampling distribution can retain asymmetry at finite sample sizes, which is especially consequential for tail probabilities.

Definition: For independent, identically distributed observations with population mean $\mu$ and finite, positive population standard deviation $\sigma$, the standardized sample mean is $$ Z_n=\frac{\bar{x}-\mu}{\sigma/\sqrt{n}}. $$ A CLT-based normal approximation treats $Z_n$ as approximately standard normal, with mean 0 and standard deviation 1. This approximation is a guide to probabilities, not a claim that the finite-sample distribution is exactly normal.

For an event such as $\bar{x}>c$, first express its boundary on the standardized scale: $$ z_c=\frac{c-\mu}{\sigma/\sqrt{n}}. $$ The normal approximation to the event is then $P(Z>z_c)$, where $Z$ is standard normal. This standardization makes clear whether a probability is central or extreme; the same raw cutoff can be central for one sample size and far into a tail for another.

How Skewness Shrinks—and Why It Still Matters

If the population has a finite third absolute moment, its skewness is defined by $\gamma=E[(X-\mu)^3]/\sigma^3$. Under this additional condition, the skewness of $Z_n$ is $\gamma/\sqrt{n}$. Thus, increasing the sample size reduces the sampling distribution’s standardized asymmetry. This result helps explain why normal approximations often improve with $n$, but it does not say how accurate a particular tail probability will be.

A Short Derivation of the Skewness Result

Let $Y_i=(X_i-\mu)/\sigma$, so each $Y_i$ has mean 0, variance 1, and third moment $\gamma$. Independence gives $Z_n=(Y_1+\cdots+Y_n)/\sqrt{n}$. Expand the cube of the sum and take its expectation. Terms involving a factor that appears only once have expectation zero, because the $Y_i$ are independent and have mean zero. Only the $n$ terms $E(Y_i^3)$ remain:

E(Z_n^3) =\frac{1}{n^{3/2}}E\left[\left(\sum_{i=1}^{n}Y_i\right)^3\right] =\frac{n\gamma}{n^{3/2}} =\frac{\gamma}{\sqrt{n}}.

Since $Z_n$ has mean 0 and variance 1, its third standardized moment is its skewness. The finite third absolute moment condition ensures the moments used here exist. Finite mean and variance alone would not be enough to guarantee a defined skewness.

Worked Example: Skewness and a Standardized Cutoff

Suppose observations are independent exponential random variables with mean 1 hour. The population standard deviation is also 1 hour, and the population skewness is 2. For a sample of $n=25$, find the skewness of the standardized sample mean and standardize a sample-mean cutoff of 1.4 hours.

The exponential population has a finite third absolute moment, so the skewness result applies. The standardized sample mean has skewness $$ \frac{\gamma}{\sqrt{n}}=\frac{2}{\sqrt{25}}=0.4000. $$ Its distribution is still positively skewed according to this moment measure, although less so than an individual observation.

The standard error is $1/\sqrt{25}=0.2$ hour. The cutoff is $$ z_c=\frac{1.4-1}{0.2}=2. $$ Thus, a normal approximation would estimate the probability that the sample mean exceeds 1.4 hours by $P(Z>2)$, approximately 0.02275. The cutoff is two standard errors above the mean, so it concerns an upper-tail event, not a central one. The skewness calculation warns that the approximation may be imperfect; it does not by itself determine the exact tail probability or the direction of the error.

Worked Comparison: Central Interval and Upper Tail

A direct comparison makes the difference between central and tail accuracy visible. Suppose $X_1,\ldots,X_4$ are independent exponential observations with mean 1 unit, and let $S=X_1+\cdots+X_4$. Then $E(S)=4$ and $SD(S)=2$. The exact distribution of this sum has cumulative distribution function $$ F_S(s)=1-e^{-s}\left(1+s+\frac{s^2}{2}+\frac{s^3}{6}\right),\quad s\geq 0. $$ The CLT-based approximation treats $S$ as normal with mean 4 and standard deviation 2. Equivalently, $(S-4)/2$ is approximated by a standard normal variable.

Worked Example: A Central Probability and a More Extreme Tail

Compare the exact and normal-approximation probabilities for (a) $2<S<6$, a central interval, and (b) $S>8$, an upper-tail event. The observations are independent, have finite variance, and have an exponential distribution with positive skewness. These conditions permit a CLT approximation, but do not guarantee close tail accuracy for this sample size.

(a) Central interval. The exact probability is $$ \begin{aligned} P(2<S<6) &=F_S(6)-F_S(2)\\ &=(1-61e^{-6})-\left(1-\frac{19}{3}e^{-2}\right)\\ &\approx 0.8487961-0.1428765\\ &=0.7059196. \end{aligned} $$ The standardized endpoints are $(2-4)/2=-1$ and $(6-4)/2=1$. The normal approximation is therefore $$ P(-1<Z<1)=\Phi(1)-\Phi(-1)\approx 0.6826895. $$ Here the normal approximation underestimates the exact central probability by about $0.7059196-0.6826895=0.02323$.

(b) Upper tail. At $s=8$, the exact survival probability is $$ \begin{aligned} P(S>8) &=e^{-8}\left(1+8+\frac{8^2}{2}+\frac{8^3}{6}\right)\\ &=\frac{379}{3}e^{-8}\\ &\approx 0.04238. \end{aligned} $$ The standardized cutoff is $(8-4)/2=2$, so the normal approximation is $P(Z>2)\approx 0.02275$. It underestimates this tail probability by about $0.04238-0.02275=0.01963$. The error is substantial relative to the probability being estimated. In this example, the positive skewness leaves more probability in the far upper tail than the symmetric normal approximation suggests.

This comparison illustrates a tendency, not a universal rule. Skewness can help explain why a finite-sample distribution differs from a normal curve, but neither skewness alone nor the standardized cutoff determines the exact size or direction of an approximation error in every problem. The particular distribution and event matter.

A Bounded Population Can Expose Tail Errors

A different warning arises when the population is bounded. A normal curve has unbounded support, so it assigns positive probability beyond every finite cutoff. A sample mean from a bounded population, by contrast, cannot exceed the population’s upper bound. Near such a boundary, an extreme-tail normal approximation can assign probability to outcomes that are impossible.

Worked Example: A Central Interval and an Impossible Tail

Let $X_1,\ldots,X_4$ be independent observations uniformly distributed from 0 to 1. Then $\mu=0.5$, $\sigma=1/\sqrt{12}$, and the sample mean has standard error $1/\sqrt{48}\approx0.1443$. The independence and finite-variance conditions hold. Compare the probability that the mean lies between 0.25 and 0.75 with the probability it exceeds 1.

To calculate the exact central probability, write $T=X_1+\cdots+X_4$, so $\bar{x}=T/4$. For four independent uniform observations, the sum has cumulative distribution function $$ F_T(t)=\frac{t_+^4-4(t-1)_+^4+6(t-2)_+^4-4(t-3)_+^4+(t-4)_+^4}{24}, $$ where $u_+=\max(u,0)$. This formula follows by inclusion-exclusion: $t^4/24$ gives the volume in the nonnegative region below the sum limit, and the subtracted and added terms remove regions in which one or more coordinates exceed 1. Since $0.25<\bar{x}<0.75$ is equivalent to $1<T<3$, $$ P(0.25<\bar{x}<0.75)=F_T(3)-F_T(1)=\frac{23}{24}-\frac{1}{24}=\frac{11}{12}\approx0.9167. $$

For the normal approximation, the endpoints 0.25 and 0.75 are each $\sqrt{3}$ standard errors from 0.5. Thus $$ P(0.25<\bar{x}<0.75)\approx P(-\sqrt{3}<Z<\sqrt{3})\approx0.9167. $$ The central approximation is exceptionally close in this example. But $\bar{x}\leq1$ always, so $P(\bar{x}>1)=0$ exactly. The CLT normal approximation gives $$ P\left(Z>\frac{1-0.5}{1/\sqrt{48}}\right)=P(Z>\sqrt{12})\approx0.000266. $$ That positive estimate is wrong as an exact description: the event is impossible. This boundary case emphasizes why tail estimates deserve particular caution.

Interpreting an Approximation Carefully

The CLT is powerful because it describes how standardized sums and means behave as sample size increases. It does not say that a normal approximation is uniformly accurate at every finite sample size, nor that a small error in a central probability implies a small relative error in a rare-event probability. For example, an absolute error of 0.01 is modest for a probability near 0.5 but large compared with a true probability of 0.02.

When assessing a CLT-based probability, consider:

Conditions: Are the observations independent, identically distributed, and of finite variance, as required for the basic sample-mean CLT?
Shape: Is the population substantially skewed or otherwise unlike a normal distribution? If skewness exists, its effect on the standardized mean decreases at rate $1/\sqrt{n}$, but it need not vanish at the sample size in hand.
Location: How far is the event boundary from the mean in standard-error units? A distant cutoff asks for a tail probability, where a small distributional discrepancy can strongly affect the answer.
Support: Does the normal approximation assign probability to values that the original variable or sample mean cannot take?

These checks support a measured conclusion: a central probability may be adequately approximated even when a far-tail probability is not. They do not replace an exact calculation or other suitable method when one is available.

Common Mistakes

Assuming that the CLT makes every finite-sample distribution normal. The CLT gives an approximation whose quality depends on sample size and population shape.
Using the raw cutoff to judge extremeness. A cutoff is meaningful relative to the standard error; calculate its standardized value before calling it central or extreme.
Assuming skewness must always make the normal approximation underestimate the upper tail. Skewness is a clue about asymmetry, not a universal formula for the direction or size of a probability error.
Ignoring the size of the probability. A seemingly small absolute error can be consequential when the event is rare.
Forgetting that normal support is unbounded. If the sample mean has a finite possible range, check whether a proposed normal tail includes impossible outcomes.

Key takeaway: The CLT supports normal approximation for standardized means, but central and extreme-tail probabilities need not be equally accurate. Check the assumptions, the distribution’s shape, the standardized cutoff, and the support before relying on a tail estimate.

Check Your Understanding

Use the ideas in this tutorial to answer the following questions.

Why is a cutoff of three standard errors from the mean generally a greater concern for approximation accuracy than a cutoff near the mean?
For a population with skewness 1.5 and finite third absolute moment, what is the skewness of the standardized sample mean when $n=36$?
In the exponential-sum example, why does the normal approximation underestimate the exact probability $P(S>8)$?
Why can a normal approximation assign positive probability to an event that is impossible for a sample mean from a bounded population?
Does a close approximation to a central probability guarantee a close relative approximation to a rare tail probability? Explain.