Central Limit Problems For Sample Averages

Why Sample Averages Become Easier to Approximate

A sum of many independent random quantities can have a complicated distribution, even when each individual quantity has a simple or unfamiliar one. The central limit theorem provides a practical approximation: when the sample size is sufficiently large, a standardized sum, and therefore a standardized sample average, is approximately standard normal under specified conditions. This makes it possible to estimate tail and interval probabilities without deriving the exact distribution of the sum.

The central limit theorem does not say that every sample average is exactly normal, nor does it make the observations independent. The model and the scaling matter. In particular, the standard deviation of a sample average is smaller than the standard deviation of an individual observation by a factor of $\sqrt{n}$.

Definition: Let $X_1,\ldots,X_n$ be independent and identically distributed random variables with finite mean $E[X_i]=\mu$ and finite, positive variance $\operatorname{Var}(X_i)=\sigma^2$. Their sum and sample average are $$ S_n=\sum_{i=1}^n X_i,\qquad \bar{X}_n=\frac{S_n}{n}. $$ The central limit theorem states that, as $n$ grows, the distributions of $$ \frac{S_n-n\mu}{\sigma\sqrt{n}} \quad\text{and}\quad \frac{\bar{X}_n-\mu}{\sigma/\sqrt{n}} $$ are approximated by the standard normal distribution. The approximation is used for sufficiently large $n$; how large is sufficient depends in part on the shape of the distribution of each $X_i$.

Write $\Phi(z)=P(Z\le z)$, where $Z\sim N(0,1)$. Once an event has been expressed in standard-deviation units, $\Phi$ gives the corresponding normal probability. For a sample average, the standard deviation used in this standardization is the standard error, $\sigma/\sqrt{n}$, not $\sigma$.

Means, Variances, and the Standardization

Under the independent, identically distributed model above, linearity of expectation gives $E[S_n]=n\mu$. Independence makes the variance of the sum equal to the sum of the individual variances, so $\operatorname{Var}(S_n)=n\sigma^2$. Dividing the sum by $n$ divides its mean by $n$ and its variance by $n^2$.

E[\bar{X}_n]=\mu,\qquad \operatorname{Var}(\bar{X}_n)=\frac{\sigma^2}{n},\qquad \operatorname{SD}(\bar{X}_n)=\frac{\sigma}{\sqrt{n}}.

These mean and variance identities are exact for every $n$. It is the normal approximation to the distribution—not these identities—that depends on the central limit theorem and a sufficiently large sample size. For a sum, the corresponding exact mean and standard deviation are $n\mu$ and $\sigma\sqrt{n}$.

Theorem: If the conditions for the central limit theorem are reasonable, then for fixed $a<b$, $$ P(a<\bar{X}_n<b)\ \approx\ \Phi\!\left(\frac{b-\mu}{\sigma/\sqrt{n}}\right) -\Phi\!\left(\frac{a-\mu}{\sigma/\sqrt{n}}\right). $$ For an upper-tail event, $P(\bar{X}_n>b)\approx 1-\Phi\!\left((b-\mu)/(\sigma/\sqrt{n})\right)$. For a sum, standardize using mean $n\mu$ and standard deviation $\sigma\sqrt{n}$. These are approximations unless the underlying variables are normal, in which case the sum and average are exactly normal.

Why the Standard Error Has a Square Root

The square root in the standard error follows directly from independence. If the observations were dependent, the variance of their sum would generally include covariance terms, so the calculation below would not automatically apply.

Start with one observation.
Each $X_i$ has variance $\sigma^2$, and the observations are independent.

Add the variances.
Independence gives $\operatorname{Var}(S_n)=\sum_{i=1}^n\operatorname{Var}(X_i)=n\sigma^2$.

Divide to obtain the average.
Because $\bar{X}_n=S_n/n$, the rule $\operatorname{Var}(cX)=c^2\operatorname{Var}(X)$ gives $\operatorname{Var}(\bar{X}_n)=n\sigma^2/n^2=\sigma^2/n$. Taking the square root gives $\sigma/\sqrt{n}$.

Thus, multiplying the sample size by four halves the standard error. It does not divide the standard error by four. This square-root relationship is central to both probability calculations and sample-size planning.

Worked Example: An Interval Probability for Average Repair Cost

Suppose hypothetical repair costs $X_1,\ldots,X_{100}$ are independent and identically distributed, each with mean $\$80$ and standard deviation $\$20$. The individual-cost distribution is not specified, but it has finite variance. Use the central limit theorem to estimate the probability that the average cost is between $\$78$ and $\$84$.

Method. Find the standard error of the average, standardize both endpoints, and subtract the lower-tail probability from the upper-tail probability.

The standard error is $20/\sqrt{100}=20/10=\$2$. Therefore the standardized lower and upper endpoints are $$ z_L=\frac{78-80}{2}=-1,\qquad z_U=\frac{84-80}{2}=2. $$

The central limit theorem gives $$ P(78<\bar{X}_{100}<84)\approx \Phi(2)-\Phi(-1). $$

Using unrounded normal probabilities, $\Phi(2)=0.9772499$ and $\Phi(-1)=0.1586553$, so the difference is $0.8185946$. Rounded to four decimals, the estimated probability is 0.8186.

Check. The interval contains the mean and extends two standard errors above it but only one below it, so a probability greater than one-half is reasonable. The common mistake is to subtract rounded table entries $0.9772-0.1587$, which gives $0.8185$. Subtracting the more precise values and rounding only the final result gives $0.8186$.

Working With Sums and Upper or Lower Tails

A question may be written in terms of a total rather than an average. The same method applies, but the total uses its own mean and standard deviation. Alternatively, an event involving a total can be divided by $n$ and rewritten as an event involving the average. Either route gives the same standardized value when the algebra is carried out consistently.

Worked Example: A Lower-Tail Probability for a Total

A hypothetical service center records the number of minor equipment requests received each day. Assume daily counts $X_1,\ldots,X_{64}$ are independent and identically distributed with mean $4.2$ and variance $6.25$. Estimate the probability that the total over 64 days is at most 250. Assume the count distribution has finite variance and use the central limit theorem.

Method. Standardize the total using $E[S_{64}]=n\mu$ and $\operatorname{SD}(S_{64})=\sigma\sqrt{n}$. Here $\sigma=\sqrt{6.25}=2.5$.

The total has mean $64(4.2)=268.8$ and standard deviation $2.5\sqrt{64}=2.5(8)=20$. The standardized boundary is $$ z=\frac{250-268.8}{20}=\frac{-18.8}{20}=-0.94. $$ Thus, $$ P(S_{64}\le 250)\approx\Phi(-0.94)=0.1736. $$

Check. The cutoff is $18.8$ below the mean, or $0.94$ standard deviations below it, so a probability below one-half is appropriate. The calculation estimates the requested probability as 0.1736, to four decimals. It does not determine the exact probability: the individual counts’ distribution has not been specified. If the individual distribution is strongly skewed or has unusual tail behavior, the normal approximation may be less accurate at this sample size.

For discrete sums, the exact event at a boundary can depend on whether that boundary is included and on which integer values are possible. Continuity corrections are a separate refinement for normal approximations to discrete probabilities; use them when the problem calls for that method. They do not change the standard-error calculation.

Choosing a Sample Size From a Desired Accuracy

The standard error formula can be rearranged to plan a sample size. Under a normal approximation, a central probability of approximately $1-\alpha$ within $c$ of the mean is achieved when $z_{1-\alpha/2}\sigma/\sqrt{n}\le c$, where $z_{1-\alpha/2}$ is the standard normal quantile with cumulative probability $1-\alpha/2$. Since $n$ must be a whole number, round the resulting lower bound up, not to the nearest integer.

Worked Example: Planning for a Narrow Average-Cost Band

A hypothetical portfolio has independent, identically distributed claim costs with mean $\$50$ and standard deviation $\$10$. Using the normal approximation, find the smallest sample size for which the average cost is within $\$2$ of its mean with probability at least approximately $0.95$. Assume the claim-cost distribution has finite variance and that the sample size sought is adequate for a normal approximation.

Method. A central $0.95$ standard normal interval has endpoints approximately $-1.96$ and $1.96$. The approximate requirement is $$ 1.96\frac{10}{\sqrt{n}}\le 2. $$

Rearranging gives $$ \sqrt{n}\ge \frac{1.96(10)}{2}=9.8,\qquad n\ge 9.8^2=96.04. $$ The smallest whole number satisfying this inequality is $n=97$.

Check. At $n=97$, the standard error is $10/\sqrt{97}$, approximately $1.0153$, and $1.96(1.0153)\approx1.990$, which is below $\$2$. At $n=96$, $1.96(10/\sqrt{96})\approx2.0004$, which is just above $\$2$. The required minimum is therefore 97 observations under the stated approximation. Rounding $96.04$ down would fail the inequality.

Common Exam Traps and Limits of the Approximation

Using the individual standard deviation for an average. For $\bar{X}_n$, use $\sigma/\sqrt{n}$. Using $\sigma$ gives a scale that is too large and can substantially distort the probability.
Using the average’s standard error for a sum. For $S_n$, use $\sigma\sqrt{n}$, not $\sigma/\sqrt{n}$. Check whether the event concerns a total or an average before standardizing.
Assuming the central limit theorem makes the observations normal. It approximates the distribution of a standardized sum or average for suitable large samples. It does not change the distribution of any individual $X_i$.
Ignoring model conditions. The variance-addition argument used here requires independence. Identical distribution and finite variance are the standard conditions for the stated version of the theorem. Dependence or infinite variance can invalidate this calculation or require different results.
Rounding too early. Keep standardized values and normal probabilities at adequate precision through the subtraction, then round the final probability to four decimals.

Key takeaway: First identify whether the random quantity is a sum or an average. For independent, identically distributed observations with mean $\mu$ and standard deviation $\sigma$, use mean $n\mu$ and standard deviation $\sigma\sqrt{n}$ for the sum, or mean $\mu$ and standard error $\sigma/\sqrt{n}$ for the average. Then standardize and use the normal approximation only when its assumptions and sample-size adequacy are reasonable. These techniques are aligned with the SOA Exam P (Probability) syllabus.

Check Your Understanding

For each problem, use a normal approximation when the stated assumptions support it, and round probability answers to four decimals.

Independent, identically distributed hypothetical inspection times have mean 30 minutes and standard deviation 6 minutes. For a sample of 36 inspections, estimate the probability that the average time exceeds 31.5 minutes. (Answer: 0.0668)
Independent, identically distributed daily counts have mean 2.5 and variance 4. For 100 days, estimate the probability that the total count exceeds 270. (Answer: 0.1587)
Independent, identically distributed hypothetical payments have mean $\$120$ and standard deviation $\$15$. For 25 payments, estimate the probability that the average is between $\$114$ and $\$126$. (Answer: 0.9545)
Independent, identically distributed measurements have standard deviation 9 units. Using the central normal approximation, what is the smallest sample size that makes the probability of the sample average lying within 1.5 units of its mean at least approximately $0.90$? (Answer: 98)
Independent, identically distributed hypothetical processing times have mean 5 minutes and variance 16. For 64 observations, estimate the probability that the average is between 4.5 and 5.75 minutes. (Answer: 0.7745)

What You'll Learn