A Simulation Study of Convergence to Normality

Why Simulate Convergence to Normality?

The Central Limit Theorem (CLT) describes a change in the distribution of sample means: under its conditions, means from sufficiently large independent samples have a distribution that is approximately normal, even when the population itself is not normal. A simulation makes this change visible. We choose a population, repeatedly draw samples from it, calculate a mean for each sample, and compare the resulting distributions at several sample sizes.

The aim is not to show that every individual sample becomes normal. The observations still come from the chosen population. It is the distribution of the repeated sample means that changes shape as sample size increases. A useful simulation also reveals a second feature: the sample means cluster more tightly around the population mean as sample size grows.

Definition: A simulation study of convergence to normality repeatedly draws samples from a specified population, calculates the sample mean for each sample size under study, and examines how the distributions of those means change as sample size increases. Each collection of simulated means approximates the sampling distribution of the sample mean at that sample size.

Specify the Population and the Sampling Process

A simulation needs a population that can be sampled from and whose important properties are known or specified. For illustration, use an exponential population with rate 1. Its values are nonnegative and strongly right-skewed, so it is clearly not normal. For \(X\) drawn from this population, the density is \(f(x)=e^{-x}\) for \(x\geq 0\), and its mean and standard deviation are both 1. In particular, its variance is finite.

For each sample size \(n\), draw \(n\) independent observations from this same population and calculate their mean, \(\bar{x}\). Repeat this process \(R\) times. The result is \(R\) simulated sample means for that value of \(n\). Then repeat the whole process for other sample sizes, keeping the population and the number of repetitions fixed so the comparisons are interpretable.

The assumptions matter: the simulated observations must follow the population model specified, and each sample should consist of independent draws. The standard CLT result for sample means applies to independent observations from a population with finite mean and finite, nonzero variance. The exponential population meets those conditions. The earlier material on the CLT gives the theorem and its conditions; here, simulation provides a way to see the changing distributions.

Choose the population.
Write down its distribution and, if possible, its mean \(\mu\) and standard deviation \(\sigma\). Confirm that it is nonnormal if the purpose is to illustrate convergence from a nonnormal population.

Set sample sizes and repetitions.
Choose several values of \(n\) and a reasonably large number \(R\) of repeated samples for each value. Use the same \(R\) across sample sizes when making direct comparisons.

Generate and summarize samples.
For each of the \(R\) repetitions, draw \(n\) independent observations and save their mean. This creates one simulated sampling distribution for each \(n\).

Plot and interpret.
Use histograms to compare shape and spread. Consider both the original sample means and standardized means, which make shape easier to compare across different sample sizes.

Worked Example: One Simulated Sample Mean

Suppose one hypothetical simulation run draws four observations from the rate-1 exponential population: \(0.22,\ 0.51,\ 1.20,\ 2.30\). Find the sample mean and standardize it using the population mean and standard deviation.

The sample mean is

\bar{x}=\frac{0.22+0.51+1.20+2.30}{4} =\frac{4.23}{4}=1.0575.

For this population, \(\mu=1\), \(\sigma=1\), and \(n=4\). The standard deviation of the sample mean is \(\sigma/\sqrt{n}=1/\sqrt{4}=0.5\). The standardized mean is therefore

z=\frac{\bar{x}-\mu}{\sigma/\sqrt{n}} =\frac{1.0575-1}{0.5}=0.115.

This one result is only one point in a simulated sampling distribution; it does not establish that the distribution is normal. Repeating the draw many times produces the collection of means to plot. Standardizing expresses each mean in units of its sampling standard deviation, so standardized histograms at different sample sizes can be compared on a common scale.

What the Histograms Should Show

For a population with mean \(\mu\) and standard deviation \(\sigma\), the sample mean from \(n\) independent observations has mean \(\mu\) and standard deviation \(\sigma/\sqrt{n}\). The CLT says that, as \(n\) increases, the distribution of the standardized sample mean is approximately standard normal:

\frac{\bar{x}-\mu}{\sigma/\sqrt{n}} \ \approx\ N(0,1).

The symbol \(N(0,1)\) denotes a normal distribution with mean 0 and variance 1. The approximation concerns the distribution across repeated samples, not the shape of the original population or the appearance of any one sample.

There is a simple reason the spread of the means decreases. If the observations are independent and each has variance \(\sigma^2\), the variance of their sum is \(n\sigma^2\). Dividing that sum by \(n\) to form the sample mean divides its variance by \(n^2\). Thus:

\operatorname{Var}(\bar{x}) =\operatorname{Var}\left(\frac{X_1+\cdots+X_n}{n}\right) =\frac{n\sigma^2}{n^2} =\frac{\sigma^2}{n}, \qquad SE(\bar{x})=\frac{\sigma}{\sqrt{n}}.

This establishes the spread result; the CLT supplies the result about approximate normal shape. In practice, plot the original sample means to see both features: increasing symmetry and decreasing spread. Also plot standardized means to assess shape without letting the shrinking spread dominate the comparison. Use the same horizontal scale and comparable bins within each set of plots; otherwise, differences in plotting choices can be mistaken for differences in distribution.

Worked Example: Comparing Several Sample Sizes

Consider a hypothetical simulation from the rate-1 exponential population, using \(R=10{,}000\) repetitions at each of \(n=1,\ 4,\ 16\). Before looking at the histograms, calculate the expected center and standard error for each sampling distribution.

The population has \(\mu=1\) and \(\sigma=1\). For \(n=1\), the mean of repeated sample means is 1 and their standard deviation is

SE(\bar{x})=\frac{1}{\sqrt{1}}=1.

For \(n=4\), the center remains 1, while the standard error is \(1/\sqrt{4}=0.5\). For \(n=16\), the center is still 1 and the standard error is \(1/\sqrt{16}=0.25\). The expected spread therefore shrinks by a factor of 2 each time \(n\) increases fourfold.

The \(n=1\) histogram should retain the population’s strong right skew. The \(n=4\) histogram is expected to be less skewed, although it may still show noticeable asymmetry. The \(n=16\) histogram should generally be more symmetric and mound-shaped, and narrower on the original scale. These are qualitative expectations, not guaranteed descriptions of every finite simulation. The standardized histograms should be centered near 0 and have standard deviation near 1; their shapes are the more direct comparison with the standard normal curve.

A useful additional shape check for this particular population is its theoretical skewness. The exponential population has skewness 2, and the skewness of a mean of \(n\) independent exponential observations is \(2/\sqrt{n}\). Thus the expected skewness values are 2 at \(n=1\), 1 at \(n=4\), and 0.5 at \(n=16\). This calculation is consistent with the expected movement toward symmetry, while reminding us that convergence is gradual.

Simulation Variation and Interpretation

A simulated histogram is not the exact sampling distribution. It is an estimate based on \(R\) repetitions, so its bars vary from run to run. With too few repetitions, a roughly normal distribution may look irregular, or a small apparent tail may be caused by chance. Increasing \(R\) makes the histogram a more stable representation of the sampling distribution. Increasing \(n\), by contrast, changes the sampling distribution itself. These are different adjustments: repetitions improve the simulation’s resolution, while sample size affects the shape and spread being studied.

For example, if \(R=10{,}000\) simulated means are used to estimate the fraction below a specified cutoff and 8,400 fall below it, the estimated fraction is \(8{,}400/10{,}000=0.84\). The simulation result is still subject to Monte Carlo variation. Treating each repetition as an independent indicator of whether the mean falls below the cutoff, the estimated standard error of this fraction is

SE(\hat{p})\approx \sqrt{\frac{0.84(1-0.84)}{10{,}000}} =\sqrt{0.00001344} \approx 0.00367.

This calculation describes uncertainty from using a finite number of simulation repetitions; it is not the standard error of the sample mean in the original sampling process. It helps explain why a reported simulated fraction should not be treated as exact.

Worked Example: Is an Irregular Histogram Evidence Against the CLT?

Suppose a hypothetical run uses \(R=100\) samples of size \(n=16\) from the rate-1 exponential population. Its standardized-mean histogram is centered near 0 but has a lopsided set of bars in the right tail. Should this alone be taken as evidence that the CLT prediction has failed?

No. First, \(R=100\) is a limited number of repetitions, so the bars can fluctuate substantially. Second, the CLT predicts an approximate distribution for a fixed sample size, not a perfectly smooth histogram from a finite simulation. Third, the exponential population is skewed, so some remaining asymmetry at \(n=16\) is plausible.

A more informative check is to repeat the simulation with a much larger \(R\), without changing the population or \(n\). If the broad pattern persists, compare the standardized histogram with a normal curve and report the remaining difference rather than expecting exact agreement. Then examine larger sample sizes as a separate comparison. A smoother histogram after increasing \(R\) reflects less simulation noise; a shape change after increasing \(n\) reflects a different sampling distribution.

Common Mistakes and Why the Study Matters

Looking at the raw observations instead of the sample means. The original data remain exponential in this example. To study the CLT, plot one mean per repeated sample.
Claiming every sample becomes normal. The theorem concerns the distribution of sample means over repeated sampling, not the data within an individual sample.
Confusing spread with shape. The raw means become less variable as \(n\) grows. Standardization helps compare shape while accounting for this changing scale.
Treating a finite simulation as a proof. A simulation illustrates a result under a specified model; it cannot establish a general theorem or guarantee that a finite histogram will look smooth.
Changing several design features at once. If the population, \(n\), and \(R\) all change together, it is difficult to tell which change caused a different plot. Vary one feature at a time when possible.

Simulation is valuable because it connects a mathematical result to an observable pattern. It can show how the population’s skewness is reflected in small-sample means, how larger samples tend to produce more nearly normal distributions, and how the standard error shrinks. It also makes the limits of a finite illustration visible: the result is approximate, and every simulated plot contains randomness.

Key takeaway: To study convergence to normality, repeatedly sample from one specified nonnormal population, calculate a mean for every sample, and compare mean distributions across sample sizes. Interpret both shape and spread, use standardization for fair shape comparisons, and remember that finite simulation adds its own random variation.

Check Your Understanding

Use the population and simulation ideas in this tutorial to answer the following questions.

In a simulation of the CLT, what should be plotted: the original observations or one sample mean from each repetition? Explain.
For a population with \(\sigma=6\), find the standard error of the sample mean when \(n=9\). What happens to this standard error if \(n\) increases to 36?
Why can standardizing the simulated sample means help compare their shape across sample sizes?
A histogram based on 50 repetitions looks irregular. Name one reason this may happen that does not imply the CLT is false.
In a simulation study, what does increasing the number of repetitions change, and what does increasing the sample size change?