Simulating the Sampling Distribution of a Sample Mean

Why Simulate a Sampling Distribution?

A sample mean is random because its value depends on which observations enter the sample. Its sampling distribution describes the possible values of that statistic across repeated samples of the same size from the same population. Sometimes that distribution can be derived exactly; simulation gives a practical way to explore it when an exact distribution is cumbersome or unavailable.

The key is to repeat the whole sampling process, not merely to generate one large data set and inspect its observations. Each repetition must use a new sample of the specified size. The mean from each sample is then one observation from the empirical approximation to the sampling distribution.

Definition: Let $X_1,\ldots,X_n$ be an IID sample from a population with mean $\mu$ and finite variance $\sigma^2$. The sample mean is $\bar X=\frac{1}{n}\sum_{j=1}^{n}X_j$. In a simulation with $R$ repetitions, generate $R$ independent samples, each of size $n$, and record their means $\bar X_1^*,\ldots,\bar X_R^*$. The empirical sampling distribution assigns probability $1/R$ to each recorded mean, including repeated values.

The asterisk marks a simulated result; it does not define a different statistic. The population model and sample size determine the true sampling distribution. The finite list of simulated means is only an approximation to that distribution, and its particular values also depend on the random draws.

Designing and Summarizing the Simulation

Before running a simulation, specify the population distribution, the sample size $n$, the number of repetitions $R$, and how the random draws will be made. Within each repetition, the $n$ observations must be independent and have the stated population distribution. Repetitions should also use fresh independent draws. A fixed random-number seed can make a computer run reproducible, but it does not make the simulated means equal to theoretical values.

For a discrete population, a simulator can draw values according to their probabilities. For a continuous population, it can generate values from the specified density or use an appropriate random-number method. In either case, record one mean per sample. A frequency table or histogram of those $R$ means displays the empirical distribution.

The empirical center and spread can be computed directly. The empirical variance of the simulated distribution uses denominator $R$, because it describes the spread of the $R$ equally weighted points in that empirical distribution. The usual sample variance with denominator $R-1$ instead estimates the variance of the underlying sampling distribution from those $R$ simulated means.

\bar{\bar X}^{\,*}=\frac{1}{R}\sum_{r=1}^{R}\bar X_r^*, \qquad V_{\mathrm{emp}}=\frac{1}{R}\sum_{r=1}^{R}(\bar X_r^*-\bar{\bar X}^{\,*})^2, \qquad s_{\bar X^*}^{\,2}=\frac{1}{R-1}\sum_{r=1}^{R}(\bar X_r^*-\bar{\bar X}^{\,*})^2.

Here $\bar{\bar X}^{\,*}$ is the average of the simulated means. If the population has mean $\mu$ and finite variance $\sigma^2$, then the exact sampling distribution has mean $\mu$ and variance $\sigma^2/n$. These theoretical results provide useful checks on a simulation; they do not require the simulated summaries to match exactly.

Why the theoretical variance is $\sigma^2/n$: Since the observations are independent, the variance of their sum is the sum of their variances. Therefore, $$ \operatorname{Var}(\bar X) =\operatorname{Var}\left(\frac{1}{n}\sum_{j=1}^{n}X_j\right) =\frac{1}{n^2}\sum_{j=1}^{n}\operatorname{Var}(X_j) =\frac{n\sigma^2}{n^2} =\frac{\sigma^2}{n}. $$ The same linearity gives $E[\bar X]=\frac{1}{n}\sum_{j=1}^{n}E[X_j]=\mu$. Independence is essential to the variance calculation as written; with dependent observations, covariance terms may change the result.

Worked Simulations

Worked Example: A Small Simulation From a Three-Point Population

A hypothetical inspection score $X$ is equally likely to be $0$, $2$, or $4$. Simulate $R=6$ IID samples of size $n=2$, with the generated pairs $(0,0),(0,2),(2,4),(4,4),(2,0),(4,0)$. Find the empirical distribution of the sample mean, its center and variance, and compare them with the exact theoretical values.

Method. Compute one mean from each pair. The six results, in order, are $0,1,3,4,1,2$. Thus the empirical distribution has frequencies $1,2,1,1,1$ at means $0,1,2,3,4$, respectively. Its relative frequencies are those counts divided by $6$.

The simulated center is $$ \bar{\bar X}^{\,*}=\frac{0+1+3+4+1+2}{6}=\frac{11}{6}=1.8333. $$

The average of the squared simulated means is $$ \frac{0^2+1^2+3^2+4^2+1^2+2^2}{6} =\frac{31}{6}. $$ Subtracting the square of the simulated center gives the empirical variance: $$ V_{\mathrm{emp}}=\frac{31}{6}-\left(\frac{11}{6}\right)^2 =\frac{65}{36}=1.8056. $$

For the exact comparison, the population mean is $\mu=(0+2+4)/3=2$. Its variance is $$ \sigma^2=\frac{(0-2)^2+(2-2)^2+(4-2)^2}{3}=\frac{8}{3}. $$ So the exact sample-mean mean is $2$, and its variance is $(8/3)/2=4/3=1.3333$. The simulated center and variance differ from these values, which is plausible with only six repetitions.

Check. The six relative frequencies sum to $6/6=1$. The discrepancy in the empirical variance is not a calculation error: the simulation is small. A common mistake would be to treat the six simulated means as all possible sample means. They are only the six outcomes generated in this run.

Worked Example: Simulating a Bernoulli Sample Mean

In a hypothetical screening model, each independent test is positive with probability $p=0.30$. For $n=4$ tests per repetition, a simulation with $R=10$ repetitions produces these success counts: $$ 0,\ 1,\ 2,\ 1,\ 3,\ 2,\ 0,\ 1,\ 2,\ 1. $$ Calculate the empirical distribution of the sample proportion, its center and variance, and the corresponding exact values.

Method. Each sample proportion is its success count divided by $4$. The counts $0,1,2,3$ occur $2,4,3,1$ times, so the proportions $0,0.25,0.50,0.75$ have empirical probabilities $0.2,0.4,0.3,0.1$, respectively.

The sum of the ten counts is $13$, so the simulated center of the proportions is $$ \bar{\bar X}^{\,*}=\frac{13}{10\cdot4}=0.325. $$ The average squared count is $$ \frac{2(0^2)+4(1^2)+3(2^2)+1(3^2)}{10} =\frac{25}{10}=2.5. $$ The count mean is $13/10=1.3$, giving empirical count variance $2.5-(1.3)^2=0.81$. Since each proportion is its count divided by $4$, its empirical variance is $$ V_{\mathrm{emp}}=\frac{0.81}{4^2}=0.050625. $$

For the exact comparison, a Bernoulli observation has mean $0.30$ and variance $0.30(0.70)=0.21$. Thus the sample proportion has exact mean $0.30$ and variance $0.21/4=0.0525$. The simulated center $0.325$ and variance $0.050625$ are close here, but a different set of ten repetitions could be less close.

Check. The empirical probabilities sum to $0.2+0.4+0.3+0.1=1$. A common mistake is to report the variance of the success count, $0.81$, as the variance of the sample proportion. Dividing the count by $4$ scales its variance by $4^2$, so the proportion variance is $0.81/16$.

Worked Example: Reading Shape From Simulated Continuous Means

Suppose claim processing times are modeled as IID exponential random variables with mean $1$ hour and variance $1$ hour squared. A random-number generator produces six samples of size $3$, shown below in hours. Find the six sample means, their empirical center and variance, and compare their spread with the exact theoretical spread.

Repetition	Three generated times (hours)	Sample mean (hours)
1	0.2, 0.7, 1.1	0.6667
2	0.4, 0.9, 2.1	1.1333
3	0.1, 1.3, 3.0	1.4667
4	0.6, 0.8, 1.6	1.0000
5	0.3, 2.0, 2.7	1.6667
6	0.5, 1.0, 4.0	1.8333

Method. For example, the first mean is $(0.2+0.7+1.1)/3=2.0/3=0.6667$ hours. The other five means are calculated the same way. Their exact sum, using the unrounded means, is $$ \frac{2}{3}+\frac{17}{15}+\frac{22}{15}+1+\frac{5}{3}+\frac{11}{6} =\frac{233}{30}. $$ Therefore, the empirical center is $(233/30)/6=233/180=1.2944$ hours.

The average of their squares is $$ \frac{(2/3)^2+(17/15)^2+(22/15)^2+1^2+(5/3)^2+(11/6)^2}{6} =\frac{9917}{5400}=1.8365. $$ Subtracting the squared center gives $$ V_{\mathrm{emp}}=\frac{9917}{5400}-\left(\frac{233}{180}\right)^2 =0.1609\ \text{hours}^2. $$ The exact sample-mean variance is $1/3=0.3333$ hours squared, because $n=3$. The small simulation happens to have less spread than the theoretical distribution.

Check. The input times are positive, as required by the exponential model, and the sample means are in hours, not minutes. A common mistake is to infer from these six points that the true sampling distribution has variance $0.1609$. That is the empirical variance for this particular run; the model's exact variance remains $1/3$ hours squared.

Interpreting the Empirical Distribution

The empirical distribution is useful for seeing which sample means occurred often in a simulation and for estimating probabilities. For example, the empirical probability of a set $A$ is the number of simulated means in $A$, divided by $R$. It is an estimate of the true probability, not a guaranteed exact value. Increasing $R$ gives more simulated results on which to base the estimate, but does not change the population model or the theoretical sampling distribution.

Histograms can make the distribution's shape easier to see, especially when many repetitions produce a wide range of means. Their appearance depends partly on bin width and bin boundaries. With few repetitions, a histogram can look irregular simply because only a few outcomes were recorded. For a discrete sample mean, a frequency table often preserves the possible values and their counts more clearly than a histogram.

Keep three sources of information separate:

The model: the population distribution and the sample size determine the true sampling distribution.
The simulation: random draws produce a finite empirical distribution that may differ from the truth.
The theoretical checks: when the population has finite mean and variance and the sample is IID, the sample mean has mean $\mu$ and variance $\sigma^2/n$.

Simulation is especially helpful for investigating a sampling distribution whose exact form is difficult to derive. It should not be used to claim a shape that the evidence does not support, or to replace exact results that are already available. In particular, a histogram that looks roughly symmetric in a small run is not by itself proof that the true distribution is normal.

Common Errors to Avoid

Simulating one sample rather than repeated samples. One sample gives one realized mean, not a distribution of sample means.
Changing the sample size between repetitions. The sampling distribution being studied must correspond to the stated fixed $n$.
Mixing up the two variances. The population variance is $\sigma^2$; the exact sample-mean variance is $\sigma^2/n$. Scaling observations by $1/n$ scales variance by $1/n^2$, while summing $n$ independent variances contributes a factor of $n$.
Using $R-1$ for the empirical distribution's variance. Use $R$ to describe the variance of the $R$ equally weighted simulated means. Use $R-1$ when calculating the conventional sample variance to estimate the underlying sampling variance.
Expecting perfect agreement with theory. A simulation is a finite random run. Report its results as empirical estimates and compare them with theoretical values rather than treating small differences as contradictions.

Key takeaway: A simulation of a sample-mean distribution requires many fresh IID samples of the same size, one recorded mean per sample, and careful separation of empirical results from exact theoretical properties. The empirical center and spread are informative checks, not replacements for the model.

Check Your Understanding

For each problem, distinguish the simulated results from the exact sampling-distribution properties.

A population takes values $1$ and $3$, each with probability $1/2$. Four simulated means from samples of size $2$ are $1,2,2,3$. Calculate their empirical center and variance using denominator $R$, then compare with the exact mean and variance of the sample mean. (Answer: empirical center $2$, empirical variance $0.5$; exact mean $2$, exact variance $0.5$.)
A Bernoulli population has success probability $0.40$. For samples of size $5$, eight simulated success counts are $1,2,0,3,2,1,4,3$. Calculate the empirical center and variance of the sample proportions, then give their exact theoretical center and variance. (Answer: empirical center $0.40$, empirical variance $0.0600$; exact center $0.40$, exact variance $0.0480$.)
A population has mean $10$ and variance $36$. A simulation uses $n=4$ and $R=20$. State the exact mean and variance of the sample mean, and explain what denominator to use when computing the variance of the 20 equally weighted simulated means. (Answer: exact mean $10$, exact variance $9$; use denominator $20$ for the empirical distribution's variance.)
A hypothetical model has IID exponential observations with mean $2$ minutes. A simulation uses samples of size $8$ and records $500$ means. Find the exact center and variance of the sample-mean distribution, and state what an empirical histogram can and cannot establish. (Answer: exact center $2$ minutes and variance $4/8=0.5$ minutes squared; the histogram displays the simulated approximation but cannot by itself establish the exact shape or exact probabilities.)

What You'll Learn