Why Simulate Sample Means?
A population can be strongly skewed even when the statistic of interest is the sample mean. To understand what repeated sample means look like, we can specify a population, repeatedly draw samples from it, and calculate the mean of each sample. The resulting collection of means is a simulated sampling distribution: it gives a practical picture of the values the statistic can take under the stated sampling model.
The simulation is useful because one sample mean alone does not show how much sample means vary. A second sample from the same population may produce a different mean, especially when a few unusually large population values are possible. Comparing simulations at different sample sizes helps make this variation visible. The results are random: a particular run illustrates the model but does not determine what every run must look like.
Specify the Population and the Sampling Process
Consider a hypothetical population with four possible values. Most observations are small, but a value of 10 occurs occasionally, producing a long right tail.
| Population value | Probability |
|---|---|
| 0 | 0.60 |
| 1 | 0.20 |
| 2 | 0.10 |
| 10 | 0.10 |
These probabilities sum to 1. The population mean, denoted by \(\mu\), is the probability-weighted average of the possible values:
The population standard deviation is about 2.94, so individual values vary considerably around the mean. The value 10 is especially influential: it occurs with probability 0.10, but it is much larger than the other outcomes.
For the examples here, each draw is made with replacement. Each draw is independent of the others and has the probabilities in the table, even if the same value has just been drawn. A sample of size \(n\) consists of \(n\) such draws. Repeating the sampling process produces a new sample and a new sample mean each time.
Worked Example: Simulating and Checking Individual Sample Means
Worked Example: Simulating and Checking Individual Sample Means
Suppose a simulation generates two samples of size 10 from the population above. For sample D, the outcomes include six 0s, two 1s, no 2s, and two 10s. For sample H, they include five 0s, three 1s, one 2, and one 10. Calculate both means.
For sample D, the counts add to \(6+2+0+2=10\), as required. Its sum is the count of each value multiplied by that value:
Therefore, the sample mean for D is \(22/10=2.20\). In sample H, the counts also add to 10. Its sum and mean are:
The two means are different even though both samples came from the same population and have the same size. The large value 10 contributes 20 of D's 22 total points and 10 of H's 15. This illustrates why an occasional tail value can move a sample mean substantially.
From Repeated Samples to a Distribution
A simulation run might produce 10 sample means at \(n=10\) equal to 0, 0.5, 0, 1.5, 5, 0.5, 2, 0.5, 0, and 5.5. Each is the mean of a separate sample of 10 draws. To find the average of these simulated means, add them and divide by the number of repetitions:
The simulated means in this run average 1.55, compared with the population mean of 1.40. That difference is not an arithmetic error or proof that the sample mean systematically overestimates the population mean. It is one possible result from a limited number of random repetitions. A different run would generally produce different sample means and a different average of those means.
Worked Example: Reading a Small Simulation Run
For the 10 simulated means just listed, find how many are below 1, how many are at least 5, and what proportion of the simulated means are at least 5.
The means below 1 are 0, 0.5, 0, 0.5, 0.5, and 0: six values. The means at least 5 are 5 and 5.5: two values. Thus the observed fraction at least 5 is:
This says that 2 of the 10 simulated sample means in this run were at least 5. It does not establish that the true probability of a sample mean at least 5 is 0.20; with only 10 repetitions, the observed fraction can be quite variable. The result is best read as a description of this particular run.
A frequency table or graph can make a larger run easier to inspect. For example, one can group the simulated means into ranges and count how many fall in each range. The horizontal scale must represent sample means, not individual population values. A histogram of the original draws would answer a different question from a histogram of the repeated means.
Comparing Two Sample Sizes
To compare sample sizes, use the same population model and the same number of simulation repetitions for each \(n\). The following hypothetical small runs each contain 12 simulated sample means. The numbers are illustrative; a computer simulation with a large number of repetitions would provide a more detailed picture.
| Sample size | 12 simulated sample means | Average of these means | Observed range |
|---|---|---|---|
| \(n=2\) | 0, 0.5, 5, 1, 0.5, 6, 0, 1.5, 5.5, 0.5, 1, 0 | 1.79 | 0 to 6 |
| \(n=10\) | 0.8, 1.2, 0.5, 1.5, 2.2, 0.9, 1.4, 1.1, 2.4, 0.7, 1.3, 0.9 | 1.24 | 0.5 to 2.4 |
Worked Example: Comparing Simulated Means Across Sample Sizes
Use the table to compare the centers and observed ranges for the two runs. Then explain what can and cannot be concluded.
For \(n=2\), the 12 means sum to 21.5, so their average is \(21.5/12\), or about 1.79. The smallest is 0 and the largest is 6, giving a range of \(6-0=6\).
For \(n=10\), the 12 means sum to 14.9, so their average is \(14.9/12\), or about 1.24. The smallest is 0.5 and the largest is 2.4, giving a range of \(2.4-0.5=1.9\).
In these particular runs, the \(n=10\) means occupy a narrower range than the \(n=2\) means. The larger-sample means also include no value as extreme as 5, 5.5, or 6, which appear in the \(n=2\) run. This is consistent with the idea that averaging more observations tends to reduce sample-mean variability. It is not a guarantee that every larger-sample simulation will have a narrower range: random runs can differ, and the range depends on both the model and the finite number of repetitions.
The average of the 12 means is not exactly the population mean in either run. The \(n=2\) average is about 1.79 and the \(n=10\) average about 1.24, while \(\mu=1.40\). Small simulation runs can deviate from 1.40 by chance. Comparing the two centers is therefore not, by itself, a reliable way to decide which sample size has the greater spread.
Why Rare Values Still Matter
A simulation should preserve the specified probabilities, including probabilities of rare outcomes. It would be misleading to remove the value 10 simply because it does not appear in every sample. For instance, with \(n=10\), the probability that every draw is 0 is the probability of getting a 0 on each of 10 independent draws:
If every draw is 0, the sample mean is 0. Similarly, the probability that all 10 draws are 10 is:
That outcome is extremely unlikely, but it is allowed by the model. More often, a sample contains a mixture of values. Occasionally, it contains enough large values to pull its mean noticeably upward. Repeated simulation helps display this range of possible outcomes; a short run may fail to show some rare possibilities simply because it includes relatively few repetitions.
A Practical Simulation Workflow
A careful simulation keeps the model, the sample size, and the calculated statistic distinct. The following sequence works whether the draws are made by hand with random numbers or by software.
List every possible population value and its probability. Check that the probabilities sum to 1.
State \(n\) and specify how one draw is generated. Here, each draw is independent and uses the same probabilities.
Record the \(n\) outcomes, add them, and divide by \(n\). Check that the number of outcomes is exactly \(n\).
Repeat the sampling procedure, keeping one mean per sample. Describe the simulated distribution using a graph, a frequency table, or appropriate summaries.
When comparing sample sizes, keep the population model and number of repetitions fixed where possible. Describe what the simulation shows without treating a small run as a universal rule.
Common Mistakes
- Mixing up individuals and means. The population table describes individual draws. The simulated sampling distribution contains one mean per repeated sample.
- Using incorrect count arithmetic. For a sample with six 0s, two 1s, no 2s, and two 10s, the sum is 22 and the mean for \(n=10\) is 2.2. Check each count times its value before dividing.
- Dividing by the wrong number. Divide a sample total by its sample size to find that sample's mean. Divide the sum of simulated means by the number of repetitions to find their average.
- Treating a simulation fraction as an exact probability. If 2 of 10 simulated means exceed a threshold, the observed fraction is 0.20. It is not automatically the exact probability under the model.
- Expecting every run to center exactly at \(\mu\). Random samples vary. A finite collection of simulated means can average above or below the population mean.
- Assuming every larger-sample run must look narrower. In a particular comparison, random variation can obscure the general tendency. Report the observed comparison and avoid turning it into an absolute rule.
Check Your Understanding
Answer each question using the population model and simulation ideas from this tutorial.
- For the population with values 0, 1, 2, and 10 and probabilities 0.60, 0.20, 0.10, and 0.10, verify that the probabilities sum to 1 and calculate \(\mu\).
- A sample of size 10 has five 0s, three 1s, one 2, and one 10. Find its sum and sample mean.
- In a run of 50 simulated sample means, 8 are at least 5. What observed fraction is at least 5, and why should it not automatically be reported as the exact probability?
- Why might a small simulation run have a mean of its simulated means noticeably different from the population mean of 1.40?
- When comparing simulations at \(n=2\) and \(n=10\), name two features of the setup that should be held fixed for a useful comparison.