From One Sample to a Sampling Model
A single random sample produces one value of a statistic. A different sample of the same size, drawn under the same plan, may produce a different value. Repeating the sampling process lets us see how the statistic varies when the population and sampling plan are held fixed. This is useful when the statistic’s behavior is difficult to anticipate from one sample alone.
A sampling distribution describes the values a statistic takes across all possible samples of a specified size drawn under a specified sampling plan. When we simulate only some of those samples, the resulting distribution is an approximation: it displays the simulated statistic values and their frequencies. Its center, spread, and shape describe the simulation, and may provide evidence about the corresponding sampling distribution.
The statistic might be any defined summary calculated from a sample. In the examples below, we use the sample median, sample range, and sample maximum. The central idea is not tied to one statistic: specify how the sample is drawn, apply the same calculation to each sample, and examine the resulting values.
Specify the Model Before Simulating
A useful simulation starts with a clear model. State which population is being represented, how each sample is selected, the fixed sample size, and the statistic to calculate. Also decide whether sampling is with replacement or without replacement. With replacement, a selected population member goes back before the next selection, so it can appear more than once in a sample. Without replacement, a member cannot appear twice in the same sample.
For each repetition, draw a new sample using the same plan. The samples in different repetitions are new draws; even when sampling without replacement, the prohibition on repeats applies within each sample, not across separate repetitions. Calculate the statistic and record it, not the individual sample values as though they were the simulation’s main results.
Identify what values can be selected and the selection rules, including whether sampling is with or without replacement.
Fix the number of observations in each sample and state the exact calculation to apply to every sample.
Generate many random samples under the same plan. Calculate one statistic for each and keep a record of the results.
Use a frequency table, dotplot, or another suitable display to assess the simulated values’ center, spread, and shape.
A simulation should represent the sampling process relevant to the question. If the actual process selects distinct people, a with-replacement simulation is a different model. If the population has unequal selection chances, a simulation that gives every member the same chance would also fail to represent that process. Randomness in the simulation does not compensate for a poorly chosen model.
Worked Examples
Worked Example: Simulating Sample Medians
Consider a hypothetical population with five values: 2, 4, 6, 8, and 10. For each repetition, draw \(n=3\) values independently with replacement and calculate their median. Suppose ten simulated samples are:
(2, 4, 6), (10, 2, 8), (6, 6, 4), (8, 10, 10), (4, 2, 2), (6, 8, 4), (10, 6, 2), (4, 8, 8), (2, 10, 6), (8, 4, 10).
The median of the first sample is 4, since the ordered values are 2, 4, 6. The median of the second is 8, since its ordered values are 2, 8, 10. Applying the same calculation to all ten samples gives the sequence of simulated medians:
The frequency table is:
| Simulated median | Frequency |
|---|---|
| 2 | 1 |
| 4 | 1 |
| 6 | 4 |
| 8 | 3 |
| 10 | 1 |
The arithmetic mean of these ten simulated medians is
The simulated medians range from 2 to 10, and 6 occurs most often. These are descriptions of the ten simulated results, not exact facts about all possible sample medians. Ten repetitions provide only a limited view of the model.
Worked Example: Describing Simulated Sample Ranges
Suppose a hypothetical population consists of the values 1, 2, 3, 4, and 5. Draw pairs with replacement, so \(n=2\), and calculate each sample’s range: its largest value minus its smallest value. Eight simulated pairs are (1, 1), (2, 5), (4, 2), (3, 3), (5, 1), (4, 5), (2, 2), and (3, 5).
For the first pair, the range is \(1-1=0\); for the second, it is \(5-2=3\). The eight ranges, in order, are \(0, 3, 2, 0, 4, 1, 0, 2\). Their frequencies are 0: three, 1: one, 2: two, 3: one, and 4: one.
The arithmetic mean of the simulated ranges is
The sorted values are \(0, 0, 0, 1, 2, 2, 3, 4\), so the median of the eight simulated ranges is the average of the fourth and fifth values:
In this small simulation, the ranges span 0 to 4, and zero is the most frequent result. The frequency pattern is uneven, but eight results are too few for a confident description of the distribution’s overall shape. The calculation describes only the simulated repetitions.
Worked Example: Simulating Sample Maxima
A hypothetical population has values 5, 7, 8, 11, and 14. Draw samples of size \(n=2\) with replacement and record the maximum from each sample. For the ten simulated samples (5, 7), (14, 8), (11, 5), (8, 8), (7, 14), (5, 5), (11, 14), (8, 7), (14, 11), and (7, 8), the maxima are:
The frequencies are 5: one, 7: one, 8: three, 11: one, and 14: four. The mean of the ten simulated maxima is
The median is the average of the fifth and sixth ordered values, 8 and 11, so it is \((8+11)/2=9.5\). The simulated values range from 5 to 14, and 14 appears most often. A larger simulation would usually give a more detailed picture than ten repetitions, but it would still be an approximation to the model’s sampling distribution.
Reading the Simulation as a Distribution
Once the statistic values have been recorded, a display makes their distribution easier to inspect. A frequency table lists each value and its count. A dotplot can show the same information visually, with one dot for each repetition. For numerical statistics, consider three features:
- Center: a typical value, described using a summary such as the mean or median of the simulated statistics.
- Spread: how much the statistics vary, described using their range or another suitable measure of variability.
- Shape: the distribution’s overall pattern, such as whether values cluster in one area, have a tail, or appear roughly balanced.
The display represents statistics, not the original observations. For example, in the first example, each dot in a dotplot would represent a median calculated from one sample of three population values. It would not represent one of the five population values by itself. Keeping this distinction clear is essential: the simulation asks how the statistic changes across samples.
The number of repetitions matters. A small number can produce a rough display in which chance makes some values appear unusually common or absent. With more repetitions, the simulated frequencies tend to give a more stable picture of the model, though they need not match any exact distribution perfectly. Repeating a simulation more times cannot repair a mismatch between the simulation’s sampling plan and the process being represented.
Why the Simulation Works—and What It Does Not Show
Each repetition is one draw from the sampling model, followed by a fixed calculation. If \(R\) repetitions produce a statistic value \(t\) exactly \(k\) times, the simulated relative frequency of \(t\) is \(k/R\). More generally, if \(k\) repetitions give statistics in a chosen set of values, \(k/R\) is the simulated proportion of results in that set. This is an empirical description: it counts what happened in the simulation.
The justification for treating these frequencies as informative is that each repetition uses the same random sampling mechanism. Therefore, the simulated outcomes are generated according to the model of interest. Repeating that mechanism gives repeated opportunities to see which statistic values occur and how often. The resulting distribution is not a guarantee about the next sample, nor does a simulation alone establish that its population or sampling assumptions match reality.
Common Mistake
A frequent error is to treat a distribution of simulated statistics as though it were the distribution of individual population observations. For instance, seeing many sample medians near 6 does not mean that an individual selected from the population is likely to have value 6. Each median is calculated from an entire sample, so it is a different kind of quantity from one individual observation.
Another error is to describe ten simulated results as if they were the exact sampling distribution. The correction is to state what was simulated, report the number of repetitions, and qualify the conclusion: “In these 10 simulated samples, the statistic ranged from 2 to 10.” That statement is supported by the simulation. A claim about all possible samples requires an exact analysis or a sufficiently careful approximation, not just a small batch of results.
Check Your Understanding
Use the ideas in this tutorial to answer the following questions.
- A simulation draws samples of size 4 and records the same statistic 500 times. Which feature of the sampling process is fixed, and what varies from repetition to repetition?
- Explain why the values in a dotplot of simulated medians are not the same thing as the individual population values.
- In a simulation, a statistic falls between 3 and 9 in 42 of 200 repetitions. What is its simulated relative frequency in that interval?
- Why would a simulation that samples with replacement be a poor model for a process that selects four distinct people for each sample?
- A display from eight repetitions has a single most frequent value. Why should you be cautious about using that fact to describe the sampling distribution’s shape?