One Sample Is Only One Possible Result
In Sample Size and Precision, you learned that larger random samples generally produce statistics with less sampling variability. To see that variability directly, imagine repeating the same random sampling process many times and calculating the same statistic each time. The resulting sample proportions will usually differ, even though every sample comes from the same population.
A simulation lets us explore this idea when the population proportion is known or specified. We use chance to create many samples that follow a stated sampling method, calculate \(\hat{p}\) for each one, and examine how the results vary. The simulation is not another survey of the population. It is a model for what repeated sampling could produce.
For sample proportions, the pattern to look for is whether the simulated \(\hat{p}\) values cluster around the true population proportion \(p\), and how much they vary from one sample to another. A single simulated sample may produce a proportion above or below \(p\). The important pattern appears across many repetitions, not in every individual result.
How to Simulate Repeated Samples
A useful simulation needs to reflect the situation being modeled. First specify the population proportion and define which outcome counts as a “success.” Then choose a way to simulate one sample of the stated size. Repeat that same process, calculate \(\hat{p}\) for every sample, and summarize the results.
Specify the true proportion \(p\), the outcome being counted, the sample size \(n\), and the sampling method.
Use a chance device or a computer to select outcomes in a way that matches the model.
For each simulated sample, calculate \(\hat{p}=\frac{\text{number of successes}}{n}\).
Repeat the sample process many times. Look at the center and spread of the recorded proportions.
A simulation can be done with shuffled labels, random numbers, or software. For example, if a population has 30% of its members with a characteristic, one could label the members who have it and randomly select from the full list. The method should match the sampling plan: if the intended sample is selected without replacement, the simulation should also select without replacement within each sample.
After one simulated sample is complete, start a new selection to form the next sample. The new sample may include some of the same individuals as an earlier sample; that is normal when separate random samples are drawn from the same population. The important point is to use the same method and sample size for each repetition.
Worked Example: Twelve Samples From a Known Population
Worked Example: Residents Who Use a Bus Route
Suppose 30% of residents in a fictional town use a particular bus route. A computer repeatedly selects simple random samples of 20 residents from the town’s full resident list. Suppose the numbers of bus-route users in 12 simulated samples are 4, 5, 7, 6, 3, 8, 5, 6, 4, 7, 5, and 6. These are invented simulation results for illustration, not results from a real survey.
State: The population proportion is \(p=0.30\). The statistic is \(\hat{p}\), the proportion of people in each sample of 20 who use the route.
Plan: Divide each simulated count by 20 to find its sample proportion. Then describe how the proportions compare with \(p=0.30\). Because each sample uses the same size and selection method, their variation illustrates sampling variability.
Do: For the first sample, 4 of 20 residents use the route, so \(\hat{p}=4/20=0.20\). Applying the same calculation to all 12 samples gives:
| Simulated sample | Number of users | Sample proportion |
|---|---|---|
| 1 | 4 | 0.20 |
| 2 | 5 | 0.25 |
| 3 | 7 | 0.35 |
| 4 | 6 | 0.30 |
| 5 | 3 | 0.15 |
| 6 | 8 | 0.40 |
| 7 | 5 | 0.25 |
| 8 | 6 | 0.30 |
| 9 | 4 | 0.20 |
| 10 | 7 | 0.35 |
| 11 | 5 | 0.25 |
| 12 | 6 | 0.30 |
The mean of these 12 simulated proportions is the total number of users divided by the total number of sampled residents:
In these repetitions, the simulated proportions range from \(3/20=0.15\) to \(8/20=0.40\). Some are below 0.30, one equals 0.30, and others are above it.
Conclude: The simulated sample proportions vary from sample to sample and, in this short run, are centered near the true proportion of 0.30. Their mean, 0.275, is not exactly 0.30. That difference is unsurprising: a limited set of random repetitions need not have a mean exactly equal to the population proportion.
More Repetitions Reveal a Pattern
A few repetitions can look uneven just by chance. For example, it would not be surprising if the first several simulated samples all happened to have proportions above \(p\). That would not mean the simulation was designed incorrectly; it would mean the results so far do not show the longer-run pattern very clearly. Generating more samples gives a fuller picture of how the statistic varies.
In a display of many simulated proportions, look for three features. The center describes where the values tend to cluster. The spread describes how far the values tend to lie from one another and from the center. The individual results show the variation that can occur in any one sample. The values need not be perfectly balanced around \(p\), especially in a simulation with relatively few repetitions.
The number of simulated samples and the size of each sample play different roles. More repetitions make the simulation’s overall pattern easier to see. A larger sample size generally makes the sample proportions less variable from sample to sample, as discussed in Sample Size and Precision. Increasing the number of repetitions does not make each sample larger; it gives us more simulated outcomes to inspect.
Worked Example: Comparing Two Sample Sizes
Worked Example: A Preference for Reusable Bottles
In a fictional population, 40% of students prefer a reusable bottle to a disposable one. Two simulations each produce 10 random samples. One uses samples of 10 students; the other uses samples of 50. Suppose the counts of students with that preference are:
For \(n=10\): 2, 5, 3, 6, 4, 5, 7, 1, 4, and 3.
For \(n=50\): 18, 21, 23, 20, 22, 17, 24, 19, 21, and 15.
These are illustrative simulated results, not data from an actual school survey.
State: Both simulations model the same population proportion, \(p=0.40\), but use different sample sizes. We will compare the average and range of their simulated sample proportions.
Plan: Divide each count by its sample size to obtain \(\hat{p}\). Find the mean and range for each group of 10 simulated proportions. A narrower range in this particular simulation would illustrate, but not prove by itself, the general tendency for larger random samples to have less sampling variability.
Do: With \(n=10\), the sample proportions are 0.20, 0.50, 0.30, 0.60, 0.40, 0.50, 0.70, 0.10, 0.40, and 0.30. The total count is 40 out of 100 sampled students, so their mean is \(40/100=0.40\). The range is from 0.10 to 0.70.
With \(n=50\), the proportions are 0.36, 0.42, 0.46, 0.40, 0.44, 0.34, 0.48, 0.38, 0.42, and 0.30. Their mean is
Their range is from 0.30 to 0.48.
Conclude: In these simulated runs, both sets of proportions average 0.40, while the proportions from samples of 50 have a narrower range than those from samples of 10. This illustrates how estimates from larger random samples tend to vary less. The particular ranges depend on chance, and a different simulation could produce different results.
Building a Simulation With Random Numbers
The simulation model should give each outcome the intended chance. Suppose a fictional health survey aims to model a population in which 18% of people have a certain characteristic. One simple model for a single simulated person is to generate a random integer from 00 through 99. Treat 00 through 17 as “has the characteristic,” and 18 through 99 as “does not.” There are 18 integers in the first group out of 100 equally likely integers, so the modeled probability of the characteristic is \(18/100=0.18\).
To simulate a sample of 25 people using this model, generate 25 outcomes and count how many are in the “has the characteristic” group. Divide that count by 25 to get one simulated \(\hat{p}\). Then repeat the process to create another simulated sample and another proportion. A computer can do these repetitions quickly, but the logic of the model should still be clear.
This random-number approach treats outcomes as independent draws with the same probability. If the real plan is to select people without replacement from a particular finite population, a more exact simulation is to label everyone, mark which labels have the characteristic, and randomly select the required number of distinct labels for each sample. Matching the model to the actual selection method matters.
Worked Example: Simulating a Health-Related Proportion
Worked Example: A Characteristic in a Fictional Population
A fictional population has a proportion \(p=0.18\) with a particular characteristic. Using the random-number rule above, a computer generates 10 simulated samples of 25 outcomes each. Suppose the numbers with the characteristic are 3, 5, 2, 6, 4, 3, 7, 1, 5, and 4.
State: The model uses a 0.18 chance of the characteristic for each simulated outcome. The statistic from each sample is the proportion of its 25 outcomes that have the characteristic.
Plan: Divide each count by 25. Then calculate the mean and range of the 10 simulated proportions and compare them with \(p=0.18\).
Do: The proportions are \(3/25=0.12\), \(5/25=0.20\), \(2/25=0.08\), \(6/25=0.24\), \(4/25=0.16\), \(3/25=0.12\), \(7/25=0.28\), \(1/25=0.04\), \(5/25=0.20\), and \(4/25=0.16\). Their mean is
The smallest proportion is 0.04 and the largest is 0.28.
Conclude: These 10 simulated proportions vary from one sample to another and are centered reasonably near the model value of 0.18. Their mean is 0.16, not exactly 0.18, and their range includes values both below and above 0.18. A different set of random outcomes could produce a different mean and range.
What Simulation Can—and Cannot—Show
A simulation makes sampling variability visible by showing many possible results under a stated model. It can help answer questions such as “Do sample proportions change from sample to sample?” and “Do larger samples tend to give results that vary less?” It can also help students understand why one observed sample proportion should not be expected to match the population proportion exactly.
A simulation does not prove that a particular future sample will be close to \(p\). It does not remove bias from a flawed sampling frame, convenience selection, nonresponse, or influenced answers. As in Sample Size and Precision, a large sample size addresses variability under the sampling method; it does not repair a systematic problem in how people were selected or measured.
The simulation also depends on its assumptions. If the population proportion used in the model is wrong, or if the random selection process does not match the intended sample design, the simulated results may not describe the real sampling process well. State the model and selection method so a reader can judge what the simulation represents.
Common Mistakes and AP Exam Tips
- Claiming every sample proportion should equal \(p\). Random samples vary. A full-credit explanation says that repeated sample proportions tend to cluster around the population proportion, while individual samples can be above or below it.
- Assuming a short run must be perfectly centered. A simulation with a small number of repetitions can have a mean noticeably different from \(p\). Describe the results actually shown and avoid treating small differences as evidence that the model failed.
- Confusing the number of repetitions with sample size. The sample size \(n\) is the number of individuals in each sample. The number of repetitions is how many samples are simulated. More repetitions clarify the pattern; a larger \(n\) generally reduces variability in the sample proportions.
- Using an unfair random-number rule. The number of random outcomes assigned to each category must match the stated probability. If 18% is the target, assigning 18 of 100 equally likely integers to the characteristic models that proportion.
- Forgetting what is being calculated. For each sample, calculate the number with the characteristic divided by that sample’s size. Do not report the count alone as the sample proportion.
- Claiming a simulation fixes bias or guarantees accuracy. A simulation models a sampling process; it does not improve the real process. Identify possible selection or measurement problems separately.
When describing a simulation, name the population proportion, the sample size, the random selection model, and the statistic recorded. Then describe the pattern in context, using words such as “varies,” “clusters,” and “tends to.” Avoid saying that every result must be close to the true value.
Check Your Understanding
Use the idea of repeated sampling to answer each question.
- A population has a known proportion \(p=0.25\). In one simulated sample of 40 individuals, 8 have the characteristic. Find \(\hat{p}\) and compare it with \(p\).
- A student simulates 5 samples of 20 and then 500 samples of 20. What changes when the number of repetitions increases, and what stays the same?
- A computer simulation uses samples of 15 and samples of 60 from the same population. Which set of proportions would generally be expected to show less variability? Explain.
- In a simulation with \(p=0.18\), a sample proportion of 0.28 occurs. Does this mean the simulation is incorrect? Explain.
- A simulation models a random sample, but the real survey uses a list that omits some members of the population. Does the simulation account for that undercoverage? Explain.