Why Run More Than 20 Trials?
In Estimating Probability of at Least One Success, you used the proportion of simulated trials in which an event occurred to estimate its probability. But that estimate depends on the particular results generated. If you repeat the simulation, you may get a different number of successes and a different estimate.
This tutorial focuses on what happens when a simulation is repeated many times. We will compare estimates based on 20 trials with an estimate based on 500 trials. The key idea is that a larger number of repetitions generally makes the relative frequency less variable, but it does not guarantee that every larger-run estimate is closer to the probability being modeled.
Imagine a digital prize wheel with a chance model represented by random digits 0 through 9. Digits 0, 1, and 2 mean “bonus”; the other digits mean “no bonus.” Use one generated digit as one trial. As in Assigning Random Digits to Outcomes, the same digit assignment must be used throughout the simulation. As in Estimating a Probability from Simulation Results, the estimate is the number of successful trials divided by the total number of trials.
A 20-Trial Estimate Can Move Noticeably
With only 20 trials, one success changes the estimate by \(1/20=0.05\), or five percentage points. For example, 6 successes give an estimate of 0.30, while 7 give an estimate of 0.35. The estimate changes in fairly large steps because the denominator is small.
Worked Example: Estimate from 20 Trials
A student generates 20 digits using the bonus-wheel model. The outcomes are recorded below, with “B” for a bonus and “N” for no bonus. Estimate the chance of receiving a bonus in this simulation.
| Trials | Outcomes | Bonus count |
|---|---|---|
| 1–10 | N, B, N, N, B, N, N, B, N, N | 3 |
| 11–20 | B, N, N, N, B, N, B, N, N, N | 4 |
| Total | 20 trials | 7 |
Identify: One trial is one generated digit. The event of interest is receiving a bonus, represented by a digit of 0, 1, or 2.
Count: There are 3 bonuses in the first ten trials and 4 in the next ten, for \(3+4=7\) bonuses in all. There are \(10+10=20\) completed trials.
Calculate: Divide the number of bonus outcomes by the number of trials.
Interpret: In these 20 simulated trials, the estimated probability of receiving a bonus is 0.35, or 35%. This is the relative frequency in this run. It is not a promise that the next 20 trials—or the prize wheel itself—will produce exactly 35% bonuses.
This one estimate does not tell us how much estimates from 20 trials might vary. To see variation, repeat the same 20-trial simulation several times, using the same model and trial definition each time. Each run will have its own count and estimate.
Repeated 20-Trial Runs Can Give Different Estimates
The next table summarizes five independent runs, each consisting of 20 trials. The runs all use the same digit assignment. Their success counts differ even though the simulated chance process has not changed.
Worked Example: Compare Five Runs of 20 Trials
Use the success counts to calculate the relative frequency for each run. Then describe how much the estimates vary across these five runs.
| Run | Trials | Bonus outcomes | Estimate |
|---|---|---|---|
| 1 | 20 | 4 | \(4/20=0.20\) |
| 2 | 20 | 8 | \(8/20=0.40\) |
| 3 | 20 | 5 | \(5/20=0.25\) |
| 4 | 20 | 7 | \(7/20=0.35\) |
| 5 | 20 | 3 | \(3/20=0.15\) |
For example, Run 2 has 8 bonus outcomes, so its estimate is \(8/20=0.40\), or 40%. Run 5 has 3, so its estimate is \(3/20=0.15\), or 15%. Across the five runs, the smallest estimate is 0.15 and the largest is 0.40. Their difference is \(0.40-0.15=0.25\), or 25 percentage points.
The combined total is \(4+8+5+7+3=27\) bonuses out of \(5(20)=100\) trials. Combining the trials gives an overall relative frequency of \(27/100=0.27\), or 27%. That combined estimate describes all 100 trials together; it does not erase the variation among the five separate estimates.
The example shows that an estimate from 20 trials can depend substantially on which outcomes happen to occur. The five runs do not establish every possible result from a 20-trial simulation, but they make the variability visible. As discussed in Sampling Variability in Repeated Samples, repeated chance-based processes can produce different results even when the method stays the same.
Pool More Trials to Get a Steadier Estimate
A useful way to organize a long simulation is to run it in batches and keep a cumulative total. The estimate after each batch is based on all trials so far, not just the most recent batch. This allows you to observe how the cumulative relative frequency changes as repetitions are added.
Here, 500 trials are arranged as 25 batches of 20. The table gives the number of bonus outcomes in each batch. Five batches at a time make 100 trials, so the cumulative totals can be checked at each 100-trial mark.
Worked Example: Follow the Estimate Through 500 Trials
Calculate the cumulative estimate after each 100 trials and after all 500 trials. Also compare the estimate from the first 20 trials with the estimate from all 500.
| Batches | Bonus counts in the five batches | Bonuses in these 100 trials | Cumulative bonuses |
|---|---|---|---|
| 1–5 | 5, 6, 7, 8, 4 | 30 | 30 |
| 6–10 | 7, 5, 6, 6, 9 | 33 | 63 |
| 11–15 | 4, 8, 5, 7, 6 | 30 | 93 |
| 16–20 | 7, 6, 5, 8, 7 | 33 | 126 |
| 21–25 | 5, 6, 4, 9, 3 | 27 | 153 |
For the first 100 trials, add the first five batch counts: \(5+6+7+8+4=30\). At 200 trials, the cumulative count is \(30+33=63\). Continuing, the cumulative counts are 93 after 300 trials, 126 after 400 trials, and 153 after 500 trials.
Divide each cumulative count by the corresponding number of trials to get each cumulative estimate:
The first batch contains 5 bonuses, so its 20-trial estimate is \(5/20=0.25\). After all 500 trials, the estimate is \(153/500=0.306\), or 30.6%. The cumulative estimate changes as results are added, but the changes between the 100-trial marks are small in this particular run. The separate 20-trial batches still vary: their counts range from 3 to 9 bonuses, equivalent to estimates from \(3/20=0.15\) to \(9/20=0.45\).
This is a useful illustration, not a rule that every 500-trial estimate must be near a particular value. Pooling many trials tends to smooth out the effect of unusually high or low results in individual batches. Still, a 500-trial run could produce a less typical estimate by chance, and the estimate from 20 trials could happen to be close to the probability being modeled.
What More Repetitions Do—and Do Not—Tell You
The Law of Large Numbers, introduced in The Law of Large Numbers in Simulations, describes the general tendency of the relative frequency to settle near the event’s probability as the number of repeated, independent trials grows. This tutorial emphasizes how that tendency appears when estimates from different run sizes are compared.
More repetitions are useful because each new trial adds information from the same chance process to the relative frequency. An unusual run of outcomes has less influence on the overall estimate when it is combined with many more trials. This is why a 500-trial estimate is generally more stable than an estimate based on only 20 trials.
However, “more stable” is not the same as “guaranteed correct.” A particular 500-trial estimate can be farther from the probability being modeled than a particular 20-trial estimate. Also, repeating a flawed or mismatched simulation many times does not fix the model. As in Setting Up a Simulation Model, the chance process, event, and definition of one trial must represent the question being studied.
Common Mistakes and AP Exam Tips
- Assuming a larger run must be closer. Say that more trials tend to make estimates less variable, not that 500 trials guarantee the “right” answer. A larger run can still produce an estimate that differs from the probability being modeled.
- Comparing counts instead of relative frequencies. A run with 500 trials will usually have more successes in total than a run with 20 simply because it has more trials. Compare \(x/n\), the success count divided by the number of trials, rather than comparing raw counts alone.
- Changing the event or the model partway through. A fair comparison uses the same definition of success and the same chance model for every run. Otherwise, differences may come from changing the simulation rather than from the number of repetitions.
- Confusing a batch with the whole run. In the 500-trial example, each batch contains 20 trials, but the final estimate uses all 500 trials. State which denominator is being used whenever you report an estimate.
- Treating one comparison as proof. A 20-trial estimate and a 500-trial estimate are two outcomes of chance simulations. Their difference illustrates variability; it does not prove that every 500-trial estimate will be closer.
A strong response identifies the estimate as a relative frequency, notes that the model and trial definition remain fixed, and explains the typical—not guaranteed—effect of increasing the number of trials. For example: “The estimate from 500 trials is generally expected to vary less than an estimate from 20 trials because it combines more repetitions of the same chance process. This does not guarantee that this particular 500-trial estimate is closer.”
Check Your Understanding
Use the idea that each simulated trial is one generated digit and that a bonus is represented by 0, 1, or 2.
- A run has 6 bonuses in 20 trials. What is its simulated estimate?
- Two 20-trial runs have 4 and 9 bonuses. Calculate each estimate and describe the difference in percentage points.
- A student combines five batches of 20 trials and counts 31 bonuses. What is the estimate for all 100 trials?
- Why is it misleading to say that 500 trials guarantee a more accurate estimate than 20 trials?
- What must remain consistent for estimates from two different run sizes to be meaningfully compared?