Three Places a Simulation Can Go Wrong
In Reading a Dotplot of Simulation Results, you learned how to interpret results after a simulation has been carried out. But a dotplot can only be useful when the simulation that produced it matches the chance situation being modeled. A flawed setup can produce a precise-looking display of results that answers the wrong question.
Common design problems often arise at three points: assigning random labels to outcomes, defining what counts as one trial, and repeating the trial. A quick audit of those points can reveal whether the simulation represents the intended chance model. We will call it a mapping–trial–repeat audit: check the outcome mapping, identify one complete trial, and verify that the same trial is repeated as planned.
This audit builds on Setting Up a Simulation Model, Assigning Random Digits to Outcomes, and Describing a Simulation in Words. It does not replace those steps; it helps catch common errors in carrying them out. In particular, a mapping must match the model’s probabilities, not merely use all available labels.
Error 1: Giving Unequally Likely Outcomes Equal Numbers of Labels
If the possible random digits are equally likely, an outcome’s share of the labels should match its probability in the model. Assigning one label to each outcome is appropriate only when the outcomes are equally likely. When the probabilities differ, equal-sized assignments change the model.
For two-digit labels from 00 through 99, there are 100 equally likely labels. A 40% outcome should receive 40 labels, a 35% outcome should receive 35 labels, and a 25% outcome should receive 25 labels. The labels can be assigned in consecutive blocks, as long as the blocks do not overlap and account for all 100 possibilities.
Worked Example: Repair an Unequal-Probability Mapping
A community garden records which of three equally sized observation periods a randomly selected visitor arrives in: morning, afternoon, or evening. For a particular model, the arrival probabilities are 0.40 in the morning, 0.35 in the afternoon, and 0.25 in the evening. A student proposes assigning 00–32 to morning, 33–65 to afternoon, and 66–99 to evening, saying that all three periods now have labels.
State: We need a digit assignment that gives the three arrival periods their stated probabilities, not equal probabilities.
Plan: Use the 100 equally likely two-digit labels from 00 through 99. Give morning 40 labels, afternoon 35 labels, and evening 25 labels. Check both the number of labels in each range and that every label is assigned exactly once.
Do: The proposed morning range, 00–32, contains 33 labels; the afternoon range, 33–65, contains 33 labels; and the evening range, 66–99, contains 34 labels. The proposed assignment therefore models probabilities 0.33, 0.33, and 0.34—not 0.40, 0.35, and 0.25.
A corrected assignment is 00–39 for morning, 40–74 for afternoon, and 75–99 for evening. The ranges contain 40, 35, and 25 labels, respectively. Their implied probabilities are \(40/100=0.40\), \(35/100=0.35\), and \(25/100=0.25\). The ranges cover 00 through 99 without gaps or overlap.
Conclude: The corrected mapping represents the stated arrival model. In each generated two-digit label, the simulated period is selected by its assigned range; assigning the same number of labels to each period would incorrectly treat the three periods as equally likely.
A mapping can fail in other ways, too. A label might be left unassigned, assigned to two outcomes, or rejected even though the procedure does not explain how to draw a replacement. Check the whole mapping, not just one outcome’s share. When the model requires probabilities that cannot be represented exactly by the available labels, use a more suitable set of equally likely labels or a different chance device rather than quietly changing the probabilities.
Error 2: Forgetting What One Trial Represents
A probability question might be about a whole group, device, game, or sequence—not about an individual outcome inside it. A trial must represent the unit named in the question. If the event is whether a device works after several component checks, for example, one check is not one trial; one simulated device, with all of its checks, is.
An unclear trial definition can lead to the wrong tally. A student might count every successful component as a successful device, or might stop a trial before all required outcomes have been generated. State what one full trial contains, where it begins and ends, and what single result will be recorded for it.
Worked Example: Define a Trial for a Multi-Part Device
A model says that each of three components in a device independently passes a check with probability 0.9. The device works if at least two of its three components pass. A student uses one random digit per component, with digits 0–8 representing “pass” and 9 representing “fail,” but counts each digit from 0–8 as a successful device.
State: The event of interest is that a complete device works. That requires at least two passes among its three components.
Plan: One trial should simulate all three component checks for one device. Generate three digits, translate each into pass or fail, and record whether the device has at least two passes. Repeat that whole process for each simulated device.
Do: For instance, the digits 8, 9, 2 represent pass, fail, pass, so that device works. The digits 9, 9, 0 represent fail, fail, pass, so that device does not work. The single digit 0 means only that one component passed; by itself, it does not say whether the device works.
Conclude: The corrected trial produces one device-level result from three component outcomes. Recording the event this way answers the question about the probability that a device works, rather than the different question of whether an individual component passes.
In a simulation with many trials, record one result per complete device. If a trial is cut short after only one or two checks, it is incomplete and should not be treated as a completed device. The model also says the component checks are independent, so the chance process must generate each check in a way that preserves that assumption.
This distinction matters whenever a trial contains several outcomes. In Simulating a Fixed Number of Trials, for instance, a complete quiz—not an individual question—is one trial when the probability question concerns quiz results. In a repeated-until-success situation, the stopping rule defines where that complete trial ends. The useful question is always: “What single unit does the probability question ask about?”
Error 3: Not Repeating the Complete Trial
A simulation used to estimate a probability needs repeated trials. Generating one group, one device, or one game provides only one result. It cannot show how often the event occurs across repeated instances of the model. As discussed in How Many Trials Are Enough, more trials generally make a relative-frequency estimate less variable, although no particular number guarantees a closer estimate in every run.
Be precise about what is repeated. If one trial is four selections, repeat the full four-selection process—not just one selection—and record one result for each completed trial. The same mapping and trial definition should be used throughout. Do not change the rules partway through a simulation.
Worked Example: Repeat a Four-Selection Trial
A prize is won on each selection with probability 0.20. A person makes four selections, and the event of interest is winning at least once. Assume selections are independent, so a winning label can occur again on the next selection. A student simulates one set of four selections and reports whether at least one win occurred.
State: We want an estimate of the probability of at least one win in a complete set of four selections. One set is one trial; the individual selections are outcomes within that trial.
Plan: Use 00–19 to represent a win and 20–99 to represent no win. For one trial, generate four two-digit labels, count the wins in that set, and record whether the count is at least one. Repeat the complete trial many times. Since the selections are independent, allow the same label, including a winning label, to appear more than once in a trial.
Do: Suppose a hypothetical run produces 20 completed trials with this summary:
| Wins in one four-selection trial | Number of trials |
|---|---|
| 0 | 8 |
| 1 | 7 |
| 2 | 4 |
| 3 | 1 |
| 4 | 0 |
The frequencies account for \(8+7+4+1+0=20\) completed trials. At least one win occurred in the trials with 1, 2, 3, or 4 wins, for \(7+4+1+0=12\) trials. The simulated relative frequency is:
Conclude: In this hypothetical run, 12 of 20 simulated sets had at least one win, so the estimated probability is 0.60, or 60%. This is a result from only 20 trials, not an exact probability. One simulated set alone would give only one result and a much less informative relative-frequency estimate.
The repeated labels are not an error here. Each selection is independent, and each selection has the same 0.20 chance of a win. Removing a repeated label or skipping a repeated winning label would alter the chance process. In contrast, a model that specifies selection without replacement must follow that different rule, as explained in Sampling With and Without Replacement in Simulations.
“Repeat the trial” does not mean repeat a particular sequence of generated labels. It means run the whole chance process again under the same model. A new trial can happen to have the same sequence or event count as an earlier one; those repeated results are valid and should be counted.
Use a Mapping–Trial–Repeat Audit
When evaluating a simulation description, read it in order and ask a small set of concrete questions. The audit helps separate different problems: a correct mapping does not repair an incomplete trial, and a well-defined trial does not help if the process is run only once.
Does each possible outcome receive the right share of equally likely labels? Are all labels handled once, without gaps or overlap?
Does one trial include every chance outcome needed to answer the question? Is there exactly one clearly defined result to record for that trial?
Is the complete trial repeated the stated number of times, using the same chance model? Are repeated outcomes handled according to the model’s replacement or independence rule?
A strong critique identifies the specific flaw, explains how it changes the model or recorded result, and proposes a correction. “The simulation is wrong” is not enough. Say, for example, that equal-sized label groups model equally likely outcomes even though the specified probabilities differ, then give a mapping with the proper label counts.
Common Mistakes and AP Exam Tips
- Checking only that every outcome has a label. A mapping can cover all labels and still give outcomes the wrong probabilities. Check each outcome’s share against the model.
- Equating one outcome with one trial. When the question concerns a multi-part unit, such as a complete device or a group of selections, include all parts in each trial and record the event for the unit as a whole.
- Stopping before a trial is complete. If the trial requires four selections, generating only one or two does not produce a completed result for the event in question.
- Running the process only once. One result cannot show the relative frequency across repeated trials. Repeat the complete chance process and record a result each time.
- Deleting repeated labels automatically. Whether repeats are allowed depends on the model. For independent selections or sampling with replacement, repeats can be valid; do not remove them just because they occurred.
- Changing the rules during the simulation. Use the same assignment, trial definition, and repetition procedure throughout, or the results combine different models.
For full-credit communication, name the chance-model feature the plan fails to represent, describe how the error affects the simulated results, and state a specific repair. When explaining a corrected plan, include the label assignment, one complete trial, what to record, and how the trial is repeated. Keep your conclusion tied to the probability question rather than claiming that a small simulation run gives an exact answer.
Check Your Understanding
For each situation, identify the design problem, if any, and explain a correction.
- A model has outcomes A, B, and C with probabilities 0.50, 0.30, and 0.20. A student assigns 00–32 to A, 33–65 to B, and 66–99 to C. What is wrong, and how many labels should each outcome receive?
- A device works if at least two of four independently tested parts pass. A student generates four outcomes but records each passing part as a working device. What should one trial represent, and what should be recorded?
- A question asks for the probability of at least one success in a group of six independent attempts. A student generates one attempt, then repeats that one attempt 100 times. What does the student’s procedure fail to simulate?
- In a model with independent selections, the same outcome appears twice in one simulated trial. Should the student delete one occurrence? Explain why or why not.
- A plan gives each outcome the correct label share and defines a complete trial, but runs only one trial. Which part of the mapping–trial–repeat audit identifies the remaining problem?