The Law of Large Numbers in Simulations

What Happens to a Relative Frequency as Trials Accumulate?

In What Is a Probability Simulation, you used repeated chance outcomes to estimate a model probability. Now we will examine what tends to happen as the number of trials grows. We will use a simulated roll of a fair six-sided die and track the proportion of rolls that show a six.

The model probability of rolling a six on one roll is \(1/6\), or approximately \(0.1667\). A short sequence might contain many sixes, or none at all. As more rolls are added, however, the relative frequency of sixes tends to settle near \(1/6\). This long-run tendency is called the Law of Large Numbers.

For the die simulation, a trial is one roll, and the event of interest is rolling a six. After \(n\) rolls, the relative frequency is the number of sixes divided by \(n\). The model probability remains \(1/6\) throughout; it is the relative frequency from the simulation that changes as rolls are added.

The law depends on modeling repeated trials with the same chance of success. Here, each roll is assumed to be independent of the others, and the die is fair, so each roll has probability \(1/6\) of showing a six. A simulation that does not represent those assumptions would not illustrate this model’s long-run behavior.

A Cumulative Simulation: Proportion, Not Count

Consider one possible simulated sequence of fair-die rolls. The table shows cumulative counts from that same sequence, not separate simulations. The results are invented for illustration. Each relative frequency is calculated from the total number of rolls up to that row.

The first 10 rolls include 4 sixes, giving a relative frequency of \(0.4000\), well above \(1/6 \approx 0.1667\). By 50 rolls, the relative frequency is \(0.1800\). After 500 rolls it is \(0.1640\), and after 5,000 it is \(0.1662\). In this particular simulation, the longer-run proportions are closer to the model probability.

The number of sixes keeps increasing as rolls are added: it rises from 4 to 9 to 82 to 831. That increase alone does not show whether the estimate is improving. The quantity to compare with \(1/6\) is the proportion of sixes, because the model probability describes a proportion, not a count.

The relative frequency need not move closer to \(1/6\) with each new roll. It might be close to \(1/6\) at one point and move farther away after a run of sixes or a run without sixes. The Law of Large Numbers concerns the tendency over many trials, not a step-by-step rule.

Why Short Runs Can Vary Widely

In a short run, each observed six makes a relatively large change to the proportion. For example, one six in 12 rolls contributes \(1/12 \approx 0.0833\) to the relative frequency. When the total is only 12, adding or removing one success has a noticeable effect on the estimate. With 1,200 rolls, one additional six changes the relative frequency by only \(1/1{,}200 \approx 0.0008\).

That is one reason short runs can look quite different from one another: a small number of outcomes has a large influence on the proportion. In a long run, each individual outcome has less influence on the total relative frequency. The results still vary by chance, but a few unusually high or low outcomes tend to have less impact on the overall proportion.

This does not mean that every short run will be far from \(1/6\), or that every long run will be very close. A short run can happen to give a proportion near \(1/6\), and a long run can still be somewhat above or below it. The law describes what tends to happen as repetitions increase, rather than promising a particular outcome or a fixed amount of closeness.

Reading a Cumulative Relative-Frequency Display

A useful way to inspect a simulation is to plot the cumulative relative frequency after each trial, with the model probability marked as a reference value. For this die example, the reference is \(1/6\). Early values may move above and below that line quite a bit. As the sequence grows, the cumulative relative frequency tends to fluctuate in a narrower region around the reference.

This display uses cumulative results: the value after roll 100 includes rolls 1 through 100, and the value after roll 101 includes rolls 1 through 101. It is not a display of the outcome on each individual roll. A single roll is either a six or not a six; the cumulative relative frequency summarizes the sequence up to that point.

A cumulative graph can still move away from the reference line for a while. Suppose a long sequence is close to \(1/6\), and several additional rolls are sixes. The proportion may rise. If many additional rolls are not sixes, it may fall. What the Law of Large Numbers suggests is that, over a long sequence of independent rolls, the cumulative proportion tends to settle near the model probability—not that its path must be smooth or always move in one direction.

Common Mistakes and AP Exam Tips

• Saying the relative frequency must equal the probability. A simulation gives an estimate. A careful statement is, “The simulated relative frequency was \(0.1833\), compared with the model probability \(1/6\).” Do not call the estimate the exact probability.
• Claiming every longer run is closer. The proportion can move toward or away from the model probability as trials are added. Say that it tends to settle near the probability over many repeated trials.
• Confusing counts and proportions. A long run will generally contain more sixes simply because it contains more rolls. Compare the number of sixes divided by the number of rolls—not the count alone—with \(1/6\).
• Assuming the law changes the probability of the next roll. If the die is fair, the probability of a six on the next roll remains \(1/6\), regardless of how many sixes have already appeared.
• Treating one run as proof about fairness. A short simulation that produces a high or low proportion can occur by chance. Interpret it as a result from that run, not as proof that the model is wrong.
• Forgetting the model assumptions. The example uses a fair die and repeated independent rolls. State the chance model being simulated when explaining why its relative frequency is expected to settle near \(1/6\).

For a clear AP response, identify the event and its model probability, calculate the relative frequency using the correct total number of trials, and explain what the result suggests. When describing the Law of Large Numbers, use wording such as “tends to get closer” or “tends to settle near.” Avoid saying “must,” “always,” or “will equal.”