Empirical Frequencies Versus Model Probabilities

Two Quantities That Answer Different Questions

Suppose a model assigns probability $1/2$ to heads on a coin toss. If the coin is tossed eight times and heads appears four times, the observed proportion of heads is also $1/2$. The numbers agree, but they describe different things: the model probability describes one trial under the model, while the observed proportion describes the results of a particular set of trials.

A finite run need not produce a proportion equal to the model probability. A fair coin can land heads on all four of four tosses; in that run the observed proportion is $1$, although the model probability of heads on each toss is $1/2$. The difference is not a contradiction. It is a distinction between a probability assigned by a model and a frequency calculated after observations are made.

Definition: In $n$ repeated trials, the count $N_A$ of an event $A$ is the number of trials on which $A$ occurs. Its observed relative frequency is $N_A/n$. A model probability $P(A)$ is the probability the model assigns to $A$ on a single trial. A relative frequency is calculated from the observed results; a model probability is part of the probability law.

The count and frequency are determined once the results are known. Before the trials, the particular results are not yet known, and a model can assign probabilities to possible results or to events describing them. To calculate probabilities for a sequence of trials, we need assumptions about how those trials are related. In the examples below, each trial follows the same probability model, and the trials are independent. Independence means that the probability of any specified sequence of trial outcomes is the product of the corresponding single-trial probabilities.

From Repeated Trials to a Frequency Count

Consider a two-outcome trial, with “success” denoting the event of interest and model probability $p$, where $0<p<1$. Assume $n$ independent trials, each with success probability $p$. Any particular sequence containing $k$ successes and $n-k$ failures has probability $p^k(1-p)^{n-k}$. There are $\binom{n}{k}$ ways to choose which $k$ trial positions are successes. These sequences are disjoint, so their probabilities add.

Theorem: Under $n$ independent trials with the same success probability $p$, the probability of observing exactly $k$ successes, for $k=0,1,\ldots,n$, is $$ P(N_A=k)=\binom{n}{k}p^k(1-p)^{n-k}. $$ Here $\binom{n}{k}=n!/[k!(n-k)!]$ is the number of ways to select $k$ positions from $n$.

Proof. First fix a particular arrangement of the $n$ trial results containing exactly $k$ successes. Independence gives that arrangement probability $p^k(1-p)^{n-k}$. There are $\binom{n}{k}$ such arrangements, because choosing the $k$ success positions determines the whole arrangement. Two different arrangements cannot both describe the results of the same set of trials, so these arrangements form disjoint events. Additivity for disjoint events therefore gives $$ P(N_A=k)=\binom{n}{k}p^k(1-p)^{n-k}. $$ $\square$

The theorem concerns the probability of a count, not the relative frequency itself. If $k$ successes are observed, the frequency is $k/n$; the theorem gives the probability, under the repeated-trial assumptions, of obtaining that count. In particular, the probability $p$ on a single trial is not generally the probability of obtaining any particular frequency in a finite run.

Worked Example: Four Heads in Eight Fair-Coin Tosses

A fair coin is tossed independently eight times. What is the model probability of heads on one toss? What is the probability that the observed relative frequency of heads is exactly $1/2$?

For one toss, the model probability of heads is $p=1/2$. An observed relative frequency of $1/2$ in eight tosses means that the number of heads is $8(1/2)=4$. Apply the count formula with $n=8$ and $k=4$:

P(N_{\text{heads}}=4) =\binom{8}{4}\left(\frac12\right)^4\left(\frac12\right)^4 =70\left(\frac12\right)^8 =\frac{70}{256} =0.2734375.

Thus the model probability on a single toss is $1/2$, while the probability that this eight-toss run has relative frequency exactly $1/2$ is $70/256$, about $0.2734$. If four heads are actually observed, the frequency in that run is $4/8=1/2$. The probability of that count and the frequency calculated from the observed count are different quantities, even though both involve the number $1/2$.

Finite Samples Can Differ From the Model

For $0<p<1$, every count from zero successes through $n$ successes has positive probability under the independent, identical-trial model. This makes precise why no finite sample size forces the observed frequency to equal $p$: results that differ from $p$ can still occur under the model.

Theorem: If $0<p<1$ and $n$ is a positive integer, then for every $k\in\{0,1,\ldots,n\}$, $$ P(N_A=k)=\binom{n}{k}p^k(1-p)^{n-k}>0. $$ In particular, the count zero has positive probability, and its relative frequency $0$ differs from $p$.

Proof. The count formula applies under the stated assumptions. For each allowed $k$, the binomial coefficient is a positive integer, $p^k>0$, and $(1-p)^{n-k}>0$, since $0<p<1$. Their product is positive. For $k=0$, the relative frequency is $0/n=0$, and $0\ne p$. Therefore, with positive probability, a finite run has a relative frequency different from the single-trial model probability. $\square$

This result does not say that every frequency is equally likely. It says that all the counts are possible under these assumptions. Their probabilities can be very different. Nor does the result say that the model is correct in an actual application: it describes what follows if the repeated trials really do satisfy the specified probability and independence assumptions.

Worked Example: Four Heads in Four Tosses

A fair coin is tossed independently four times. Suppose all four tosses land heads. Compare the observed relative frequency of heads with the model probability, and calculate how probable this particular count is under the model.

The model probability of heads on one toss is $p=1/2$. The observed count is $N_{\text{heads}}=4$, so the relative frequency is $$ \frac{N_{\text{heads}}}{n}=\frac{4}{4}=1. $$ This observed frequency is not $1/2$; it is the proportion of heads in this run.

To find the probability of four heads in four independent tosses, use the count formula: $$ P(N_{\text{heads}}=4) =\binom{4}{4}\left(\frac12\right)^4\left(\frac12\right)^0 =\frac{1}{16} =0.0625. $$

Thus an observed frequency of $1$ is possible even though the model probability on each toss is $1/2$. The model assigns probability $1/16$ to this particular count. It does not require the frequency in every four-toss run to equal $1/2$.

Observed Frequencies Across Several Outcomes

The same distinction applies when a trial has more than two possible outcomes. For a die roll, the model may assign probability $1/6$ to each face. After several rolls, the observed relative frequency of sixes is the number of sixes divided by the number of rolls. Frequencies for all six faces add to $1$, because every roll contributes to exactly one face’s count. Model probabilities for the six faces also add to $1$, but the two lists need not match entry by entry.

Worked Example: Twelve Rolls of a Fair Die

A fair die is rolled independently twelve times. Suppose the result is two sixes. What is the observed relative frequency of sixes, and what is the model probability of observing exactly two sixes?

The model probability of a six on any one roll is $1/6$. The observed count is two, so the relative frequency is $$ \frac{2}{12}=\frac16. $$ In this particular run, the observed frequency happens to equal the model probability for one roll.

To calculate the probability of obtaining exactly two sixes, treat “roll is a six” as success, with $n=12$, $k=2$, and $p=1/6$: $$ P(N_{\text{sixes}}=2) =\binom{12}{2}\left(\frac16\right)^2\left(\frac56\right)^{10} =66\cdot\frac{5^{10}}{6^{12}} =\frac{644531250}{2176782336} \approx 0.2961. $$

The value $1/6$ is the observed proportion in this run and also the model probability of a six on one roll. The value approximately $0.2961$ is something else: the model probability of getting exactly two sixes in twelve rolls. Matching proportions in one run does not make these quantities interchangeable.

Why the Distinction Matters

The distinction helps answer two different questions. A model probability describes what the probability law assigns to an event. An observed relative frequency summarizes what happened in a particular finite collection of trials. When someone reports that an event occurred in $30\%$ of a sample, that statement is a frequency description. It is not, by itself, the model’s probability for the event on the next trial.

Repeated-trial calculations also depend on the assumptions used to derive them. The count formula above requires the same success probability on every trial and independence between trials. If the probability changes from trial to trial, or results are dependent, the calculation may not apply in that form. The observed frequency can still be calculated, but its interpretation as a comparison with a particular model probability requires a clearly specified model.

Common Mistakes

Treating an observed proportion as a model probability. If 7 of 10 trials are successes, $7/10$ is the observed relative frequency. It does not automatically establish that the model probability is $7/10$.
Calling a probability of a count the probability on one trial. The probability of exactly four heads in eight tosses is about $0.2734$; the probability of heads on one fair-coin toss is $1/2$. These refer to different events.
Assuming a model requires an exact match. A probability model assigns probabilities to possible outcomes; it does not say that each finite run must reproduce those probabilities as frequencies. Under the repeated-trial assumptions, different counts can have positive probability.
Using the count formula without its assumptions. The formula $\binom{n}{k}p^k(1-p)^{n-k}$ requires independent trials with the same success probability $p$. A numerical frequency alone does not establish those assumptions.

Key takeaway: Keep the questions separate: $P(A)$ is the probability assigned by a model to an event on a trial, while $N_A/n$ is the relative frequency observed across $n$ trials. A finite run can differ from the model without contradicting it.

Check Your Understanding

For each question, distinguish the model probability from the frequency or count observed in a finite run.

A fair coin is tossed six times and lands heads four times. What is the observed relative frequency of heads, and what is the model probability of heads on one toss?
Under the same independent fair-coin model, write the formula for the probability of exactly three heads in six tosses. What event does this probability describe?
A die is rolled ten times with no sixes observed. What is the observed relative frequency of sixes? Does this observation make the model probability of a six equal to zero?
State the assumptions needed to use the count formula for exactly $k$ successes in $n$ trials.
Explain why an observed relative frequency of $1/6$ for sixes in twelve die rolls is not the probability of observing exactly two sixes in those twelve rolls.