The LLN for Bernoulli Sample Proportions

From Repeated Events to a Sample Proportion

Suppose a process is repeated, and on each trial we record whether a specified event occurs. The long-run fraction of trials on which it occurs is a sample proportion. The law of large numbers gives this proportion a precise long-run interpretation: under suitable assumptions, it approaches the event’s probability as the number of trials increases.

The key is to encode each yes-or-no outcome as a random variable. An indicator variable is 1 when the event occurs and 0 otherwise. Averaging these indicators produces exactly the sample proportion. This connection lets us apply the strong law of large numbers directly to repeated-event data.

Definition: For an event $A_i$ on trial $i$, its indicator variable is $I_i=1$ if $A_i$ occurs and $I_i=0$ otherwise. For $n$ trials, the number of occurrences is $N_n=\sum_{i=1}^n I_i$, and the sample proportion is $$ \hat{p}_n=\frac{N_n}{n}=\frac{1}{n}\sum_{i=1}^n I_i. $$ If the trials are independent and have the same event probability $p=P(A_i)$, then the $I_i$ are independent and identically distributed Bernoulli random variables with parameter $p$.

For a Bernoulli variable $I_i$, the value 1 has probability $p$, and the value 0 has probability $1-p$. Thus its mean is $p$, and its variance is $p(1-p)$. If the indicators are independent and share this distribution, the strong law applies because each indicator has a finite expected absolute value.

Theorem: Let $I_1,I_2,\ldots$ be independent, identically distributed Bernoulli random variables with $P(I_i=1)=p$. Then $$ \hat{p}_n=\frac{1}{n}\sum_{i=1}^n I_i \longrightarrow p \quad\text{almost surely as } n\longrightarrow\infty. $$ Almost-sure convergence means that the probability that the sequence of sample proportions converges to $p$ is 1. In particular, $\hat{p}_n$ also converges to $p$ in probability: for every $\varepsilon>0$, the probability $P(|\hat{p}_n-p|>\varepsilon)$ tends to 0.

This result is not a claim that a finite-sample proportion equals $p$, nor that the proportion must move closer to $p$ after every additional trial. Individual outcomes can move the proportion in either direction. The conclusion concerns the limit of the sequence as the number of trials grows without bound.

Why the Proportion Has This Limit

The indicator representation gives both a direct application of the strong law and a useful finite-sample calculation. First, for a Bernoulli indicator,

E[I_i]=1\cdot p+0\cdot(1-p)=p, \qquad \operatorname{Var}(I_i)=p(1-p).

Linearity of expectation gives the mean of the sample proportion. Independence makes the covariance between distinct indicators zero, so the variance of their average also follows directly:

E[\hat{p}_n] =\frac{1}{n}\sum_{i=1}^n E[I_i] =\frac{np}{n} =p, $$ $$ \operatorname{Var}(\hat{p}_n) =\frac{1}{n^2}\operatorname{Var}\left(\sum_{i=1}^n I_i\right) =\frac{1}{n^2}\sum_{i=1}^n p(1-p) =\frac{p(1-p)}{n}.

The variance shrinks as $n$ increases. More formally, Chebyshev’s inequality gives, for every $\varepsilon>0$,

P\left(|\hat{p}_n-p|\geq\varepsilon\right) \leq \frac{\operatorname{Var}(\hat{p}_n)}{\varepsilon^2} =\frac{p(1-p)}{n\varepsilon^2} \longrightarrow 0.

This establishes convergence in probability. The strong law gives the stronger almost-sure conclusion under the stated independent, identical Bernoulli assumptions. The bound is useful for understanding why deviations become less likely, but it is only an upper bound; it does not generally give the exact probability of a deviation.

Worked Example: A Repeated Sensor Alert

Model. In a hypothetical monitoring study, each shift independently produces a sensor alert with probability $p=0.12$. Let $I_i=1$ if shift $i$ produces an alert and $I_i=0$ otherwise. The indicators are IID Bernoulli with parameter $0.12$. For $n=400$ shifts, find the expected number of alerts, the variance of the observed proportion, and a Chebyshev upper bound for the proportion to differ from $0.12$ by at least $0.04$. State the long-run conclusion.

Method. Use the Bernoulli count and proportion formulas, then apply Chebyshev’s inequality.

The number of alerts is $N_{400}=\sum_{i=1}^{400}I_i$. Its expected value is $$ E[N_{400}]=400(0.12)=48. $$ The expected number need not be an integer; it is an average over repetitions of the 400-shift study.

The proportion is $\hat{p}_{400}=N_{400}/400$. Its variance is $$ \operatorname{Var}(\hat{p}_{400}) =\frac{0.12(1-0.12)}{400} =\frac{0.12(0.88)}{400} =\frac{0.1056}{400} =0.000264. $$ For a deviation of at least $0.04$, Chebyshev’s inequality gives $$ P(|\hat{p}_{400}-0.12|\geq 0.04) \leq \frac{0.1056}{400(0.04)^2} =\frac{0.1056}{400(0.0016)} =\frac{0.1056}{0.64} =0.1650. $$ Thus the probability is at most $0.1650$, not necessarily equal to that value. By the strong law, as the number of independent shifts tends to infinity, the alert proportion converges almost surely to $0.12$.

Check. The expected alert count divided by 400 is $48/400=0.12$, matching the expected proportion. The upper bound is below 1, as a probability bound should be.

Common mistake avoided: Treating the expected count of 48 as a prediction that exactly 48 alerts will occur. The count is random; 48 is its expectation, while the LLN describes the limiting proportion over an increasing number of shifts.

What the Result Does—and Does Not—Say

The sample proportion is itself a random variable before the trials are observed. Once the outcomes are recorded, its realized value is a number between 0 and 1. The theorem concerns the sequence of random proportions generated by an indefinitely extended series of trials. It does not guarantee that any particular finite sample is close to $p$, or specify the exact chance of being within a chosen tolerance.

Two assumptions matter in the theorem’s standard form: each indicator has the same success probability, and the indicators are independent. If the trial probabilities change, the common-parameter Bernoulli theorem cannot simply be applied with one fixed $p$. If outcomes are dependent, the variance calculation above may also fail because covariance terms need not be zero. The situation must then be analyzed under its own assumptions.

A useful finite-sample consequence is the variance formula $p(1-p)/n$. It quantifies the variability of the proportion around its mean $p$, and Chebyshev’s inequality turns that variance into a bound on a deviation probability. Neither result should be mistaken for a guarantee of a particular observed value. The law of large numbers is about long-run stabilization, not a normal approximation or a precise finite-sample tail calculation.

Worked Example: At Least One Review Flag in a Day

Model. Consider a hypothetical daily review process with two independent checks. Check 1 flags an issue with probability $0.10$, and check 2 flags an issue with probability $0.20$. The checks are independent within a day, and daily pairs of checks are independent and identically distributed across 500 days. Define a flagged day as a day on which at least one check flags an issue. Find the flagged-day probability, expected number of flagged days, proportion variance, and a Chebyshev bound for a deviation of at least $0.04$. Give the long-run limit.

Method. First find the probability of the daily event using its complement. Then regard each day’s event indicator as one Bernoulli trial.

A day is not flagged only if neither check flags. By within-day independence, $$ P(\text{no flag on a day})=(1-0.10)(1-0.20)=0.90(0.80)=0.72. $$ Therefore the probability of at least one flag is $$ p=1-0.72=0.28. $$ Let $I_i=1$ for a flagged day and $I_i=0$ otherwise. Under the stated independence and identical daily setup, the 500 indicators are IID Bernoulli with parameter $0.28$.

The expected number of flagged days is $$ E[N_{500}]=500(0.28)=140. $$ The variance of the sample proportion is $$ \operatorname{Var}(\hat{p}_{500}) =\frac{0.28(1-0.28)}{500} =\frac{0.28(0.72)}{500} =\frac{0.2016}{500} =0.0004032. $$ For a deviation of at least $0.04$, the Chebyshev bound is $$ P(|\hat{p}_{500}-0.28|\geq 0.04) \leq \frac{0.2016}{500(0.04)^2} =\frac{0.2016}{500(0.0016)} =\frac{0.2016}{0.8} =0.2520. $$ The daily flagged proportion converges almost surely to $0.28$ as the number of days tends to infinity.

Check. The complement probability $0.72$ plus the flagged-day probability $0.28$ equals 1. The expected flagged-day count divided by 500 is $140/500=0.28$.

Common mistake avoided: Counting each check as a separate flagged day. The statistic here is the proportion of days with at least one flag, so each day contributes one indicator, even if both checks flag.

Worked Example: Enrollment Outcomes Across Repeated Cohorts

Model. In a hypothetical program, each person independently enrolls with probability $p=0.65$. Consider a sequence of IID cohorts, each with 250 people, and let the statistic for a cohort be its enrollment proportion. Find the expected enrollment count and the standard deviation of the cohort proportion. Then bound the probability that the proportion differs from $0.65$ by at least $0.06$, and explain the limiting statement across increasingly large cohorts.

Method. Model the 250 enrollment indicators as IID Bernoulli variables, calculate the cohort proportion’s moments, and use Chebyshev’s inequality.

For one cohort, the expected number enrolled is $$ E[N_{250}]=250(0.65)=162.5. $$ The proportion variance is $$ \operatorname{Var}(\hat{p}_{250}) =\frac{0.65(1-0.65)}{250} =\frac{0.65(0.35)}{250} =\frac{0.2275}{250} =0.00091. $$ Its standard deviation is $$ \sqrt{0.00091}=0.030166\ldots\approx 0.0302. $$ For a deviation of at least $0.06$, $$ P(|\hat{p}_{250}-0.65|\geq 0.06) \leq \frac{0.2275}{250(0.06)^2} =\frac{0.2275}{250(0.0036)} =\frac{0.2275}{0.9} =0.2528 $$ to four decimal places. The bound does not say that the actual probability is $0.2528$ or that it is zero.

For increasingly large cohorts generated under the same independent enrollment probability, the strong law says that the enrollment proportion converges almost surely to $0.65$. The expected count of $162.5$ for a 250-person cohort is not a possible realized count; it is the mean of the count’s distribution.

Check. The proportion mean is $0.65$, and $250(0.65)=162.5$ is consistent with that mean. The reported standard deviation is the square root of the proportion variance, rounded to four decimal places.

Common mistake avoided: Rounding the expected enrollment count to 163 and treating that as a certain outcome. An expectation need not be an integer, and the realized count remains random.

Common Exam Traps

Confusing count and proportion. The count is $N_n$; the proportion is $N_n/n$. Their means are $np$ and $p$, respectively.
Using a finite sample as a guarantee. Convergence as $n$ grows does not mean that $\hat{p}_n=p$ for a particular $n$.
Claiming monotonic movement. A new outcome can move the sample proportion farther from $p$ as well as closer to it.
Ignoring the model assumptions. The stated Bernoulli LLN requires independent trials with a common event probability. Dependence or changing probabilities requires additional analysis.
Calling an upper bound an exact probability. Chebyshev’s inequality gives a bound, not generally the exact deviation probability.

Key takeaway: Under repeated independent trials with common event probability $p$, the event indicators are IID Bernoulli$(p)$. Their average is the sample proportion, and the strong law ensures that this proportion converges almost surely to $p$.

Check Your Understanding

For each problem, identify the Bernoulli parameter and use the law of large numbers or proportion formulas as requested.

A hypothetical inspection independently finds a defect with probability $0.40$ on each item. For 100 items, find the expected defect count and the variance of the defect proportion, then state its almost-sure limit as the number inspected grows. (Answer: expected count $40$; proportion variance $0.0024$; limit $0.40$.)
In a hypothetical repeated review, each file independently has a missing field with probability $0.015$. For 2,000 files, find the expected number with a missing field and the variance of the sample proportion. (Answer: expected count $30$; proportion variance $0.0000073875$; the proportion converges almost surely to $0.015$.)
A device independently produces a warning on each use with probability $0.25$. For 400 uses, use Chebyshev’s inequality to bound the probability that the warning proportion differs from $0.25$ by at least $0.05$. (Answer: at most $0.1875$.)
On each hypothetical operating day, two independent components trigger a review with probabilities $0.05$ and $0.15$, respectively; daily pairs are IID. Define a review day as one with at least one trigger. For 1,000 days, find the review-day probability, expected review-day count, and long-run proportion limit. (Answer: probability $1-(0.95)(0.85)=0.1925$; expected count $192.5$; limit $0.1925$.)
Suppose a single Bernoulli variable $Z$ has $P(Z=1)=0.30$, and every recorded trial reuses that same value, so $I_i=Z$ for all $i$. Each indicator has marginal success probability $0.30$, but the indicators are dependent. What does the sample proportion equal, and does it converge almost surely to $0.30$? (Answer: $\hat{p}_n=Z$ for every $n$; it does not converge almost surely to $0.30$, because it equals either 0 or 1.)

The LLN for Bernoulli Sample Proportions

What You'll Learn

From Repeated Events to a Sample Proportion