Strong-Law Applications to Frequencies and Averages

From the Strong Law to Long-Run Frequencies

Repeated observations are often summarized by a sample average or by the fraction of observations satisfying a condition. Examples include the proportion of patients who respond to treatment, the frequency of a particular outcome, and the average of repeated measurements. The strong law explains why these summaries settle down: under suitable independence and moment assumptions, they converge almost surely to their population values.

Work on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). Let \((X_n)_{n\ge1}\) be an iid sequence of real-valued random variables, with state space \(S\subseteq\mathbb{R}\) and discrete time index \(n\ge1\). “Almost surely” means that the asserted convergence holds for every \(\omega\) in an event of probability one. The strong law with finite variance, established in the earlier material on the strong law, states that if \(\mathbb{E}[X_1]=\mu\) is finite and \(\operatorname{Var}(X_1)<\infty\), then

The applications below come from choosing what to count or average. For a measurable set \(A\subseteq S\), the indicator \(\mathbf{1}_{\{X_k\in A\}}\) equals \(1\) when observation \(k\) belongs to \(A\), and equals \(0\) otherwise. Its sample average is exactly the observed proportion of observations in \(A\).

The Frequency and Average Limits

Proof. For the first assertion, set \(Y_k=\mathbf{1}_{\{X_k\in A\}}\). Because each \(Y_k\) is a measurable function of \(X_k\), the variables \((Y_k)_{k\ge1}\) are iid. They are bounded between \(0\) and \(1\), so they have finite variance. Their common mean is \(\mathbb{E}[Y_1]=\mathbb{P}(X_1\in A)\). Applying the finite-variance strong law to \((Y_k)\) gives $$ \widehat{p}_n(A)=\frac{1}{n}\sum_{k=1}^{n}Y_k \longrightarrow \mathbb{P}(X_1\in A) \quad\text{almost surely}. $$ For the second assertion, \(g(X_1),g(X_2),\ldots\) are iid because the \(X_k\) are iid and the same measurable function is applied to each. The stated assumptions give them finite mean and finite variance. Applying the same strong law to this transformed sequence proves the claimed limit. \(\square\)

A useful consequence is that a finite collection of categories can be treated all at once. The individual limits hold almost surely; because there are only finitely many categories, their probability-one events have a probability-one intersection. Thus all category proportions converge simultaneously, not just one at a time.

Proof. By the frequency result, for each \(j\) there is an event \(E_j\) of probability one on which the frequency of \(A_j\) converges to \(\mathbb{P}(X_1\in A_j)\). The intersection \(E=\bigcap_{j=1}^{r}E_j\) has probability one: its complement is a finite union of probability-zero sets. On \(E\), all \(r\) limits hold. For every \(n\) and every observation \(X_k\), exactly one of the partition indicators is \(1\). Therefore $$ \sum_{j=1}^{r}\widehat{p}_n(A_j) =\frac{1}{n}\sum_{k=1}^{n}\sum_{j=1}^{r}\mathbf{1}_{\{X_k\in A_j\}} =\frac{1}{n}\sum_{k=1}^{n}1=1. $$ Taking the finite sum of the limits on \(E\) gives \(\sum_{j=1}^{r}\mathbb{P}(X_1\in A_j)=1\), as required. \(\square\)

Worked Applications

What the Limit Does—and Does Not—Say

An almost-sure limit is a statement about an infinite sequence of observations: with probability one, along that realized sequence, the empirical quantity approaches its stated value. It does not say that the frequency after a particular finite number of trials equals the probability, nor does it give a bound on the error at that sample size. A finite sample can have a proportion quite different from its limiting value. The strong law also does not assert that the proportion is exactly equal to its limit at any finite \(n\).

The independence assumption matters. Having the same marginal probability on every trial is not sufficient. For example, choose a single Bernoulli random variable \(Z\) with \(\mathbb{P}(Z=1)=\mathbb{P}(Z=0)=1/2\), and set \(X_n=Z\) for every \(n\). This is a sequence of dependent observations, each with probability \(1/2\) of being \(1\). But for every \(n\), $$ \frac1n\sum_{k=1}^{n}\mathbf{1}_{\{X_k=1\}} =\frac1n\sum_{k=1}^{n}Z=Z. $$ The frequency is therefore \(0\) with probability \(1/2\) and \(1\) with probability \(1/2\); it does not converge almost surely to \(1/2\). The repeated observations carry no new information after \(Z\) is chosen.

Common mistake. It is incorrect to say, “Each trial has probability \(1/2\) of success, so the observed proportion must be \(1/2\) in the first \(n\) trials.” That confuses the expected proportion with its realized value. For iid Bernoulli trials, linearity of expectation gives \(\mathbb{E}[\widehat{p}_n]=1/2\) at each \(n\), but this equality of expectations does not imply \(\widehat{p}_n=1/2\) for each sample. The strong law instead says that the sequence of proportions converges to \(1/2\) almost surely as \(n\) grows. In the dependent construction above, even that long-run conclusion fails.

For practical use, identify the quantity to average, check that its terms are iid, and verify the finite-variance condition when using the finite-variance strong law. For an event frequency, the terms are indicators and are automatically bounded; for a measurement average, the required moments must be checked for the measurements themselves. These steps distinguish a valid asymptotic conclusion from an unsupported claim about a finite sample or a dependent sequence.