Choosing a Strong Law Starts with the Hypotheses
A sample-average problem is not solved merely by identifying a plausible limiting mean. One must first check the assumptions of a strong law: are the variables independent and identically distributed, or merely independent? Is the expectation finite? If the variables are not identically distributed, is there a useful variance bound, and do their expected averages converge? The conclusion may concern averages centered by a fixed mean, or averages whose means vary with the index.
We work on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). All random variables below are real-valued, and almost-sure statements mean \(\mathbb{P}\)-a.e. For a sequence \((X_n)\), write \(S_n=\sum_{k=1}^nX_k\); its sample average is \(S_n/n\). Independence is understood for the whole sequence, not just pairwise independence.
The second result is centered: it controls deviations from the sum of the individual expectations. To obtain an uncentered limit, one must separately compute the deterministic sequence \(n^{-1}\sum_{k=1}^n\mathbb{E}[X_k]\). In particular, the variance condition alone does not say that the sample averages converge to a constant unless these expected averages converge.
Proof. For every \(n\), split the average into its centered and deterministic parts: \[ \frac{S_n}{n} = \frac1n\sum_{k=1}^n(X_k-\mathbb{E}[X_k]) + \frac1n\sum_{k=1}^n\mathbb{E}[X_k]. \] The first term tends to zero almost surely by the independent nonidentical strong law, since its independence, finite second moments, and variance-series hypothesis are assumed. The second term tends to \(L\) by hypothesis. On the probability-one event where the first convergence holds, the sum therefore tends to \(L\). \(\square\)
Identically Distributed Examples
Worked Example: Bernoulli Sample Averages
Let \((X_n)\) be independent Bernoulli variables with parameter \(p=1/3\): each \(X_n\) equals \(1\) with probability \(1/3\), and \(0\) with probability \(2/3\). Find the almost-sure limit of \(S_n/n\).
The variables are identically distributed and \(\mathbb{E}|X_1|=(1/3)\cdot1+(2/3)\cdot0=1/3<\infty\). The integrable IID strong law therefore applies, with limit \(\mathbb{E}[X_1]=1/3\). For a direct variance check, \(\operatorname{Var}(X_1)=\mathbb{E}[X_1^2]-(\mathbb{E}[X_1])^2=1/3-1/9=2/9\), so \[ \sum_{n=1}^{\infty}\frac{\operatorname{Var}(X_n)}{n^2} =\frac29\sum_{n=1}^{\infty}\frac1{n^2}<\infty. \] Thus the independent nonidentical criterion also applies, with the constant means centered at \(1/3\). Check: \(S_n/n\to1/3\) is consistent with the expectation of each individual trial, \(\mathbb{E}[X_n]=1/3\). This is an almost-sure limiting statement, not a claim that every finite sample average equals \(1/3\).
Worked Example: Gaussian Sample Averages
Let \((X_n)\) be independent and identically distributed with \(X_n\sim N(2,9)\), meaning that each variable has mean \(2\) and variance \(9\). Find the almost-sure limit of \(S_n/n\).
A Gaussian variable with finite mean and variance has finite absolute first moment, so \(\mathbb{E}|X_1|<\infty\). The integrable IID strong law gives \(S_n/n\to2\) almost surely. Alternatively, the independent nonidentical criterion applies: \(\operatorname{Var}(X_n)=9\), and \[ \sum_{n=1}^{\infty}\frac{\operatorname{Var}(X_n)}{n^2} =9\sum_{n=1}^{\infty}\frac1{n^2}<\infty. \] The centered averages tend to zero almost surely, and the averages of the means are \(n^{-1}\sum_{k=1}^n2=2\). Check: adding these two terms gives \(S_n/n\to0+2=2\). The variance \(9\) affects fluctuations but does not change the limiting mean.
Independent Variables with Changing Distributions
Worked Example: Alternating Bernoulli Parameters
Let \((X_n)\) be independent Bernoulli variables whose parameters alternate: \(p_n=1/3\) for odd \(n\), and \(p_n=2/3\) for even \(n\). Determine the limit of \(S_n/n\).
The means are \(\mathbb{E}[X_n]=p_n\). For \(n=2m\), there are \(m\) odd indices and \(m\) even indices, so \[ \frac1{2m}\sum_{k=1}^{2m}\mathbb{E}[X_k] =\frac{m(1/3)+m(2/3)}{2m} =\frac{m}{2m} =\frac12. \] For \(n=2m+1\), there are \(m+1\) odd indices and \(m\) even indices. Hence \[ \frac1{2m+1}\sum_{k=1}^{2m+1}\mathbb{E}[X_k] =\frac{(m+1)(1/3)+m(2/3)}{2m+1} =\frac{m+1/3}{2m+1}\longrightarrow\frac12. \] For each \(n\), \(\operatorname{Var}(X_n)=p_n(1-p_n)=2/9\), whether \(n\) is odd or even. Therefore \(\sum_n\operatorname{Var}(X_n)/n^2=(2/9)\sum_n1/n^2<\infty\). The proposition gives \(S_n/n\to1/2\) almost surely. Check: for \(n=2m\) the expected average is exactly \(1/2\); for \(n=2m+1\) its difference from \(1/2\) is \((m+1/3)/(2m+1)-1/2=-1/(6(2m+1))\), which tends to zero.
Worked Example: Gaussian Variables with Alternating Means and Variances
Let the \(X_n\) be independent Gaussian variables. For odd \(n\), let \(X_n\sim N(0,1)\); for even \(n\), let \(X_n\sim N(2,4)\). Determine the sample-average limit.
Every variance is finite and at most \(4\), so \[ \sum_{n=1}^{\infty}\frac{\operatorname{Var}(X_n)}{n^2} \le 4\sum_{n=1}^{\infty}\frac1{n^2}<\infty. \] For \(n=2m\), the sum of the means is \(m\cdot0+m\cdot2=2m\), and its average is \(2m/(2m)=1\). For \(n=2m+1\), there are \(m+1\) odd terms of mean zero and \(m\) even terms of mean \(2\); the mean sum is \(2m\), so the mean average is \(2m/(2m+1)\to1\). The proposition yields \(S_n/n\to1\) almost surely. Check: the centered average tends to zero, while both even- and odd-length expected averages tend to \(1\); their sum has the stated limit.
When the Usual Strong-Law Hypotheses Fail
Worked Example: IID Cauchy Averages Do Not Converge
Let \((X_n)\) be independent standard Cauchy variables, each with density \(f(x)=1/(\pi(1+x^2))\) on \(\mathbb{R}\). Show that \(S_n/n\) does not converge to a finite limit almost surely.
The Cauchy distribution has no finite absolute first moment, so the integrable IID strong law does not apply. Its variance is also infinite, so the variance-series criterion cannot be used. We can prove failure directly. For \(n\ge1\), symmetry and integration of the density give \[ \mathbb{P}(|X_n|>n) =\frac{2}{\pi}\int_n^\infty\frac{dx}{1+x^2} =\frac{2}{\pi}\arctan(1/n) \ge\frac{1}{\pi n}. \] For the inequality, \(\arctan u=\int_0^u(1+t^2)^{-1}dt\ge u/2\) when \(0\le u\le1\). The events \(A_n=\{|X_n|>n\}\) are independent, and \(\sum_n\mathbb{P}(A_n)\ge(1/\pi)\sum_n1/n=\infty\). To verify that infinitely many occur, for fixed \(N\le M\), independence and \(1-u\le e^{-u}\) give \[ \mathbb{P}\left(\bigcap_{n=N}^M A_n^c\right) =\prod_{n=N}^M(1-\mathbb{P}(A_n)) \le \exp\left(-\sum_{n=N}^M\mathbb{P}(A_n)\right)\longrightarrow0 \] as \(M\to\infty\). Thus the probability that no \(A_n\) occurs after \(N\) is zero; taking the countable union over \(N\), \(A_n\) occurs infinitely often almost surely.
Write \(A_n^{\mathrm{avg}}=S_n/n\) for the sample average, to distinguish it from the event \(A_n\). If \(A_n^{\mathrm{avg}}\) converged to a finite value, then it would be bounded and \[ \frac{X_n}{n}=A_n^{\mathrm{avg}}-\frac{n-1}{n}A_{n-1}^{\mathrm{avg}}\longrightarrow0. \] But \(|X_n|>n\) infinitely often means \(|X_n/n|>1\) infinitely often, a contradiction. Hence the sample averages do not converge to a finite value almost surely. Check: the argument uses the strict tail event \(|X_n|>n\); on each such event \(|X_n/n|>1\), which cannot happen infinitely often if \(X_n/n\to0\).
A common mistake is to assign the standard Cauchy variable “mean zero by symmetry” and then invoke the integrable IID strong law. Symmetry does give zero for a symmetric principal-value calculation, but that is not the Lebesgue expectation: \(\mathbb{E}|X_1|=\infty\), so \(X_1\) is not integrable and its expectation is not defined as a finite Lebesgue integral. The correction is to check \(\mathbb{E}|X_1|<\infty\) before using the integrable IID theorem. Failure of that hypothesis alone does not prove the strong law false; the independent-tail argument above supplies the separate proof of failure for these Cauchy averages.
For a concrete sequence, use the following order: identify whether the variables are IID or only independent; check the moments required by a candidate theorem; for nonidentical variables, test the variance series and compute the Cesàro averages of the means; then state precisely the almost-sure conclusion. A sufficient criterion that fails gives no conclusion by itself. It may be possible for a strong law to hold for other reasons, while the Cauchy example shows that a particular sequence can also genuinely fail to obey one.
Check Your Understanding
For each question, identify the theorem and verify its hypotheses before stating a limit.
- For independent IID Bernoulli variables with parameter \(p\in(0,1)\), what is the almost-sure limit of their sample averages, and which integrability condition is met?
- Suppose independent variables have means \(0,2,0,2,\ldots\) and variances at most \(5\). Check the variance-series condition and compute the limit of their average means.
- Why does the variance-series strong law not apply to independent standard Cauchy variables?
- In the Cauchy example, explain why infinitely many events \(|X_n|>n\) rule out convergence of \(S_n/n\).
- If the variance-series condition holds but \(n^{-1}\sum_{k=1}^n\mathbb{E}[X_k]\) has no limit, what does the centered strong law still conclude, and what does it not conclude?