Start With the Average and Its Proposed Limit
A law of large numbers is not a license to average any sequence and assume the result settles near a familiar constant. Before applying one, identify the random variables, their expectations, the proposed limit, and the assumptions linking observations across indices. In particular, centering, moment conditions, identical distribution, and independence are separate checks: some are needed for the proposed limit to make sense, while others are sufficient conditions for a particular law.
For random variables \(X_1,X_2,\ldots\), write \(S_n=\sum_{k=1}^n X_k\) and \(\bar X_n=S_n/n\). If \(E[|X_k|]<\infty\), define \(Y_k=X_k-E[X_k]\). Then \(\bar X_n=n^{-1}\sum_{k=1}^nY_k+n^{-1}\sum_{k=1}^nE[X_k]\). This decomposition separates random fluctuation from deterministic drift. Even if the centered averages vanish, the averages themselves approach a constant only if the averages of the expectations approach that constant.
Moment Conditions: What They Buy
For independent variables with finite variances, the variance of a centered sum is the sum of its variances. This gives a direct weak-law test. The test needs a second moment, but a finite second moment is not necessary for every law of large numbers. For example, the IID strong law applies when the common distribution has a finite absolute first moment, even if its variance is infinite. Thus a failed variance check means only that this particular variance argument is unavailable; it does not by itself prove that a law fails.
Proof. Let \(Y_k=X_k-E[X_k]\). Independence and finite variances give \[ E\left[\left(\frac{1}{n}\sum_{k=1}^nY_k\right)^2\right] = \operatorname{Var}\left(\frac{1}{n}\sum_{k=1}^nX_k\right) = \frac{1}{n^2}\sum_{k=1}^n\operatorname{Var}(X_k). \] For any \(\varepsilon>0\), Chebyshev’s inequality therefore yields \[ P\left(\left|\frac{1}{n}\sum_{k=1}^nY_k\right|>\varepsilon\right) \leq \frac{1}{\varepsilon^2n^2}\sum_{k=1}^n\operatorname{Var}(X_k) \longrightarrow 0. \] This proves convergence in probability of the centered average to zero. Adding the deterministic average of the expectations, when it converges to \(\mu\), proves the final claim. \(\square\)
For an almost-sure conclusion with non-identically distributed independent variables, one useful sufficient condition is stronger: if \(E[X_k]\) and \(\operatorname{Var}(X_k)\) are finite and \(\sum_{k=1}^{\infty}\operatorname{Var}(X_k)/k^2<\infty\), then \[ \frac{1}{n}\sum_{k=1}^n(X_k-E[X_k])\xrightarrow{a.s.}0. \] Indeed, the centered variables \(Y_k=X_k-E[X_k]\) are independent and satisfy \(\sum_k\operatorname{Var}(Y_k/k)=\sum_k\operatorname{Var}(X_k)/k^2<\infty\). Kolmogorov’s strong-law criterion implies that \(\sum_kY_k/k\) converges almost surely. Kronecker’s lemma, applied with \(b_n=n\), then gives \(n^{-1}\sum_{k=1}^nY_k\to0\) almost surely. As in the weak-law result, a limit for the uncentered averages also requires the averages of the expectations to converge.
Worked Examples: Checking the Conditions
Worked Example: An IID Discrete Average
Let \(X_1,X_2,\ldots\) be independent Bernoulli random variables with \(P(X_k=1)=p\) and \(P(X_k=0)=1-p\), where \(0<p<1\). Check the centering, moment, and dependence conditions for a law of large numbers.
For each \(k\), \(E[X_k]=p\), and \[ \operatorname{Var}(X_k)=E[X_k^2]-(E[X_k])^2=p-p^2=p(1-p). \] The variables are independent and identically distributed, and their common absolute first moment is \(E[|X_k|]=p<\infty\). The IID finite-mean strong law therefore gives \[ \bar X_n\xrightarrow{a.s.}p. \] The finite-variance weak-law calculation also applies directly: \[ \operatorname{Var}(\bar X_n)=\frac{1}{n^2}\sum_{k=1}^n p(1-p) =\frac{p(1-p)}{n}\longrightarrow0. \] Chebyshev’s inequality establishes \(\bar X_n\xrightarrow{p}p\). Here both routes work, but the finite-mean strong law gives the stronger conclusion.
Worked Example: A Continuous Heavy-Tailed Distribution
Let \(X_1,X_2,\ldots\) be IID with density \(f(x)=\tfrac{3}{2}x^{-5/2}\) for \(x\geq1\), and \(f(x)=0\) otherwise. Determine whether an infinite variance rules out a law of large numbers.
The density is normalized because \[ \int_1^\infty \frac{3}{2}x^{-5/2}\,dx =\frac{3}{2}\left[\frac{-2}{3}x^{-3/2}\right]_1^\infty=1. \] The first moment is finite: \[ E[X_k]=\int_1^\infty x\frac{3}{2}x^{-5/2}\,dx =\frac{3}{2}\int_1^\infty x^{-3/2}\,dx=3. \] But the second moment diverges: \[ E[X_k^2]=\frac{3}{2}\int_1^\infty x^{-1/2}\,dx=\infty. \] Consequently the finite-variance Chebyshev calculation is not available. Nevertheless, \(E[|X_k|]=3<\infty\), and the variables are IID. The IID finite-mean strong law gives \(\bar X_n\xrightarrow{a.s.}3\), and hence also convergence in probability to \(3\). This example shows why infinite variance must not be confused with infinite mean.
Worked Example: Independent but Not Identically Distributed
Suppose \(X_1,X_2,\ldots\) are independent Bernoulli variables with \(P(X_k=1)=3/4\) for odd \(k\) and \(P(X_k=1)=1/4\) for even \(k\). Check whether the average has a limit despite the lack of identical distributions.
Each variance is \(3/16\), since both success probabilities give \(p(1-p)=3/16\). Thus \[ \sum_{k=1}^{\infty}\frac{\operatorname{Var}(X_k)}{k^2} =\frac{3}{16}\sum_{k=1}^{\infty}\frac{1}{k^2}<\infty. \] The independent-sequence strong-law criterion gives \[ \frac{1}{n}\sum_{k=1}^n(X_k-E[X_k])\xrightarrow{a.s.}0. \] For even \(n\), there are \(n/2\) odd and \(n/2\) even indices, so \(n^{-1}\sum_{k=1}^nE[X_k]=1/2\). For odd \(n\), there is one extra odd index, giving \[ \frac{1}{n}\sum_{k=1}^nE[X_k] =\frac{1}{2}+\frac{1}{4n}\longrightarrow\frac{1}{2}. \] Adding the centered and deterministic parts shows \(\bar X_n\xrightarrow{a.s.}1/2\). Identical distribution was not needed here; independence and the variance summability condition controlled the centered fluctuations.
Worked Example: Finite Moments Do Not Repair Dependence
Let \(Z\) be Bernoulli with success probability \(1/2\), and set \(X_k=Z\) for every \(k\). Each variable has finite mean \(1/2\) and variance \(1/4\). Does the sample average converge to its expectation?
For every \(n\), \(\bar X_n=Z\). Therefore \[ P\left(\left|\bar X_n-\frac{1}{2}\right|>\frac{1}{4}\right)=1, \] so \(\bar X_n\) does not converge in probability to \(1/2\), and hence cannot converge almost surely to \(1/2\). The dependence is complete: \(\operatorname{Cov}(X_i,X_j)=1/4\) whenever \(i\ne j\). In particular, the variance of the average does not shrink: \[ \operatorname{Var}(\bar X_n)=\operatorname{Var}(Z)=\frac{1}{4}. \] Finite marginal moments alone do not make observations average out; dependence can preserve the same randomness at every sample size.
A Practical Assumption Check
For an IID sequence, a finite common mean \(\mu=E[X_1]\) is the natural target. For non-identically distributed variables, first check whether \(n^{-1}\sum_{k=1}^nE[X_k]\) itself converges.
A finite absolute first moment supports the IID finite-mean laws. If the mean does not exist, centering at that mean is not legitimate; if a nonnegative IID variable has infinite mean, consult the infinite-mean strong-law result rather than claiming a finite-mean limit.
Finite variances and independence allow the Chebyshev weak-law bound. For a strong law with independent, non-identically distributed variables, the summability condition on variances divided by \(k^2\) is a useful sufficient check. Neither test should be applied without its hypotheses.
A common mistake is to argue, “The observations have a finite variance, so their average converges to their mean.” This leaves out essential conditions: repeated dependence can prevent averaging, and non-identical expectations can produce a drifting average target. The correction is to state the exact law being used, verify its dependence and moment hypotheses, and separately calculate the average expectation. Conversely, requiring finite variance for every IID strong-law application is too restrictive: the heavy-tailed example has infinite variance but a finite mean and still obeys the strong law.
These checks matter because a convergence statement has both a mode and a target. “The average converges” is incomplete unless it specifies convergence in probability or almost surely and identifies the limit. A useful diagnosis therefore reports what is established, which assumptions justify it, and what fails when a condition is absent. Failure of a sufficient condition does not automatically imply failure of the conclusion; it may simply call for a different theorem.
Check Your Understanding
For each question, identify the relevant assumptions before choosing a conclusion.
- For IID observations with finite mean but infinite variance, which standard law of large numbers still applies, and what is its limit?
- For independent variables with finite variances, what quantity should be checked to use the variance-based weak-law argument for the centered average?
- In the alternating Bernoulli example, why does the lack of identical distributions not prevent convergence to \(1/2\)?
- For \(X_k=Z\) at every index, which dependence feature prevents the average from approaching \(E[Z]\)?
- If the centered averages of a non-identically distributed sequence converge to zero, what additional deterministic quantity must be checked before claiming a limit for the uncentered averages?