From a Variance Sum to an Almost-Sure Limit
For independent variables with a common distribution, a strong law often follows from a single variance assumption. When the distributions vary with the index, the relevant control can instead be distributed across the sequence: the variances need not be equal or even bounded, provided their appropriately weighted sum is finite. The proof turns that condition into convergence of a random series, then converts that series convergence into convergence of averages.
Throughout, \((\Omega,\mathcal{F},\mathbb{P})\) is a probability space, and all random variables in this tutorial are real-valued and measurable. Independence means joint independence of the sequence, and expectations and variances are taken with respect to \(\mathbb{P}\). Almost-sure statements mean that the stated property holds outside a set of probability zero. No assumption that the probability space is complete is needed.
The variables need not have the same distribution. The centering assumption is on each \(X_n\), while the summability condition controls the fluctuations after division by the index. We prove the result using a finite-sum maximal bound and a deterministic lemma.
The Maximal Bound and the Series Argument
Proof. For \(1\le k\le r\), let \(A_k\) be the event that \(|T_k|\ge\lambda\) and \(|T_i|<\lambda\) for each \(i<k\). The events \(A_k\) are disjoint, and their union is exactly \(\{\max_{1\le k\le r}|T_k|\ge\lambda\}\): on that event, there is a first index at which the threshold is reached. Write \(R_k=T_r-T_k\). The random variable \(R_k\) is independent of \(\mathcal{F}_k=\sigma(Z_1,\ldots,Z_k)\), and \(\mathbb{E}[R_k]=0\). Since \(A_k\) and \(T_k\) are \(\mathcal{F}_k\)-measurable, independence gives \(\mathbb{E}[T_kR_k\chi_{A_k}]=0\). Therefore $$ \mathbb{E}[T_r^2\chi_{A_k}] =\mathbb{E}[T_k^2\chi_{A_k}]+\mathbb{E}[R_k^2\chi_{A_k}] \ge \lambda^2\mathbb{P}(A_k). $$ Summing over the disjoint events and using \(T_r^2\ge0\) gives $$ \lambda^2\sum_{k=1}^{r}\mathbb{P}(A_k) \le \mathbb{E}[T_r^2\chi_{\cup_k A_k}] \le \mathbb{E}[T_r^2] =\sum_{j=1}^{r}\operatorname{Var}(Z_j). $$ The last equality follows by expanding the square: the cross terms have expectation zero by independence and centering. Dividing by \(\lambda^2\) proves the inequality. \(\square\)
Proof. Put \(t_j=a_j/j\) and \(T_k=\sum_{j=1}^{k}t_j\). A finite rearrangement gives $$ \sum_{j=1}^{n}a_j=\sum_{j=1}^{n}jt_j=nT_n-\sum_{j=1}^{n-1}T_j. $$ If \(T_k\to T\), then $$ \frac1n\sum_{j=1}^{n}a_j=T_n-\frac1n\sum_{j=1}^{n-1}T_j. $$ The averages on the right converge to \(T\): for any \(\varepsilon>0\), choose \(K\) so that \(|T_j-T|<\varepsilon\) for \(j\ge K\); the contribution of the finitely many terms \(j<K\), divided by \(n\), tends to zero, and the average contribution of the remaining differences is at most \(\varepsilon\). Also \(T_n\to T\), so the displayed difference tends to zero. \(\square\)
Proof of the theorem. Define \(Y_j=X_j/j\). These variables are independent, mean zero, and have finite variances, with $$ \sum_{j=1}^{\infty}\operatorname{Var}(Y_j) =\sum_{j=1}^{\infty}\frac{\operatorname{Var}(X_j)}{j^2}<\infty. $$ For each integer \(k\ge1\), choose an integer \(N_k\) such that \(\sum_{j=N_k}^{\infty}\operatorname{Var}(Y_j)\le 2^{-4k}\); this is possible because the series of variances converges. For \(M\ge N_k\), Kolmogorov’s maximal inequality applied to \(Y_{N_k},\ldots,Y_M\) gives $$ \mathbb{P}\!\left(\max_{N_k\le m\le M}\left|\sum_{j=N_k}^{m}Y_j\right|\ge 2^{-k}\right) \le 2^{2k}\sum_{j=N_k}^{M}\operatorname{Var}(Y_j) \le 2^{-2k}. $$ The events \(\{\max_{N_k\le m\le M}|\sum_{j=N_k}^{m}Y_j|>2^{-k}\}\) increase as \(M\) increases, and their union is exactly \(\{\sup_{m\ge N_k}|\sum_{j=N_k}^{m}Y_j|>2^{-k}\}\). Each finite event is contained in the event bounded in the preceding display. Continuity of probability from below therefore yields $$ \mathbb{P}\!\left(\sup_{m\ge N_k}\left|\sum_{j=N_k}^{m}Y_j\right|>2^{-k}\right)\le 2^{-2k}. $$ The strict threshold here is useful: a supremum strictly greater than \(2^{-k}\) is attained beyond that threshold by some finite partial sum.
The probabilities on the left are summable, since \(\sum_{k=1}^{\infty}2^{-2k}=1/3\). The first Borel–Cantelli lemma says that events with summable probabilities occur only finitely often almost surely. In this direction it follows directly from the union bound: the probability that any event with index \(k\ge K\) occurs is at most \(\sum_{k\ge K}2^{-2k}\), which tends to zero as \(K\to\infty\). Thus, almost surely, for all sufficiently large \(k\), $$ \sup_{m\ge N_k}\left|\sum_{j=N_k}^{m}Y_j\right|\le 2^{-k}. $$ For any \(q\ge p\ge N_k\), subtract the two partial sums starting at \(N_k\) (taking the sum up to \(p-1\) to be zero if \(p=N_k\)). The triangle inequality gives \(\left|\sum_{j=p}^{q}Y_j\right|\le 2^{1-k}\). As \(k\to\infty\), this bound tends to zero, so the partial sums of \(\sum_jY_j\) are Cauchy almost surely. Completeness of \(\mathbb{R}\) implies that \(\sum_jX_j/j\) converges almost surely. On every sample point where it converges, Kronecker’s lemma applied to \(a_j=X_j\) gives \(\frac1n\sum_{j=1}^{n}X_j\to0\). This proves the theorem. \(\square\)
Worked Applications
Worked Example: Rademacher Variables with Decreasing Amplitude
Let \(\varepsilon_n\) be independent variables with \(\mathbb{P}(\varepsilon_n=1)=\mathbb{P}(\varepsilon_n=-1)=1/2\), and set \(X_n=\varepsilon_n/\sqrt{n}\). Compute the variance condition and state the resulting almost-sure limit.
For each \(n\), \(\mathbb{E}[X_n]=(1/\sqrt n)(1/2-1/2)=0\). Also \(X_n^2=1/n\) for either value of \(\varepsilon_n\), so \(\operatorname{Var}(X_n)=\mathbb{E}[X_n^2]=1/n\). Consequently, $$ \sum_{n=1}^{\infty}\frac{\operatorname{Var}(X_n)}{n^2} =\sum_{n=1}^{\infty}\frac1{n^3} \le 1+\int_1^\infty x^{-3}\,dx =1+\frac12=\frac32. $$ The integral comparison applies because \(x^{-3}\) is decreasing on \([1,\infty)\), so the required sum is finite. The theorem gives \(\frac1n\sum_{j=1}^{n}\varepsilon_j/\sqrt{j}\to0\) almost surely. Check: the first three terms of the variance sum are \(1,\frac18,\frac1{27}\), obtained by substituting \(n=1,2,3\) into \(1/n^3\); each is nonnegative, as required.
Worked Example: Independent Variables with Rare Large Values
For each \(n\ge1\), let \(X_n\) take the values \(n,-n,0\) with probabilities \(1/(2n^4),1/(2n^4),1-1/n^4\), respectively, and suppose the variables are independent. Verify the assumptions, including the case \(n=1\).
The probabilities sum to \(1/(2n^4)+1/(2n^4)+1-1/n^4=1\), and \(1-1/n^4\ge0\) for every \(n\ge1\); when \(n=1\), the zero value has probability zero. The mean and second moment are $$ \mathbb{E}[X_n]=n\frac1{2n^4}-n\frac1{2n^4}=0,\qquad \mathbb{E}[X_n^2]=n^2\frac1{2n^4}+n^2\frac1{2n^4}=\frac1{n^2}. $$ Thus \(\operatorname{Var}(X_n)=1/n^2\), and $$ \sum_{n=1}^{\infty}\frac{\operatorname{Var}(X_n)}{n^2} =\sum_{n=1}^{\infty}\frac1{n^4} \le 1+\int_1^\infty x^{-4}\,dx =1+\frac13=\frac43. $$ Here \(x^{-4}\) is decreasing and positive on \([1,\infty)\), so the integral comparison applies. The theorem shows that the normalized partial sums converge to zero almost surely, even though the possible nonzero values of \(X_n\) grow with \(n\). Check: at \(n=2\), the probabilities of \(2\) and \(-2\) are each \(1/32\); their second-moment contributions are each \(4/32=1/8\), giving variance \(1/4=1/n^2\).
Worked Example: Geometrically Decreasing Variances
Let \(\eta_n\) be independent, with probabilities \(1/2\) each of taking the values \(1\) and \(-1\), and set \(X_n=2^{-n}\eta_n\). Verify the condition and identify the conclusion.
Symmetry gives \(\mathbb{E}[X_n]=0\), and \(X_n^2=4^{-n}\), so \(\operatorname{Var}(X_n)=4^{-n}\). Since \(n^{-2}\le1\), $$ \sum_{n=1}^{\infty}\frac{\operatorname{Var}(X_n)}{n^2} =\sum_{n=1}^{\infty}\frac{4^{-n}}{n^2} \le\sum_{n=1}^{\infty}4^{-n} =\frac{1/4}{1-1/4}=\frac13. $$ The geometric series formula applies because its ratio is \(1/4\), whose absolute value is less than \(1\). Hence \(n^{-1}\sum_{j=1}^{n}2^{-j}\eta_j\to0\) almost surely. Check: the first two terms of the weighted variance sum are \(1/4\) and \(1/64\), from \(4^{-1}/1^2\) and \(4^{-2}/2^2\); both are bounded above by the corresponding geometric terms \(1/4\) and \(1/16\).
Why Centering and Independence Matter
A common error is to check only the displayed variance sum and overlook the mean-zero hypothesis. For example, let \(X_n=1\) deterministically for every \(n\). Constant random variables are independent, \(\operatorname{Var}(X_n)=0\), and thus \(\sum_n\operatorname{Var}(X_n)/n^2=0\). But \(\frac1n\sum_{j=1}^{n}X_j=\frac1n\cdot n=1\) for every \(n\), not zero. The variance condition controls centered fluctuations; it does not remove a nonzero average drift.
Independence is also a substantive assumption. Let \(Z\) take the values \(1\) and \(-1\) with probability \(1/2\) each, and define \(X_n=Z\) for all \(n\). Each variable has mean zero and variance \(1\), so the weighted variance sum is \(\sum_n1/n^2<\infty\). Nevertheless, the variables are not independent, and \(\frac1n\sum_{j=1}^{n}X_j=Z\) for every \(n\). Since \(\mathbb{P}(Z=0)=0\), this sequence does not converge to zero almost surely. In the proof, independence is used in the maximal inequality to make future increments have zero cross term with the event of an earlier threshold crossing.
The theorem is useful when a sequence is independent but its distributions change with time: one can check the contribution of each index through \(\operatorname{Var}(X_n)/n^2\), rather than impose identical distribution or a uniform variance bound. The conclusion concerns normalized partial sums almost surely, not convergence in expectation or convergence in probability alone.
Check Your Understanding
For each question, identify which hypotheses are used and distinguish the weighted variance condition from an unweighted variance sum.
- If independent mean-zero variables satisfy \(\operatorname{Var}(X_n)=1\) for every \(n\), does the theorem apply? Verify the relevant series.
- For independent mean-zero variables with \(\operatorname{Var}(X_n)=1/n\), compare the weighted variance sum with \(\sum_n\operatorname{Var}(X_n)\). Which one is finite?
- In the maximal-inequality argument, why is the event that a supremum is strictly greater than a threshold a union of finite-partial-sum events?
- Give a sequence with zero variances satisfying the weighted variance condition for which the normalized sums do not converge to zero. Which theorem hypothesis fails?
- Explain how convergence of \(\sum_jX_j/j\) yields convergence of \(n^{-1}\sum_{j=1}^{n}X_j\), naming the deterministic lemma used.