From Weighted Series to Averages
Kronecker's lemma is a deterministic bridge between two kinds of convergence. It starts with convergence of a series whose \(k\)th term is divided by a growing weight, and concludes that the corresponding unweighted partial sums are small compared with that weight. In probability, this converts almost-sure convergence of weighted centered sums into a strong law for ordinary sample averages.
We first work with real sequences; no measure space is needed for the lemma. The probabilistic application will take place on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), and convergence there will be almost surely. Let \((b_n)\) be a sequence of positive real numbers. The key hypotheses are that \(b_n\) is nondecreasing and \(b_n\to\infty\). The weights need not be strictly increasing.
Proof by Finite Summation by Parts
Set \(s_0=0\) and \(s_n=\sum_{k=1}^{n}x_k/b_k\) for \(n\ge1\). The series hypothesis says \(s_n\to s\) for some finite \(s\). The finite-sum identity below is the essential step: it expresses the unweighted sum using the partial sums of the weighted series and the increments of the weights.
Proof. Since \(x_k=b_k(s_k-s_{k-1})\), for every \(n\ge1\) we have $$ \sum_{k=1}^{n}x_k =\sum_{k=1}^{n}b_k(s_k-s_{k-1}) =b_ns_n-\sum_{k=1}^{n-1}(b_{k+1}-b_k)s_k. $$ For \(n=1\), the sum on the right is empty and the identity reads \(x_1=b_1s_1\), so it also covers that case. Dividing by \(b_n>0\), and adding and subtracting \(s\), gives $$ \frac{1}{b_n}\sum_{k=1}^{n}x_k =(s_n-s)+\frac{b_1}{b_n}s +\frac{1}{b_n}\sum_{k=1}^{n-1}(b_{k+1}-b_k)(s-s_k). $$ The first term tends to zero because \(s_n\to s\). The second tends to zero because \(b_n\to\infty\). To control the last term, fix \(\varepsilon>0\). Choose \(K\) so that \(|s-s_k|<\varepsilon\) whenever \(k\ge K\). The finitely many terms with \(k<K\), divided by \(b_n\), tend to zero as \(n\to\infty\); their numerator is a fixed finite number. For \(n>K\), the remaining terms satisfy $$ \left|\frac{1}{b_n}\sum_{k=K}^{n-1}(b_{k+1}-b_k)(s-s_k)\right| \le \frac{\varepsilon}{b_n}\sum_{k=K}^{n-1}(b_{k+1}-b_k) =\varepsilon\frac{b_n-b_K}{b_n} \le\varepsilon. $$ Thus the limsup of the absolute value of the last term is at most \(\varepsilon\). Since this holds for every \(\varepsilon>0\), that term tends to zero. All terms in the identity therefore tend to zero, proving the conclusion. \(\square\)
The proof also indicates why the conditions fit together. Nondecreasing weights make every increment \(b_{k+1}-b_k\) nonnegative, so the tail error is controlled by the telescoping sum of increments. Divergence of \(b_n\) makes the contribution of any fixed initial segment vanish after division by \(b_n\).
Worked Example: Alternating Terms with Linear Weights
Take \(b_k=k\) and \(x_k=(-1)^{k+1}\). The weighted series is the alternating harmonic series \(\sum_{k\ge1}(-1)^{k+1}/k\). Its terms decrease in magnitude to zero, so the alternating-series criterion gives convergence. Kronecker's lemma then predicts that \(n^{-1}\sum_{k=1}^n(-1)^{k+1}\to0\).
Compute the partial sums directly. If \(n=2r\), the terms cancel in pairs: $$ \sum_{k=1}^{2r}(-1)^{k+1}=(1-1)+\cdots+(1-1)=0. $$ If \(n=2r+1\), there are \(r\) cancelling pairs followed by one \(+1\), so the sum is \(1\). Consequently the normalized partial sum is \(0\) for even \(n\), and \(1/n\) for odd \(n\). Both subsequences tend to zero. Check: at \(n=4\) the normalized sum is \(0/4=0\); at \(n=5\) it is \(1/5\), in agreement with the two formulas.
Worked Example: A Faster-Growing Weight
Let \(x_k=1\) and \(b_k=k^2\). The weighted series is \(\sum_{k\ge1}1/k^2\), which converges, and the weights are positive, nondecreasing, and tend to infinity. Therefore Kronecker's lemma gives \(n^{-2}\sum_{k=1}^n1\to0\).
In this case the partial sum is exactly \(n\), so the normalized value is $$ \frac{1}{n^2}\sum_{k=1}^{n}1=\frac{n}{n^2}=\frac{1}{n}\longrightarrow0. $$ Check: at \(n=4\), the numerator is \(4\) and the denominator is \(16\), giving \(1/4\), which equals \(1/n\). This example illustrates the conclusion without requiring the weighted-series limit to be zero: the series has the finite value \(\sum_{k\ge1}1/k^2\), while the normalized unweighted sums tend to zero.
The Almost-Sure Application
Proof. Let \(\Omega_0\in\mathcal{F}\) be an event with \(\mathbb{P}(\Omega_0)=1\) on which the displayed series converges. Fix any \(\omega\in\Omega_0\), and set \(x_k=X_k(\omega)-\mathbb{E}[X_k]\) and \(b_k=k\). These are real numbers and positive, nondecreasing weights tending to infinity. The series hypothesis holds for this sequence at the fixed \(\omega\), so Kronecker's lemma gives \(n^{-1}\sum_{k=1}^n(X_k(\omega)-\mathbb{E}[X_k])\to0\). Since the argument holds at every \(\omega\in\Omega_0\), the convergence is almost sure. If \(\mathbb{E}[X_k]=m\) for every \(k\), then \(n^{-1}\sum_{k=1}^nX_k=m+n^{-1}\sum_{k=1}^n(X_k-m)\), proving the final statement. \(\square\)
The corollary is an implication, not by itself a guarantee that the weighted series converges. The next example checks that hypothesis for a specific sequence of independent random variables. We use two standard results from probability: Kolmogorov's maximal inequality for independent, mean-zero, square-integrable random variables, and the first Borel–Cantelli lemma. In the form needed here, the maximal inequality says that if \(Z_k\) are independent, mean zero, and have finite variances, then for \(N\ge M\) and \(\lambda>0\), $$ \mathbb{P}\left(\max_{M\le r\le N}\left|\sum_{k=M}^{r}Z_k\right|>\lambda\right) \le \frac{\sum_{k=M}^{N}\mathbb{E}[Z_k^2]}{\lambda^2}. $$ The first Borel–Cantelli lemma says that if \(\sum_j\mathbb{P}(A_j)<\infty\), then with probability one only finitely many of the events \(A_j\) occur.
Worked Example: Bernoulli Sample Averages
Let \(X_k\) be independent Bernoulli random variables with \(\mathbb{P}(X_k=1)=1/3\) and \(\mathbb{P}(X_k=0)=2/3\), on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). We verify that \(\sum_{k\ge1}(X_k-1/3)/k\) converges almost surely, then apply the corollary.
Set \(Y_k=X_k-1/3\) and \(Z_k=Y_k/k\). Each \(Y_k\) has mean zero. Its values are \(2/3\) with probability \(1/3\) and \(-1/3\) with probability \(2/3\), so $$ \mathbb{E}[Y_k^2]=\frac13\left(\frac23\right)^2+\frac23\left(-\frac13\right)^2 =\frac{4}{27}+\frac{2}{27}=\frac29. $$ Thus the \(Z_k\) are independent, mean zero, and \(\mathbb{E}[Z_k^2]=2/(9k^2)\).
For each integer \(J\ge0\), apply Kolmogorov's maximal inequality to indices \(2^J\) through \(N\), and then let \(N\to\infty\). The events with finite upper endpoint increase to the event with supremum over all \(n\ge2^J\); continuity from below of probability justifies this limit. Also, $$ \sum_{k=2^J}^{\infty}\frac{1}{k^2} \le \frac{1}{(2^J)^2}+\int_{2^J}^{\infty}\frac{dx}{x^2} =\frac{1}{2^{2J}}+\frac{1}{2^J} \le\frac{2}{2^J}. $$ The integral bound applies to the decreasing function \(x^{-2}\), and the last inequality follows from \(2^{-2J}\le2^{-J}\). Taking \(\lambda=2^{-J/4}\), we obtain $$ \mathbb{P}\left(\sup_{n\ge2^J}\left|\sum_{k=2^J}^{n}Z_k\right|>2^{-J/4}\right) \le \frac{(2/9)(2/2^J)}{(2^{-J/4})^2} =\frac49\,2^{-J/2}. $$ The right-hand side is summable over \(J\), since it is a constant times a geometric sequence with ratio \(2^{-1/2}<1\). By the first Borel–Cantelli lemma, almost surely, for all sufficiently large \(J\), every partial sum starting at \(2^J\) has absolute value at most \(2^{-J/4}\).
On that probability-one event, the series \(\sum_k Z_k\) is Cauchy. Indeed, for \(r\ge q\ge2^J\), its sum from \(q\) to \(r\) is the difference of two partial sums starting at \(2^J\) (the second is zero if \(q=2^J\)); its absolute value is at most \(2\cdot2^{-J/4}\). This bound tends to zero as \(J\to\infty\). Hence \(\sum_k(X_k-1/3)/k\) converges almost surely. Applying Kronecker's lemma with \(b_n=n\) now yields $$ \frac1n\sum_{k=1}^{n}(X_k-1/3)\longrightarrow0 \quad\mathbb{P}\text{-a.s.}, \qquad \frac1n\sum_{k=1}^{n}X_k\longrightarrow\frac13 \quad\mathbb{P}\text{-a.s.} $$ Check: for \(n=3\), the centered average is \((X_1+X_2+X_3)/3-1/3\), since \(3(1/3)/3=1/3\). Thus adding the mean \(1/3\) to the centered average gives the sample average, as in the conclusion.
A Common Pitfall: The Weights Must Diverge
It is not enough for the weighted series to converge: the weights must also tend to infinity. For a concrete counterexample, take \(b_n=1\) for every \(n\), \(x_1=1\), and \(x_n=0\) for \(n\ge2\). The weighted series is \(1+0+0+\cdots\), which converges, but $$ \frac{1}{b_n}\sum_{k=1}^{n}x_k=1 $$ for every \(n\), not zero. This sequence of weights is positive and nondecreasing, but it does not diverge. In the proof, the fixed initial contribution would not disappear because \(b_n\) does not grow.
A second error is to treat convergence of \(\sum x_k/b_k\) as if it directly said that \(\sum_{k=1}^n x_k\) is small. The convergent series controls the partial sums \(s_k\), not the unweighted sums by themselves. The summation-by-parts identity is what connects them, and its weighted increment terms are controlled using the monotonicity of \(b_n\). In the probability application, one must also establish convergence of the weighted centered series; the lemma does not supply that hypothesis.
Check Your Understanding
For each question, identify the sequence to which Kronecker's lemma would apply and check every hypothesis.
- Suppose \(b_n=3n+2\) and \(\sum_{k\ge1}x_k/b_k\) converges. What limit does Kronecker's lemma give for \(b_n^{-1}\sum_{k=1}^n x_k\)?
- For \(b_n=n\), write the summation-by-parts identity for \(\sum_{k=1}^n x_k\) in terms of \(s_k=\sum_{j=1}^k x_j/j\).
- Why does the proof require the weights to be nondecreasing when bounding the tail involving \(s-s_k\)?
- If \(\sum_{k\ge1}(X_k-\mathbb{E}[X_k])/k\) converges almost surely and \(\mathbb{E}[X_k]=m\) for every \(k\), what almost-sure limit follows for the sample averages?
- Give a positive, nondecreasing sequence of weights that does not tend to infinity, and explain why the stated conclusion need not hold for it.