Kronecker's Lemma for Strong Laws of Averages

Why a Convergent Series Can Control Averages

Strong laws often ask whether a sequence of partial sums becomes small after division by a growing scale. A useful route is to first establish convergence of a different series, whose terms are divided by that scale, and then translate the series convergence into a statement about the original partial sums. Kronecker’s lemma supplies exactly this translation.

The lemma is deterministic: it makes no probabilistic assumptions about its terms. Probability enters when a convergence theorem, such as Kolmogorov’s convergence criterion for independent random variables, establishes the required series convergence almost surely. The resulting combination is a powerful way to turn variance summability into strong laws.

Statement of Kronecker’s Lemma

Lemma (Kronecker’s lemma): Let $b_1,b_2,\ldots$ be positive real numbers such that $b_{n+1}\ge b_n$ for every $n$ and $b_n\to\infty$. If the real series $\sum_{n=1}^{\infty} a_n/b_n$ converges, then $$ \frac{1}{b_n}\sum_{k=1}^{n} a_k \longrightarrow 0. $$

The weights need not increase strictly, but they must be nondecreasing and diverge to infinity. The series is assumed to converge in the ordinary sense; absolute convergence is not required. The same assertion applies to complex terms by applying the argument to real and imaginary parts.

Worked Example: Harmonic Partial Sums

Take $b_n=n$ and $a_n=1/n$. The series required by the lemma is $$ \sum_{n=1}^{\infty}\frac{a_n}{b_n} =\sum_{n=1}^{\infty}\frac{1}{n^2}, $$ which converges. The weights $b_n=n$ are positive, nondecreasing, and tend to infinity. Kronecker’s lemma therefore gives $$ \frac{1}{n}\sum_{k=1}^{n}\frac{1}{k}\longrightarrow 0. $$

This conclusion can also be checked directly. Since $x\mapsto 1/x$ decreases on $[1,\infty)$, for $n\ge 1$, $$ \sum_{k=1}^{n}\frac{1}{k}\le 1+\int_1^n\frac{dx}{x}=1+\log n. $$ Consequently, the average is nonnegative and bounded above by $(1+\log n)/n$, which tends to zero. The lemma obtains the limit from convergence of the squared-reciprocal series without needing this separate harmonic-sum bound.

Proof by Summation by Parts

Set $y_k=a_k/b_k$, and let $S_n=\sum_{k=1}^{n}y_k$. By assumption, $S_n\to S$ for some finite $S$. Summation by parts gives an identity for the partial sums of $a_k=b_k y_k$. The proof then separates the limiting constant $S$ from the error $S_k-S$.

Use the partial sums of the convergent series.
Writing $y_k=S_k-S_{k-1}$, with $S_0=0$, yields an exact summation-by-parts formula.

Separate the limit and the remainder.
Write each $S_k$ as $S+(S_k-S)$. The constant terms cancel except for a fixed boundary term.

Bound the remainder after division by $b_n$.
The tail errors $S_k-S$ are uniformly small for large $k$, while the finitely many earlier terms vanish after division by $b_n$.

For completeness, the summation-by-parts identity and bounds are shown explicitly. For $n\ge 1$, $$ \sum_{k=1}^{n}a_k =\sum_{k=1}^{n}b_k(S_k-S_{k-1}) =b_nS_n-\sum_{k=1}^{n-1}(b_{k+1}-b_k)S_k. $$ Let $r_k=S_k-S$. Substituting $S_k=S+r_k$ into this identity gives $$ \sum_{k=1}^{n}a_k =b_1S+b_nr_n-\sum_{k=1}^{n-1}(b_{k+1}-b_k)r_k. $$ Indeed, the coefficient of $S$ is $b_n-\sum_{k=1}^{n-1}(b_{k+1}-b_k)=b_1$. Divide by $b_n$. The fixed term $b_1S/b_n$ tends to zero.

It remains to show that the terms involving $r_k$ tend to zero after this division. Given $\varepsilon>0$, choose $K$ so that $|r_k|<\varepsilon$ for every $k\ge K$. The finitely many terms with $k<K$, after division by $b_n$, tend to zero because their numerator is fixed and $b_n\to\infty$. For $n\ge K$, the remaining terms satisfy $$ \frac{|b_nr_n|}{b_n} +\frac{1}{b_n}\sum_{k=K}^{n-1}(b_{k+1}-b_k)|r_k| \le \varepsilon+\frac{\varepsilon}{b_n}(b_n-b_K) \le 2\varepsilon. $$ Here nondecreasing weights ensure $b_{k+1}-b_k\ge0$ and make the increments telescope. Since $\varepsilon$ is arbitrary, the remainder tends to zero. This proves Kronecker’s lemma. $\square$

From Series Convergence to Almost-Sure Limits

Here is the probabilistic connection. Kolmogorov’s convergence criterion states that if $Y_1,Y_2,\ldots$ are independent, mean-zero random variables with finite variances and $\sum_{n=1}^{\infty}\operatorname{Var}(Y_n)<\infty$, then the series $\sum_{n=1}^{\infty}Y_n$ converges almost surely. Apply this established criterion to $Y_n=X_n/b_n$, then apply Kronecker’s lemma to the resulting series.

Theorem: Let $X_1,X_2,\ldots$ be independent random variables with $E[X_n]=0$ and $\operatorname{Var}(X_n)<\infty$. Let $b_n$ be positive, nondecreasing, and divergent to infinity. If $$ \sum_{n=1}^{\infty}\frac{\operatorname{Var}(X_n)}{b_n^2}<\infty, $$ then $$ \frac{1}{b_n}\sum_{k=1}^{n}X_k\xrightarrow{a.s.}0. $$

Proof. The variables $Y_n=X_n/b_n$ are independent and mean zero, and $$ \sum_{n=1}^{\infty}\operatorname{Var}(Y_n) =\sum_{n=1}^{\infty}\frac{\operatorname{Var}(X_n)}{b_n^2}<\infty. $$ Kolmogorov’s convergence criterion therefore gives convergence of $\sum_{n=1}^{\infty}X_n/b_n$ almost surely. On every outcome where this series converges, Kronecker’s lemma, applied to the numerical sequence of observed values $X_n$, gives $b_n^{-1}\sum_{k=1}^{n}X_k\to0$. The series converges on an event of probability one, so the asserted limit holds almost surely. $\square$

Taking $b_n=n$ yields a strong law for centered averages whenever $\sum_n\operatorname{Var}(X_n)/n^2$ is finite. The condition is satisfied for independent, identically distributed variables with finite variance: if their variance is $\sigma^2<\infty$, then the variance sum is $\sigma^2\sum_n 1/n^2<\infty$. Thus, for such variables with mean $\mu$, applying the theorem to $X_n-\mu$ gives $n^{-1}\sum_{k=1}^{n}X_k\xrightarrow{a.s.}\mu$.

Worked Example: An IID Bernoulli Sample

Let $Z_1,Z_2,\ldots$ be independent Bernoulli random variables with $P(Z_n=1)=1/3$ and $P(Z_n=0)=2/3$. Their expectations are $1/3$, and their variances are $$ \operatorname{Var}(Z_n)=\frac{1}{3}\left(1-\frac{1}{3}\right)=\frac{2}{9}. $$ Set $X_n=Z_n-1/3$ and $b_n=n$. Then $E[X_n]=0$, and $$ \sum_{n=1}^{\infty}\frac{\operatorname{Var}(X_n)}{b_n^2} =\frac{2}{9}\sum_{n=1}^{\infty}\frac{1}{n^2}<\infty. $$ The theorem applies, giving $n^{-1}\sum_{k=1}^{n}(Z_k-1/3)\xrightarrow{a.s.}0$. Adding $1/3$ to these averages proves $n^{-1}\sum_{k=1}^{n}Z_k\xrightarrow{a.s.}1/3$. The centered-sum conclusion is precisely the ordinary-average statement after accounting for the mean.

Worked Example: A Stronger Normalization Than the Ordinary Average

Suppose $X_1,X_2,\ldots$ are independent with $X_n\sim N(0,1)$. Choose $b_n=n^{3/4}$. These weights are positive and increasing to infinity, and $$ \sum_{n=1}^{\infty}\frac{\operatorname{Var}(X_n)}{b_n^2} =\sum_{n=1}^{\infty}\frac{1}{n^{3/2}}<\infty. $$ The theorem shows that $n^{-3/4}\sum_{k=1}^{n}X_k\xrightarrow{a.s.}0$. In particular, $$ \frac{1}{n}\sum_{k=1}^{n}X_k =n^{-1/4}\left(\frac{1}{n^{3/4}}\sum_{k=1}^{n}X_k\right)\xrightarrow{a.s.}0, $$ because $n^{-1/4}\to0$ and the parenthesized sequence converges almost surely, hence is almost surely bounded eventually. This choice of weights gives a direct almost-sure bound at a scale different from the ordinary average.

What the Lemma Does—and Does Not—Say

Kronecker’s lemma is useful because its hypothesis concerns a convergent weighted series, while its conclusion concerns partial sums of the unweighted terms. The probabilistic theorem above turns a variance-summability condition into that series convergence. The three ingredients should be checked separately: the variables must meet the hypotheses of Kolmogorov’s criterion, the weights must be positive and nondecreasing, and the weights must diverge.

A common mistake is to think that convergence of $\sum_n a_n/b_n$ implies $\sum_n a_n$ converges. It does not. For instance, $a_n=1$ and $b_n=n$ give the divergent harmonic series $\sum_n a_n/b_n=\sum_n 1/n$, so this is not a counterexample to the lemma. A valid illustration is $a_n=(-1)^{n+1}$, $b_n=n$: the alternating harmonic series $\sum_n(-1)^{n+1}/n$ converges, while $\sum_n(-1)^{n+1}$ does not converge because its partial sums alternate between $1$ and $0$. Kronecker’s lemma instead asserts that the latter partial sums divided by $n$ tend to zero.

Worked Example: Conditional Series Convergence Is Enough

Let $a_n=(-1)^{n+1}$ and $b_n=n$. The alternating harmonic series converges: its terms decrease in magnitude to zero, and its even partial sums increase while its odd partial sums decrease to the same limit. Thus the hypothesis of Kronecker’s lemma holds, even though the series is not absolutely convergent because $\sum_n |a_n|/b_n=\sum_n1/n$ diverges.

The partial sums $\sum_{k=1}^{n}(-1)^{k+1}$ equal $1$ when $n$ is odd and $0$ when $n$ is even. Therefore their absolute values are at most $1$, and division by $n$ gives a quantity between $0$ and $1/n$. Hence $n^{-1}\sum_{k=1}^{n}a_k\to0$, as the lemma guarantees.

Key takeaway: If $\sum_n a_n/b_n$ converges and the positive weights $b_n$ do not decrease and tend to infinity, then $b_n^{-1}\sum_{k=1}^{n}a_k\to0$. For independent centered random variables, a finite sum of $\operatorname{Var}(X_n)/b_n^2$ is one standard way to obtain the required series convergence almost surely.

Check Your Understanding

Use the hypotheses carefully: both the series condition and the behavior of the weights matter.

Which conditions on $b_n$ are required in Kronecker’s lemma, and where are they used in its proof?
If $\sum_n a_n/b_n$ converges conditionally, can Kronecker’s lemma still be applied? Explain.
Suppose independent, mean-zero $X_n$ have finite variances. What summability condition gives $n^{-1}\sum_{k=1}^{n}X_k\xrightarrow{a.s.}0$ by this argument?
For IID variables with finite mean and finite variance, how do you use centering to obtain convergence of the sample averages to their mean?
Why does convergence of $\sum_n a_n/b_n$ not imply convergence of $\sum_n a_n$?