Kolmogorov's Strong-Law Criterion for Independent Sequences

Why a Weighted Variance Condition Helps

A strong law asks for more than convergence in probability: it asks whether the averages converge to their target with probability one. For independent variables, a useful way to establish this is to examine each variable after dividing it by its index. If the resulting variances have a finite sum, independence gives control of the tails of the corresponding partial sums.

The criterion below needs neither identical distributions nor a common underlying model. It is a sufficient condition: when the weighted variance sum is finite, the averages of the centered variables converge almost surely to zero. The proof combines Kolmogorov’s maximal inequality with a direct summation identity.

Definition: A sequence \(X_1,X_2,\ldots\) satisfies Kolmogorov’s strong-law criterion if the variables are independent, \(E[X_n]=0\), \(0\leq \operatorname{Var}(X_n)<\infty\) for every \(n\), and

\sum_{n=1}^{\infty}\frac{\operatorname{Var}(X_n)}{n^2}<\infty.

The factor \(n^{-2}\) is not an arbitrary decoration. It is the variance scaling obtained by considering \(X_n/n\). The proof first shows that the series of these scaled variables converges almost surely, then uses an algebraic identity to recover the desired averages.

The Strong-Law Criterion

Theorem: Suppose \(X_1,X_2,\ldots\) are independent, \(E[X_n]=0\), and each \(X_n\) has finite variance. If

\sum_{n=1}^{\infty}\frac{\operatorname{Var}(X_n)}{n^2}<\infty,

then

\frac{1}{n}\sum_{k=1}^{n}X_k\xrightarrow{a.s.}0.

We use two standard results from the earlier material. Kolmogorov’s maximal inequality says that if \(W_r,\ldots,W_s\) are independent, mean-zero, finite-variance variables, then for every \(\lambda>0\),

P\!\left(\max_{r\leq m\leq s}\left|\sum_{k=r}^{m}W_k\right|\geq\lambda\right) \leq \frac{1}{\lambda^2}\sum_{k=r}^{s}\operatorname{Var}(W_k).

We also use the first Borel–Cantelli lemma: if a sequence of events has summable probabilities, then with probability one only finitely many of those events occur.

Proof. Define \(Y_n=X_n/n\). The \(Y_n\) are independent and mean zero, and

\sum_{n=1}^{\infty}\operatorname{Var}(Y_n) = \sum_{n=1}^{\infty}\frac{\operatorname{Var}(X_n)}{n^2} <\infty.

Write \(v_n=\operatorname{Var}(Y_n)\). Since the tail sums of the convergent nonnegative series \(\sum_n v_n\) tend to zero, for every integer \(j\geq1\) we can choose an increasing sequence of indices \(r_j\) such that

\sum_{k=r_j}^{\infty}v_k\leq 2^{-4j}.

For a finite endpoint \(M\geq r_j\), Kolmogorov’s maximal inequality, applied to \(Y_{r_j},\ldots,Y_M\) with threshold \(2^{-j}\), gives

P\!\left( \max_{r_j\leq m\leq M} \left|\sum_{k=r_j}^{m}Y_k\right|>2^{-j} \right) \leq 2^{2j}\sum_{k=r_j}^{M}v_k \leq 2^{-2j}.

As \(M\) increases, the events on the left increase to the event that the same inequality holds for at least one finite \(m\geq r_j\). Continuity of probability for increasing events therefore yields

P\!\left( \sup_{m\geq r_j} \left|\sum_{k=r_j}^{m}Y_k\right|>2^{-j} \right)\leq 2^{-2j}.

The bounds are summable in \(j\), so the first Borel–Cantelli lemma implies that, with probability one, only finitely many of these events occur. Consequently, almost surely, for all sufficiently large \(j\),

\left|\sum_{k=r_j}^{m}Y_k\right|\leq 2^{-j} \quad\text{for every }m\geq r_j.

On this probability-one event, the series \(\sum_{k=1}^{\infty}Y_k\) is Cauchy. Indeed, if \(q\geq p\geq r_j+1\), then

\left|\sum_{k=p}^{q}Y_k\right| = \left|\sum_{k=r_j}^{q}Y_k-\sum_{k=r_j}^{p-1}Y_k\right| \leq 2\cdot 2^{-j}.

The right-hand side tends to zero as \(j\) increases. Thus the series converges almost surely. Let \(T_n=\sum_{k=1}^{n}Y_k\); on this same event, \(T_n\) has a finite limit \(T\).

For each \(n\), a finite-sum identity gives

\sum_{k=1}^{n}kY_k = nT_n-\sum_{k=1}^{n-1}T_k.

To check it, the coefficient of \(Y_r\) on the right is \(n-(n-r)=r\), as on the left. Dividing by \(n\), and using \(kY_k=X_k\), gives

\frac{1}{n}\sum_{k=1}^{n}X_k = T_n-\frac{1}{n}\sum_{k=1}^{n-1}T_k.

Because \(T_k\to T\), its averages also converge to \(T\). Explicitly, for any \(\varepsilon>0\), choose \(K\) so that \(|T_k-T|<\varepsilon\) for \(k\geq K\). The contribution of the finitely many terms with \(k<K\), divided by \(n\), tends to zero, while the remaining terms differ from \(T\) by at most \(\varepsilon\) each. Hence \(n^{-1}\sum_{k=1}^{n-1}T_k\to T\). Also \(T_n\to T\), so their difference tends to zero. This proves the asserted almost-sure convergence. \(\square\)

Worked Applications

Worked Example: Independent Signs

Let \(X_1,X_2,\ldots\) be independent, with \(P(X_n=1)=P(X_n=-1)=1/2\) for every \(n\). Check the hypotheses and find the almost-sure limit of their averages.

Symmetry gives \(E[X_n]=0\), and \(X_n^2=1\) always, so \(\operatorname{Var}(X_n)=1\). Therefore

\sum_{n=1}^{\infty}\frac{\operatorname{Var}(X_n)}{n^2} = \sum_{n=1}^{\infty}\frac{1}{n^2} = \frac{\pi^2}{6}<\infty.

All conditions of the theorem hold. It follows that

\frac{X_1+\cdots+X_n}{n}\xrightarrow{a.s.}0.

The variables are identically distributed in this example, but identical distribution is not needed for the criterion.

Worked Example: Rare Bernoulli Events With Changing Probabilities

Let \(B_n\) be independent Bernoulli variables with \(P(B_n=1)=p_n=1/(n+1)\), and set \(X_n=B_n-p_n\). Apply the criterion, then determine the limit of the uncentered averages \(n^{-1}\sum_{k=1}^n B_k\).

Since \(E[B_n]=p_n\), we have \(E[X_n]=0\). A Bernoulli variable with success probability \(p_n\) has variance \(p_n(1-p_n)\), so

\operatorname{Var}(X_n)=p_n(1-p_n) \leq p_n=\frac{1}{n+1}.

It follows that

\sum_{n=1}^{\infty}\frac{\operatorname{Var}(X_n)}{n^2} \leq \sum_{n=1}^{\infty}\frac{1}{n^2(n+1)} \leq \sum_{n=1}^{\infty}\frac{1}{n^3}<\infty.

Independence and finite variances also hold, so \(n^{-1}\sum_{k=1}^n(B_k-p_k)\to0\) almost surely. The average of the probabilities tends to zero: using \(\sum_{k=1}^{n}1/(k+1)\leq \log(n+1)\),

0\leq \frac{1}{n}\sum_{k=1}^{n}p_k \leq \frac{\log(n+1)}{n}\longrightarrow 0.

Since \(B_k=(B_k-p_k)+p_k\), adding these two limits gives \(n^{-1}\sum_{k=1}^{n}B_k\to0\) almost surely. This example illustrates how the criterion can handle independent variables whose distributions change with the index.

Worked Example: When the Criterion Does Not Apply

Let \(X_n\) be independent with \(X_n\sim N(0,n)\). Does the criterion establish that their averages tend to zero? In fact, do the averages tend to zero even in probability?

Here \(E[X_n]=0\) and \(\operatorname{Var}(X_n)=n\), so the weighted variance sum is

\sum_{n=1}^{\infty}\frac{\operatorname{Var}(X_n)}{n^2} = \sum_{n=1}^{\infty}\frac{1}{n} =\infty.

Thus the criterion gives no conclusion. To check whether the averages nevertheless converge to zero in probability, use the fact that independent normal variables have a normal sum. We obtain

\frac{1}{n}\sum_{k=1}^{n}X_k \sim N\!\left(0,\frac{1}{n^2}\sum_{k=1}^{n}k\right) = N\!\left(0,\frac{n+1}{2n}\right).

The variance tends to \(1/2\), not zero. In particular,

P\!\left(\left|\frac{1}{n}\sum_{k=1}^{n}X_k\right|>\frac12\right) \longrightarrow 2\left[1-\Phi\!\left(\frac{1}{\sqrt{2}}\right)\right]>0,

where \(\Phi\) is the standard normal cumulative distribution function. So the averages do not converge to zero in probability, and therefore cannot converge to zero almost surely. This example shows a case where the condition fails and the conclusion also fails; failure of the condition alone, however, is not a general proof that a strong law fails.

Using the Criterion Carefully

The criterion is useful because it reduces an almost-sure convergence question to a numerical series. For independent centered variables, calculate each variance and test whether its contribution after division by \(n^2\) is summable. The variables need not share a distribution, and the variances need not be equal.

A common mistake is to argue that each \(X_n/n\) becomes small, and therefore the averages must converge almost surely. Small individual terms alone do not guarantee convergence of a series or of averages. The proof needs control of whole blocks of partial sums: the maximal inequality supplies that control, and Borel–Cantelli turns summable probability bounds into an almost-sure statement.

Another important distinction is between a sufficient condition and a necessary one. If \(\sum_n\operatorname{Var}(X_n)/n^2\) diverges, this theorem is silent. Another argument might still establish a strong law; alternatively, the conclusion might fail, as in the normal example. The condition must be checked before invoking this particular result, and its failure must not be mistaken for a conclusion in either direction.

Key takeaway: For independent, mean-zero variables with finite variances, summability of \(\operatorname{Var}(X_n)/n^2\) implies that the averages converge to zero almost surely. The proof works by establishing almost-sure convergence of \(\sum_n X_n/n\), then relating its partial sums to the averages through a finite-sum identity.

Check Your Understanding

Try these questions using the hypotheses and proof above.

Which assumptions on the variables are needed, in addition to the finite weighted variance sum?
For independent mean-zero variables with \(\operatorname{Var}(X_n)=1/n\), does the variance condition hold? Show the series you need to check.
Why does the proof apply Kolmogorov’s maximal inequality to \(Y_n=X_n/n\), rather than directly to \(X_n\)?
In the Bernoulli example, why is it valid to add the limit of the centered averages to the limit of the average probabilities?
If the weighted variance series diverges, what can and cannot be concluded from this criterion?

Kolmogorov's Strong-Law Criterion for Independent Sequences

What You'll Learn