From Orthogonal Increments to Mean-Square Convergence
For a martingale, future increments have conditional mean zero given the past. This makes increments from different times orthogonal in \(L^2\): their expected product is zero. Consequently, the squared distance between two values of the martingale is exactly the sum of the intervening increment variances. If the martingale's second moments stay bounded, those variances cannot keep accumulating, and the martingale must be Cauchy in \(L^2\).
Work on a probability space \((\Omega,\mathcal{F},\mathbb{P})\) with an increasing filtration \((\mathcal{F}_n)_{n\ge0}\). Let \((M_n)_{n\ge0}\) be a real-valued process in discrete time, with state space \(S=\mathbb{R}\). It is a martingale if each \(M_n\) is \(\mathcal{F}_n\)-measurable and integrable and \(\mathbb{E}[M_{n+1}\mid\mathcal{F}_n]=M_n\) almost surely. It is square-integrable if \(\mathbb{E}[M_n^2]<\infty\) for every \(n\). Convergence in \(L^2\) to \(M_\infty\) means \(\mathbb{E}[(M_n-M_\infty)^2]\to0\).
Proof. Since \(\Delta_r\) is \(\mathcal{F}_{s-1}\)-measurable and the martingale property gives \(\mathbb{E}[\Delta_s\mid\mathcal{F}_{s-1}]=0\), conditional expectation yields \(\mathbb{E}[\Delta_r\Delta_s]=\mathbb{E}[\Delta_r\mathbb{E}[\Delta_s\mid\mathcal{F}_{s-1}]]=0\). The products are integrable by square-integrability and Cauchy–Schwarz. Now \(M_m-M_n=\sum_{k=n+1}^m\Delta_k\). Expanding its square, all cross terms have expectation zero by the result just proved, giving the sum of increment second moments. In particular, taking \(n=0\) shows that \(M_m=M_0+\sum_{k=1}^m\Delta_k\), with \(M_0\) orthogonal to every \(\Delta_k\), because \(M_0\) is \(\mathcal{F}_{k-1}\)-measurable. Thus \(\mathbb{E}[M_m^2]=\mathbb{E}[M_0^2]+\sum_{k=1}^m\mathbb{E}[\Delta_k^2]\). Subtracting the corresponding identity for \(n\) proves the last equality. \(\square\)
The L2 Convergence Theorem
Proof. Write \(a_n=\mathbb{E}[M_n^2]\). The orthogonality lemma gives \(a_{n+1}-a_n=\mathbb{E}[\Delta_{n+1}^2]\ge0\). Thus \((a_n)\) is nondecreasing. By hypothesis it is bounded above, so it has a finite limit \(a\). For \(m\ge n\), the lemma gives \(\mathbb{E}[(M_m-M_n)^2]=a_m-a_n\le a-a_n\). As \(n\to\infty\), \(a-a_n\to0\), uniformly over \(m\ge n\). Therefore \((M_n)\) is Cauchy in \(L^2\).
For completeness, a Cauchy sequence in \(L^2\) has an \(L^2\) limit. To see this directly here, choose a subsequence \((M_{n_j})_{j\ge1}\) such that \(\|M_{n_{j+1}}-M_{n_j}\|_2\le2^{-j}\), where \(\|X\|_2=(\mathbb{E}[X^2])^{1/2}\). The inequality \(\mathbb{E}|X|\le\|X\|_2\) implies \(\sum_j\mathbb{E}|M_{n_{j+1}}-M_{n_j}|\le\sum_j2^{-j}<\infty\). By Tonelli's theorem, the sum of the absolute differences is finite almost surely. Hence the subsequence converges almost surely to a random variable \(M_\infty\).
For \(l>j\), Minkowski's inequality gives \(\|M_{n_l}-M_{n_j}\|_2\le\sum_{r=j}^{l-1}2^{-r}\le2^{1-j}\). Fatou's lemma, applied as \(l\to\infty\), then gives \(\mathbb{E}[(M_\infty-M_{n_j})^2]\le2^{2-2j}\). Thus the subsequence converges to \(M_\infty\) in \(L^2\). Finally, the original sequence is Cauchy: given \(\epsilon>0\), choose \(N\) so that \(\|M_m-M_n\|_2<\epsilon/2\) for \(m,n\ge N\), and choose \(j\) with \(n_j\ge N\) and \(\|M_{n_j}-M_\infty\|_2<\epsilon/2\). For \(n\ge N\), the triangle inequality gives \(\|M_n-M_\infty\|_2<\epsilon\). This proves the claim. \(\square\)
The estimate in the proof is more informative than convergence alone: \(\mathbb{E}[(M_m-M_n)^2]=a_m-a_n\). A bound on all the \(a_n\) forces the tail of this increasing sequence of second moments to be small, which controls every later difference at once. No assumption that the martingale is bounded on each sample path is needed.
Worked Examples
Worked Example: A Martingale That Becomes Constant
Let \(\Omega=\{0,1\}\), with each outcome having probability \(\tfrac12\), and let \(U(\omega)=\omega\). Use the filtration \(\mathcal{F}_0=\{\varnothing,\Omega\}\) and \(\mathcal{F}_n=\sigma(U)\) for \(n\ge1\). Define \(M_0=\tfrac12\) and \(M_n=U\) for \(n\ge1\). The process has state space \(S=\mathbb{R}\) and discrete time. Verify the martingale property and its \(L^2\) limit.
At time \(0\), \(\mathbb{E}[M_1\mid\mathcal{F}_0]=\mathbb{E}[U]=\tfrac12=M_0\). For \(n\ge1\), \(M_{n+1}=M_n=U\), so \(\mathbb{E}[M_{n+1}\mid\mathcal{F}_n]=M_n\). The second moment at time zero is \(\mathbb{E}[M_0^2]=(\tfrac12)^2=\tfrac14\). At every \(n\ge1\), \(U^2=U\), so \(\mathbb{E}[M_n^2]=\mathbb{E}[U]=\tfrac12\). These moments are bounded.
The only nonzero increment is \(\Delta_1=U-\tfrac12\). Its square is \(\tfrac14\) at both outcomes, so \(\mathbb{E}[\Delta_1^2]=\tfrac14\). The variance decomposition checks: \(\mathbb{E}[M_1^2]=\mathbb{E}[M_0^2]+\mathbb{E}[\Delta_1^2]=\tfrac14+\tfrac14=\tfrac12\). For all \(n\ge1\), \(M_n=U\), so the \(L^2\) limit is \(M_\infty=U\), with \(\mathbb{E}[(M_n-M_\infty)^2]=0\). Substitution confirms the limit directly.
Worked Example: A Weighted-Sign Martingale and Its Error
On the product probability space \(\Omega=\{-1,1\}^{\mathbb{N}}\) with independent, equally likely coordinates \((\varepsilon_k)_{k\ge1}\), let \(\mathcal{F}_n=\sigma(\varepsilon_1,\ldots,\varepsilon_n)\) and \(M_n=\sum_{k=1}^n2^{-k}\varepsilon_k\), with \(M_0=0\). This is a discrete-time process with state space \(S=\mathbb{R}\). Find a bound on its second moments and compute its mean-square error after \(n=2\).
Its increment is \(\Delta_n=2^{-n}\varepsilon_n\). Independence of \(\varepsilon_n\) from \(\mathcal{F}_{n-1}\) and \(\mathbb{E}[\varepsilon_n]=0\) give \(\mathbb{E}[\Delta_n\mid\mathcal{F}_{n-1}]=0\), so the process is a martingale. Each sign has square \(1\), and distinct signs have expected product zero. Therefore \(\mathbb{E}[M_n^2]=\sum_{k=1}^n4^{-k}=\frac{1-4^{-n}}{3}<\frac13\). In particular, at \(n=2\) this is \(\tfrac14+\tfrac1{16}=\tfrac5{16}\), verifying the formula \((1-\tfrac1{16})/3=\tfrac5{16}\).
The theorem gives an \(L^2\) limit. Its squared distance from \(M_n\) is the remaining sum of increment variances: \(\mathbb{E}[(M_\infty-M_n)^2]=\sum_{k=n+1}^\infty4^{-k}=\frac{4^{-n}}{3}\). At \(n=2\), the error is \(\frac{1}{16\cdot3}=\tfrac1{48}\). To check, the first omitted variance is \(4^{-3}=\tfrac1{64}\), and the tail is \(\tfrac1{64}(1+\tfrac14+\tfrac1{16}+\cdots)=\tfrac1{64}\cdot\tfrac43=\tfrac1{48}\). The tail tends to zero, as required for \(L^2\) convergence.
Worked Example: What Fails Without the Uniform Bound
On a product space with independent, equally likely signs \((\varepsilon_k)_{k\ge1}\), let \(\mathcal{F}_n=\sigma(\varepsilon_1,\ldots,\varepsilon_n)\) and \(M_n=\sum_{k=1}^n\varepsilon_k\), with \(M_0=0\). This is a discrete-time process with state space \(S=\mathbb{R}\). Check the martingale property and determine why the convergence theorem does not apply.
Independence and \(\mathbb{E}[\varepsilon_{n+1}]=0\) give \(\mathbb{E}[M_{n+1}\mid\mathcal{F}_n]=M_n\). As each sign has second moment \(1\), and distinct signs have expected product zero, \(\mathbb{E}[M_n^2]=n\). Thus the second moments are not uniformly bounded. For \(m\ge n\), the increment identity gives \(\mathbb{E}[(M_m-M_n)^2]=m-n\). For example, at \(m=5\) and \(n=3\), the squared distance is \(5-3=2\); more generally, choosing \(m=n+1\) gives squared distance \(1\) for every \(n\). The process is not Cauchy in \(L^2\), so it cannot converge in \(L^2\). This verifies that the uniform second-moment hypothesis cannot simply be omitted.
Reading the Conclusion Correctly
Convergence in \(L^2\) is a statement about expected squared error; it is not a claim that the values converge for every outcome. It does imply convergence in probability: for every \(\epsilon>0\), Markov's inequality gives \(\mathbb{P}(|M_n-M_\infty|>\epsilon)\le\mathbb{E}[(M_n-M_\infty)^2]/\epsilon^2\to0\). The earlier result on almost-sure convergence of \(L^2\)-bounded martingales gives almost-sure convergence under the same boundedness hypothesis. If that almost-sure limit and the \(L^2\) limit are both considered, they agree almost surely: convergence in \(L^2\) implies convergence in probability, and limits in probability are unique up to almost-sure equality.
Common mistake. Orthogonality of the increments does not mean the second moment of \(M_n\) is just the sum of increment second moments; the initial value contributes too. For the constant martingale \(M_n=3\) on a one-point probability space with the trivial filtration, every increment is zero. The incorrect calculation \(\mathbb{E}[M_n^2]=\sum_{k=1}^n\mathbb{E}[\Delta_k^2]=0\) contradicts the actual value \(9\). The correct decomposition is \(\mathbb{E}[M_n^2]=\mathbb{E}[M_0^2]+\sum_{k=1}^n\mathbb{E}[\Delta_k^2]=9+0=9\). For differences \(M_m-M_n\), the initial value cancels, which is why the exact Cauchy estimate involves only the intervening increments.
Check Your Understanding
Use the increment identity to justify each conclusion, and distinguish a second-moment bound from a pathwise bound.
- For a square-integrable martingale, why are \(\Delta_r\) and \(\Delta_s\) orthogonal when \(r<s\)? State the conditioning sigma-field.
- If \(\mathbb{E}[M_n^2]\) increases to a finite limit \(a\), give an upper bound for \(\mathbb{E}[(M_m-M_n)^2]\) that holds for every \(m\ge n\).
- For the weighted-sign martingale, compute \(\mathbb{E}[(M_\infty-M_3)^2]\) from the remaining increment variances.
- Why does a bounded sequence of first moments alone not supply the second-moment Cauchy estimate in the theorem?
- For the simple random walk example, evaluate \(\mathbb{E}[(M_8-M_5)^2]\) and explain which hypothesis fails.