Wald’s Second Identity for Centered Increments

From Increment Variance to Stopped-Sum Variance

Suppose a process accumulates independent, centered increments \(X_1,X_2,\ldots\), each with variance \(\sigma^2\). At a fixed time \(n\), the sum \(S_n=X_1+\cdots+X_n\) has second moment \(n\sigma^2\): the individual variances add, while the cross terms vanish. If the number of increments is random, those cross terms require more care. The key is not merely that the increments are independent; the time at which we stop must not use information from the future.

Work on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). Let \(X_1,X_2,\ldots\) be iid real-valued random variables with \(\mathbb{E}[X_k]=0\) and \(\operatorname{Var}(X_k)=\sigma^2<\infty\). Use the filtration \(\mathcal{F}_0=\{\varnothing,\Omega\}\) and \(\mathcal{F}_n=\sigma(X_1,\ldots,X_n)\) for \(n\ge1\). Thus the increments observed through time \(n\) are exactly the information in \(\mathcal{F}_n\). Define \(S_0=0\) and \(S_n=\sum_{k=1}^n X_k\), a discrete-time process with state space \(\mathbb{R}\).

For the identity below, assume \(\tau\) is a stopping time and \(\mathbb{E}[\tau]<\infty\). This assumption implies \(\tau<\infty\) almost surely, so \(S_\tau\) is defined almost surely. The finite expectation condition also ensures that the stopped sum has a finite second moment. We will establish that fact as part of the proof rather than assume it.

Proof. Since \(\{\tau\ge k\}\in\mathcal{F}_{k-1}\) and \(X_k\) is independent of \(\mathcal{F}_{k-1}\), we have \(\mathbb{E}[Y_k^2]=\mathbb{E}[\mathbf{1}_{\{\tau\ge k\}}X_k^2]=\mathbb{P}(\tau\ge k)\mathbb{E}[X_k^2]=\sigma^2\mathbb{P}(\tau\ge k)\). Now take \(i<j\). The random variable \(X_i\mathbf{1}_{\{\tau\ge i\}}\mathbf{1}_{\{\tau\ge j\}}\) is \(\mathcal{F}_{j-1}\)-measurable: \(X_i\) is observed by time \(j-1\), and both indicators are determined by then. Conditioning on \(\mathcal{F}_{j-1}\), and using \(\mathbb{E}[X_j\mid\mathcal{F}_{j-1}]=0\), gives \(\mathbb{E}[Y_iY_j]=\mathbb{E}[X_i\mathbf{1}_{\{\tau\ge i\}}\mathbf{1}_{\{\tau\ge j\}}\mathbb{E}[X_j\mid\mathcal{F}_{j-1}]]=0\). Interchanging \(i\) and \(j\) covers the other ordering. \(\square\)

The stopping-time property is used precisely in making \(\{\tau\ge j\}\) known before \(X_j\) is observed. This makes the decision to include the \(j\)th increment independent of its centered value, conditional on the past.

Wald’s Second Identity

Proof. For every integer \(m\ge1\), the stopped sum through the first \(m\) possible increments is \(S_{\tau\wedge m}=\sum_{k=1}^mY_k\), where \(a\wedge b=\min(a,b)\). By the proposition, expanding the square gives \(\mathbb{E}[S_{\tau\wedge m}^2]=\sum_{k=1}^m\mathbb{E}[Y_k^2]=\sigma^2\sum_{k=1}^m\mathbb{P}(\tau\ge k)\). For a nonnegative integer-valued \(\tau\), the pointwise identity \(\tau=\sum_{k\ge1}\mathbf{1}_{\{\tau\ge k\}}\) implies \(\mathbb{E}[\tau]=\sum_{k\ge1}\mathbb{P}(\tau\ge k)\), by taking expectations of the increasing partial sums. Hence, for \(r>m\), the same orthogonality calculation yields \(\mathbb{E}[(S_{\tau\wedge r}-S_{\tau\wedge m})^2]=\sigma^2\sum_{k=m+1}^{r}\mathbb{P}(\tau\ge k)\). Because \(\mathbb{E}[\tau]<\infty\), the tail of this convergent series tends to zero as \(m\to\infty\). Thus \(S_{\tau\wedge m}\) is Cauchy in \(L^2\), so it has an \(L^2\) limit. Also, \(\mathbb{P}(\tau=\infty)=0\), since a positive probability of \(\tau=\infty\) would force \(\mathbb{E}[\tau]=\infty\). Therefore \(S_{\tau\wedge m}\to S_\tau\) almost surely, and its \(L^2\) limit must equal \(S_\tau\) almost surely. Taking \(m\to\infty\) in the displayed second-moment formula gives \(\mathbb{E}[S_\tau^2]=\sigma^2\mathbb{E}[\tau]\). Finally, each \(Y_k\) has mean zero by independence of \(X_k\) from \(\mathcal{F}_{k-1}\). The finite sums \(S_{\tau\wedge m}\) therefore have mean zero; their \(L^2\), hence \(L^1\), convergence gives \(\mathbb{E}[S_\tau]=0\). It follows that \(\operatorname{Var}(S_\tau)=\mathbb{E}[S_\tau^2]\), proving the variance statement as well. \(\square\)

The identity says that the expected squared fluctuation is the variance per increment multiplied by the expected number of increments. It is not a claim that \(S_\tau\) has a particular distribution. Nor does the equation alone license optional stopping at an arbitrary random time: its proof depends on both the integrability condition and the fact that \(\tau\) is a stopping time.

Why the Stopping-Time Condition Matters

A stopping time decides whether to continue using information available at that time. An arbitrary random time may look ahead. In that case, the indicator for including an increment can depend on that increment or on later ones, and the conditional-centering argument in the orthogonality proof no longer applies.

Common mistake. Consider iid Rademacher increments \(X_1,X_2\), and define the random time \(\tau=1\) if \(X_1X_2=1\), and \(\tau=2\) if \(X_1X_2=-1\). This is not a stopping time: deciding at time \(1\) whether \(\tau=1\) requires knowing \(X_2\). Each of the four pairs \((X_1,X_2)\in\{(1,1),(1,-1),(-1,1),(-1,-1)\}\) has probability \(\tfrac14\). For the two pairs with product \(1\), \(S_\tau=X_1\) and \(S_\tau^2=1\). For the two pairs with product \(-1\), \(\tau=2\) and \(S_\tau=X_1+X_2=0\), so \(S_\tau^2=0\). Consequently \(\mathbb{E}[S_\tau^2]=\tfrac12\), whereas \(\sigma^2\mathbb{E}[\tau]=1\cdot(\tfrac12(1)+\tfrac12(2))=\tfrac32\). The identity fails here because \(\tau\) uses future information. The incorrect calculation would apply the stopping-time identity merely because the increments are iid and \(\mathbb{E}[\tau]\) is finite; both conditions hold, but the stopping-time condition does not.

The other hypothesis also has a role. The proof uses \(\mathbb{E}[\tau]<\infty\) to make the sum of the stopped increments converge in \(L^2\) and to guarantee a finite right-hand side. Without it, this argument does not establish the identity. In practice, verify the filtration, stopping-time property, centering, finite increment variance, and finite expected stopping time before applying the formula.