Why Wald’s Conditions Matter
Wald’s identities turn a random sum into a product involving the expected number of terms. That simplification is powerful, but it depends on how the sample size is chosen and on the structure and integrability of the increments. A random index that looks like a stopping time may use information from the future; increments with the right marginal distribution may still be dependent; and a stopping time that is finite almost surely may have infinite expectation. In each case, the formal product can be wrong or fail to be a meaningful finite quantity.
We work on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). Let \((X_k)_{k\ge1}\) be increments and let \(\mathcal{F}_n\) be the information available after observing the first \(n\) increments, with \(\mathcal{F}_0\) containing the initial information. Write \(S_n=\sum_{k=1}^n X_k\), with \(S_0=0\). A nonnegative integer-valued random variable \(\tau\) is a stopping time if \(\{\tau\le n\}\in\mathcal{F}_n\) for every \(n\). Equivalently, whether sampling has ended by time \(n\) can be decided from information available by time \(n\).
The key point in both formulas is that \(\{\tau\ge k\}=\{\tau>k-1\}\) is known by time \(k-1\). The decision to include \(X_k\) is made before \(X_k\) is observed. The next lemma isolates this requirement for the first identity.
Proof. Independence gives \(\mathbb{E}[X_k\mathbf{1}_{A_k}]=\mathbb{E}[X_k]\mathbb{P}(A_k)=\mu\mathbb{P}(A_k)\). Also, \(\mathbb{E}\sum_{k\ge1}|X_k|\mathbf{1}_{A_k}=\mathbb{E}|X_1|\sum_{k\ge1}\mathbb{P}(A_k)<\infty\). Thus the series is absolutely integrable in expectation, so its expectation is the sum of the expectations of its terms. Substitution of the termwise formula proves the claim. \(\square\)
Proof of the theorem. For the first identity, set \(A_k=\{\tau\ge k\}\). The stopping-time property implies \(A_k\in\mathcal{F}_{k-1}\), and \(\sum_{k\ge1}\mathbb{P}(\tau\ge k)=\mathbb{E}\tau<\infty\), by the tail-sum formula for a nonnegative integer-valued random variable. The lemma therefore gives \(\mathbb{E}S_\tau=\mu\sum_{k\ge1}\mathbb{P}(\tau\ge k)=\mu\mathbb{E}\tau\).
For the second identity, define \(D_k=X_k\mathbf{1}_{\{\tau\ge k\}}\). For \(k<\ell\), \(D_k\mathbf{1}_{\{\tau\ge\ell\}}\) is \(\mathcal{F}_{\ell-1}\)-measurable, whereas \(X_\ell\) is independent of \(\mathcal{F}_{\ell-1}\) and has mean zero. Hence \(\mathbb{E}[D_kD_\ell]=0\). Further, \(\mathbb{E}[D_k^2]=\sigma^2\mathbb{P}(\tau\ge k)\), since \(\{\tau\ge k\}\in\mathcal{F}_{k-1}\) and \(X_k\) is independent of that event. For each \(n\), expanding the square and using the zero cross terms gives \(\mathbb{E}[(\sum_{k=1}^nD_k)^2]=\sigma^2\sum_{k=1}^n\mathbb{P}(\tau\ge k)\). The same calculation for a tail from \(n+1\) to \(m\) bounds its second moment by \(\sigma^2\sum_{k>n}\mathbb{P}(\tau\ge k)\), which tends to zero because \(\mathbb{E}\tau<\infty\). Thus the partial sums converge in \(L^2\) to \(S_\tau\), and their second moments converge to \(\mathbb{E}[S_\tau^2]\). Taking \(n\to\infty\) yields \(\mathbb{E}[S_\tau^2]=\sigma^2\mathbb{E}\tau\). \(\square\)
Failure 1: The Rule Uses Future Information
Worked Example: A Bounded Index That Is Not a Stopping Time
Let \(X_1,X_2,\ldots\) be independent fair increments, each equal to \(1\) or \(-1\) with probability \(\tfrac12\), and use their natural filtration \(\mathcal{F}_n=\sigma(X_1,\ldots,X_n)\). Define \(\tau=1\) if \(X_2=1\), and \(\tau=2\) if \(X_2=-1\). This index is bounded and finite, but it is not a stopping time: \(\{\tau\le1\}=\{X_2=1\}\notin\mathcal{F}_1\).
Its expected value is \(\mathbb{E}\tau=1\cdot\mathbb{P}(X_2=1)+2\cdot\mathbb{P}(X_2=-1)=1\cdot\tfrac12+2\cdot\tfrac12=\tfrac32\). The increment mean is \(\mu=0\), so a blind use of Wald’s first identity would predict \(\mathbb{E}S_\tau=0\).
But \(S_\tau=X_1+X_2\mathbf{1}_{\{X_2=-1\}}\). Independence gives \(\mathbb{E}X_1=0\), and direct evaluation of the second term gives \(\mathbb{E}[X_2\mathbf{1}_{\{X_2=-1\}}]=0\cdot\mathbb{P}(X_2=1)+(-1)\cdot\mathbb{P}(X_2=-1)=0\cdot\tfrac12+(-1)\cdot\tfrac12=-\tfrac12\). Therefore \(\mathbb{E}S_\tau=-\tfrac12\), not \(0\).
Check by listing outcomes. The pair \((X_1,X_2)\) has four equally likely outcomes. If \(X_2=1\), then \(S_\tau=X_1\), whose two possible values are \(1\) and \(-1\). If \(X_2=-1\), then \(S_\tau=X_1-1\), whose two possible values are \(0\) and \(-2\). The average is \((1-1+0-2)/4=-\tfrac12\). The discrepancy occurs because whether \(X_2\) is included depends on \(X_2\) itself, rather than information available before that increment.
Failure 2: The Increments Are Dependent
Worked Example: Correct Stopping Rule, Dependent Increments
Let \(Y\) be fair on \(\{-1,1\}\), set \(X_1=Y\), \(X_2=-Y\), and let \(X_3,X_4,\ldots\) be independent fair increments, independent of \(Y\). Every increment has mean zero and the same marginal distribution, but \(X_1\) and \(X_2\) are dependent. Use the natural filtration and set \(\tau=1\) if \(X_1=1\), and \(\tau=2\) if \(X_1=-1\). This is a stopping time, because the decision at time \(1\) uses only \(X_1\).
Since each value of \(Y\) has probability \(\tfrac12\), \(\mathbb{E}\tau=1\cdot\tfrac12+2\cdot\tfrac12=\tfrac32\). The marginal increment mean is \(\mu=0\), so the independent-increment formula would give \(0\). If \(Y=1\), the rule stops at \(1\) and \(S_\tau=X_1=1\). If \(Y=-1\), it stops at \(2\) and \(S_\tau=X_1+X_2=-1+1=0\). Consequently, \(\mathbb{E}S_\tau=1\cdot\tfrac12+0\cdot\tfrac12=\tfrac12\), which differs from \(0\).
Check: the dependence makes the second increment cancel the first exactly when the first increment is \(-1\); the sampling rule selects precisely that cancellation case. The marginal means alone do not determine the expectation of a randomly selected sum. Wald’s identities require the stated independence structure, not merely identically distributed increments.
Failure 3: Almost-Sure Finiteness Is Not Enough
Worked Example: A Simple Random Walk Return Time
Let \(X_k\) be independent fair increments on \(\{-1,1\}\), let \(S_n=\sum_{k=1}^nX_k\), and let \(\tau=\inf\{n\ge1:S_n=0\}\), the first return time to zero. This is a stopping time. The one-dimensional simple symmetric random walk returns to its starting point almost surely; this is the standard recurrence result for the simple symmetric random walk. At the return time, \(S_\tau=0\) almost surely.
Nevertheless, \(\mathbb{E}\tau=\infty\). To verify this without assuming a return-time formula, suppose instead that \(\mathbb{E}\tau<\infty\). The second Wald identity proved above applies: the increments are centered with variance \(1\), and \(\tau\) is a stopping time. It would give \(\mathbb{E}[S_\tau^2]=1\cdot\mathbb{E}\tau\). The left side is \(0\), because \(S_\tau=0\) almost surely. The right side is at least \(1\), because \(\tau\ge1\) almost surely. This contradiction proves \(\mathbb{E}\tau=\infty\).
Thus dropping the finite-mean condition from the second identity would suggest \(0=\infty\). For the first identity, the increment mean is zero, but the expression \(\mu\mathbb{E}\tau\) becomes \(0\cdot\infty\), which is undefined; it is not a valid prediction for \(\mathbb{E}S_\tau\). In this particular example \(S_\tau=0\), but that fact does not make the unqualified Wald formula valid.
What to Check Before Applying Wald
The examples separate three distinct issues. A random index can be bounded but fail to be a stopping time. A sampling rule can be a stopping time while the increments are dependent. And a stopping time can be finite almost surely while its expectation is infinite. These are not interchangeable objections: each breaks a different step in the indicator-sum argument or the integrability passage.
| Condition to verify | Why it matters | Failure illustrated here |
|---|---|---|
| \(\{\tau\ge k\}\in\mathcal{F}_{k-1}\) | The decision to include \(X_k\) must precede observing \(X_k\). | The rule based on \(X_2\) does not qualify at time \(1\). |
| \(X_k\) independent of \(\mathcal{F}_{k-1}\), with the required common distribution | This gives the termwise expectation and removes cross terms. | The pair \(X_1=Y,\ X_2=-Y\) is dependent. |
| \(\mathbb{E}\tau<\infty\) | It makes the tail-probability sum finite and justifies the limiting identities. | The simple random walk return time has infinite mean. |
| \(\mathbb{E}|X_1|<\infty\), and for the second identity \(\operatorname{Var}(X_1)<\infty\) | The expected sum and second-moment calculation must be defined and finite as required. | Without these assumptions, the corresponding expectations may not exist as finite quantities. |
Common mistake. In the first example, a tempting but incorrect calculation is to treat \(\mathbb{E}[X_2\mathbf{1}_{\{X_2=-1\}}]\) as though the product were \(1\) on the outcome \(X_2=1\). It is not: the indicator is zero there, so the product is zero. Evaluating the two outcomes gives \(0\cdot\mathbb{P}(X_2=1)+(-1)\cdot\mathbb{P}(X_2=-1)=0\cdot\tfrac12+(-1)\cdot\tfrac12=-\tfrac12\). A related error is to replace a random-index sum by a Wald product before checking the stopping-time and integrability conditions. First write the sum with indicators, then verify that each inclusion event is known before its increment is observed.
Check Your Understanding
For each question, identify the relevant hypothesis before computing.
- In the non-stopping-rule example, compute \(\mathbb{E}\tau\) and \(\mathbb{E}S_\tau\) directly. Which event prevents \(\tau\) from being a stopping time?
- In the dependent-increment example, what are the possible values of \(S_\tau\), and why does the stopping-time property alone not justify Wald’s identity?
- For a centered increment sequence with variance \(\sigma^2\), state the additional conditions needed to conclude \(\mathbb{E}[S_\tau^2]=\sigma^2\mathbb{E}\tau\).
- Why does “\(\tau<\infty\) almost surely” not guarantee that \(\mathbb{E}\tau<\infty\)?
- In the simple random walk return example, explain why the expression \(0\cdot\mathbb{E}\tau\) is not a finite value when \(\mathbb{E}\tau=\infty\).