Orthogonality of Martingale Differences

Why Martingale Differences Are Orthogonal

A martingale difference has conditional mean zero given the information available just before it occurs. That condition is stronger than having an unconditional mean of zero: it says that, even after the past is known, the expected next difference remains zero. This feature makes distinct differences orthogonal in expectation. Consequently, when a finite sum of such differences is squared, all the cross terms vanish, leaving only the individual second moments.

Work on a probability space $(\Omega,\mathcal{F},\mathbb{P})$ with a filtration $(\mathcal{F}_n)_{n\ge0}$, meaning an increasing sequence of sub-$\sigma$-fields of $\mathcal{F}$. All processes below are real-valued, so their state space is $S=\mathbb{R}$. A discrete-time process $(M_n)_{n\ge0}$ is a martingale with respect to this filtration if each $M_n$ is integrable and $\mathcal{F}_n$-measurable, and $\mathbb{E}[M_n\mid\mathcal{F}_{n-1}]=M_{n-1}$ almost surely for every $n\ge1$. We call it square-integrable when $\mathbb{E}[M_n^2]<\infty$ for every $n$.

Definition: For an adapted process $(M_n)_{n\ge0}$, its $n$th martingale difference is $D_n=M_n-M_{n-1}$, for $n\ge1$. The process is a martingale exactly when its differences are integrable and satisfy $\mathbb{E}[D_n\mid\mathcal{F}_{n-1}]=0$ almost surely for every $n\ge1$, in addition to the required adaptedness and integrability of $M_n$. In this tutorial, a square-integrable martingale difference sequence means that $D_n$ is $\mathcal{F}_n$-measurable, $\mathbb{E}[D_n^2]<\infty$, and $\mathbb{E}[D_n\mid\mathcal{F}_{n-1}]=0$ almost surely for each $n\ge1$.

In particular, a square-integrable martingale has square-integrable differences: $D_n=M_n-M_{n-1}$ and $D_n^2\le 2M_n^2+2M_{n-1}^2$, so its second moment is finite. Also, each difference has mean zero. Indeed, taking expectations of its conditional-mean condition gives $\mathbb{E}[D_n]=0$. These facts alone do not yet show that two different differences have zero expected product; the conditional-mean condition supplies the additional argument.

The Orthogonality Result

Theorem: Let $(D_n)_{n\ge1}$ be a square-integrable martingale difference sequence with respect to $(\mathcal{F}_n)_{n\ge0}$. For any integers $1\le i<j$, $\mathbb{E}[D_iD_j]=0$. Thus differences at distinct times are orthogonal in expectation. Here orthogonality means zero expected product; it does not mean that the random variables are independent.

Proof. Fix $i<j$. Since the filtration is increasing, $\mathcal{F}_i\subseteq\mathcal{F}_{j-1}$; since $D_i$ is $\mathcal{F}_i$-measurable, it is also $\mathcal{F}_{j-1}$-measurable. Both $D_i$ and $D_j$ are square-integrable, so their product is integrable: by Cauchy–Schwarz, $\mathbb{E}[|D_iD_j|]\le(\mathbb{E}[D_i^2])^{1/2}(\mathbb{E}[D_j^2])^{1/2}<\infty$. The conditional expectation identity for square-integrable variables gives $\mathbb{E}[D_iD_j]=\mathbb{E}[D_i\,\mathbb{E}[D_j\mid\mathcal{F}_{j-1}]]$. The inner conditional expectation is zero almost surely by the martingale difference condition at time $j$. Therefore $\mathbb{E}[D_iD_j]=0$. $\square$

The measurability step is essential: the earlier difference $D_i$ is known by time $j-1$, when the conditional mean of $D_j$ is taken. For general random variables with zero means, there is no reason their product should have mean zero. The theorem also does not say that the product $D_iD_j$ is zero on every outcome; it says that its expectation is zero.

Corollary (Variance of a finite sum): Under the theorem’s hypotheses, for any integers $1\le r\le s$, $$ \operatorname{Var}\!\left(\sum_{k=r}^{s}D_k\right)=\sum_{k=r}^{s}\mathbb{E}[D_k^2]. $$

Proof. Each $D_k$ is integrable and has mean zero, so the finite sum $Y=\sum_{k=r}^{s}D_k$ has mean zero and $\operatorname{Var}(Y)=\mathbb{E}[Y^2]$. Expanding the square gives $\mathbb{E}[Y^2]=\sum_{k=r}^{s}\mathbb{E}[D_k^2]+2\sum_{r\le i<j\le s}\mathbb{E}[D_iD_j]$. Every cross term is zero by the theorem, so only the sum of the individual second moments remains. If $r=s$, there are no cross terms, and the same formula reduces directly to $\operatorname{Var}(D_r)=\mathbb{E}[D_r^2]$. $\square$

For a square-integrable martingale, $\sum_{k=1}^{n}D_k=M_n-M_0$, so the corollary gives the variance of the martingale’s change from its initial value. It does not by itself give $\operatorname{Var}(M_n)$ when $M_0$ is random: that would also require accounting for the relationship between $M_0$ and the sum of the differences. If $M_0$ is a constant, then $\operatorname{Var}(M_n)=\sum_{k=1}^{n}\mathbb{E}[D_k^2]$.

Worked Calculations

Worked Example: Differences of a Conditional-Expectation Martingale

Let $\Omega=\{a,b,c,d\}$, with each outcome having probability $\tfrac14$. Let $\mathcal{F}_0$ contain only the empty set and $\Omega$, let $\mathcal{F}_1$ distinguish the two groups $\{a,b\}$ and $\{c,d\}$, and let $\mathcal{F}_2=\mathcal{F}$ distinguish all four outcomes. Define $H(a)=0$, $H(b)=2$, $H(c)=1$, and $H(d)=5$. Set $M_0=\mathbb{E}[H]$, $M_1=\mathbb{E}[H\mid\mathcal{F}_1]$, and $M_2=H$. Find the differences and verify the variance formula.

The unconditional mean is $\mathbb{E}[H]=(0+2+1+5)/4=2$, so $M_0=2$. On $\{a,b\}$, the conditional mean is $(0+2)/2=1$; on $\{c,d\}$, it is $(1+5)/2=3$. Thus $M_1(a)=M_1(b)=1$ and $M_1(c)=M_1(d)=3$. The differences $D_1=M_1-M_0$ and $D_2=M_2-M_1$ are:

\begin{array}{c|c|c|c|c} \omega & M_0 & M_1 & M_2 & (D_1,D_2)\\ \hline a&2&1&0&(-1,-1)\\ b&2&1&2&(-1,1)\\ c&2&3&1&(1,-2)\\ d&2&3&5&(1,2) \end{array}

On each group in $\mathcal{F}_1$, the two values of $D_2$ sum to zero. Therefore $\mathbb{E}[D_2\mid\mathcal{F}_1]=0$, and $D_1$ is $\mathcal{F}_1$-measurable. Also $\mathbb{E}[D_1]=(-1-1+1+1)/4=0$. The second moments are $\mathbb{E}[D_1^2]=(1+1+1+1)/4=1$ and $\mathbb{E}[D_2^2]=(1+1+4+4)/4=\tfrac52$. Directly, $\mathbb{E}[D_1D_2]=(1-1-2+2)/4=0$.

Check by the sum: $D_1+D_2=M_2-M_0$ takes the values $-2,0,-1,3$. Its mean is $(-2+0-1+3)/4=0$, and its variance is $(4+0+1+9)/4=\tfrac72$. The formula gives $1+\tfrac52=\tfrac72$, in agreement.

Worked Example: Orthogonal Does Not Mean Independent

Let $X$ and $Y$ take values in $\{-1,1\}$, with the four pairs $(X,Y)$ equally likely. Let $\mathcal{F}_0$ be trivial, $\mathcal{F}_1$ contain the information in $X$, and $\mathcal{F}_2$ contain the information in $(X,Y)$. Define $D_1=X$, and define $D_2=2Y$ when $X=1$ and $D_2=Y$ when $X=-1$.

Since $X$ is equally likely to be $1$ or $-1$, $\mathbb{E}[D_1]=0$ and $\mathbb{E}[D_1^2]=1$. For either fixed value of $X$, the two values of $Y$ are equally likely and have opposite signs. Hence $\mathbb{E}[D_2\mid\mathcal{F}_1]=0$. Its second moment is $\mathbb{E}[D_2^2]=\tfrac12(2^2)+\tfrac12(1^2)=\tfrac52$. The theorem gives $\mathbb{E}[D_1D_2]=0$.

In detail, the four equally likely $(D_1,D_2)$ pairs are $(1,2),(1,-2),(-1,1),(-1,-1)$. Their products are $2,-2,-1,1$, whose average is $(2-2-1+1)/4=0$. Yet $D_1$ and $D_2$ are not independent: $D_1=1$ implies $|D_2|=2$, while $D_1=-1$ implies $|D_2|=1$. The variance of their sum is $1+\tfrac52=\tfrac72$; directly, the sum takes values $3,-1,0,-2$, whose mean is zero and whose second moment is $(9+1+0+4)/4=\tfrac72$.

Worked Example: Three Differences on a Finite Probability Space

Let $\Omega=\{-1,1\}^3$, with all eight triples $(X,Y,Z)$ equally likely. Let $\mathcal{F}_0$ be trivial, let $\mathcal{F}_1$ contain the information in $X$, let $\mathcal{F}_2$ contain the information in $(X,Y)$, and let $\mathcal{F}_3$ contain the information in $(X,Y,Z)$. Define $D_1=X$, $D_2=2Y$ if $X=1$ and $D_2=Y$ if $X=-1$, and $D_3=2Z$ if $X=1,Y=1$ and $D_3=Z$ otherwise.

Because $X$ is equally likely to be either sign, $\mathbb{E}[D_1]=0$ and $\mathbb{E}[D_1^2]=1$. Given $X$, the two signs of $Y$ are equally likely, so $\mathbb{E}[D_2\mid\mathcal{F}_1]=0$. As in the preceding calculation, $\mathbb{E}[D_2^2]=\tfrac12(4)+\tfrac12(1)=\tfrac52$. Given $X,Y$, the two signs of $Z$ are equally likely, so $\mathbb{E}[D_3\mid\mathcal{F}_2]=0$. The multiplier of $Z$ is $2$ on one quarter of the outcomes and $1$ on three quarters, giving $\mathbb{E}[D_3^2]=\tfrac14(4)+\tfrac34(1)=\tfrac74$. All three variables are bounded, hence square-integrable. The variance formula predicts $1+\tfrac52+\tfrac74=\tfrac{21}4$.

Direct check: For $(X,Y)=(1,1)$, the two possible sums $D_1+D_2+D_3$ are $5$ and $1$. For $(1,-1)$, they are $0$ and $-2$. For $(-1,1)$, they are $1$ and $-1$. For $(-1,-1)$, they are $-1$ and $-3$. The eight squared sums are therefore $25,1,0,4,1,1,1,9$. Their average is $42/8=\tfrac{21}4$. The sum has mean zero because each difference has mean zero, so this second moment is its variance, as predicted.

A Common Error and the Scope of the Formula

Common mistake. Zero means do not imply orthogonality. For a concrete calculation, let $X$ equal $1$ or $-1$ with probability $\tfrac12$ each, let $\mathcal{F}_0$ be trivial and $\mathcal{F}_1=\mathcal{F}$, and set $D_1=X$, $D_2=X$. Both variables have mean zero and second moment $1$, but $\mathbb{E}[D_1D_2]=\mathbb{E}[X^2]=1$, not zero. The sum $D_1+D_2=2X$ has variance $4$, not $1+1=2$. The error is treating unconditional mean zero as if it were the condition $\mathbb{E}[D_2\mid\mathcal{F}_1]=0$. In this example that conditional expectation is $X$, which is not zero.

The hypotheses in the theorem and corollary have distinct roles. The filtration and conditional-mean condition yield orthogonality; square-integrability makes each second moment finite and ensures that products are integrable, so the finite variance calculation is valid. The formula applies to finite sums and needs no independence assumption. For an infinite sum, additional conditions would be needed to justify convergence and any corresponding variance identity; the finite-sum result alone does not assert such a limit.

Check Your Understanding

For each question, identify the conditional-mean or integrability condition being used.

If $i<j$, why is $D_i$ measurable with respect to $\mathcal{F}_{j-1}$?
Suppose $D_1,D_2,D_3$ are square-integrable martingale differences with second moments $2,3,5$. What is the variance of $D_1+D_2+D_3$?
In the conditional-expectation example, compute $\mathbb{E}[D_2^2]$ from the four listed outcomes and compare it with $\tfrac52$.
Can two orthogonal martingale differences be dependent? Explain using the conditional sizes in the second worked example.
For $D_1=D_2=X$ with $X$ equally likely to be $1$ or $-1$, which martingale-difference condition fails for $D_2$ relative to $\mathcal{F}_1=\sigma(X)$?

Orthogonality of Martingale Differences

What You'll Learn