Three Ways to Compare a Process with Its Conditional Future
A stochastic process may have a conditional mean at the next time that equals its current value, is at least its current value, or is at most its current value. These three possibilities define martingales, submartingales, and supermartingales. The comparison is conditional on the information currently available: comparing only the ordinary expectations can miss important state-by-state behavior.
Work on a probability space \((\Omega,\mathcal{F},\mathbb{P})\) with a discrete-time filtration \((\mathcal{F}_n)_{n\ge0}\), meaning that each \(\mathcal{F}_n\) is a sigma-field and \(\mathcal{F}_n\subseteq\mathcal{F}_{n+1}\). Consider a real-valued process \((X_n)_{n\ge0}\). It is adapted if \(X_n\) is \(\mathcal{F}_n\)-measurable for every \(n\). It is integrable at time \(n\) if \(\mathbb{E}|X_n|<\infty\). Integrability ensures that the conditional expectation of \(X_{n+1}\) given \(\mathcal{F}_n\) is defined.
Each condition is required at every time, with the equality or inequality holding almost surely; the exceptional null set may depend on \(n\). The three types all require adaptation and integrability. They differ only in the direction of the conditional comparison. A process can be both a submartingale and a supermartingale, but then the two inequalities force equality almost surely.
When \(X_n\) takes values in a finite subset of the real numbers, it is bounded, so \(\mathbb{E}|X_n|<\infty\) automatically. This does not remove the adaptation requirement, nor does boundedness establish any of the conditional comparisons. Also, the filtration matters: the comparison is made using precisely the information represented by \(\mathcal{F}_n\). No Markov property is required for the definitions.
Checking the Conditional Comparison in a Finite-State Chain
For a finite-state Markov chain with transition matrix \(P=(p_{ij})\), the rows are probability distributions: \(p_{ij}\ge0\) and \(\sum_jp_{ij}=1\) for every state \(i\). With the natural filtration \(\mathcal{F}_n=\sigma(X_0,\ldots,X_n)\), the Markov property says that the conditional distribution of \(X_{n+1}\), given \(\mathcal{F}_n\), is the row of \(P\) indexed by \(X_n\). Thus, for a real-valued function \(g\) on the state space, \(\mathbb{E}[g(X_{n+1})\mid\mathcal{F}_n]=\sum_jp_{X_nj}g(j)\). For the process \(X_n\) itself, the relevant conditional mean is obtained by taking \(g(j)=j\). In each example below, the state labels are real numbers and the natural filtration is used.
Worked Example: A Martingale on Three States
Let \(S=\{0,1,2\}\), let \(X_0=1\), and use the transition matrix \(\begin{bmatrix}1&0&0\\ \tfrac12&0&\tfrac12\\0&0&1\end{bmatrix}\). The row sums are \(1+0+0=1\), \(\tfrac12+0+\tfrac12=1\), and \(0+0+1=1\). The initial distribution is \(\mu_0=(0,1,0)\), whose entries sum to \(1\). Use the natural filtration.
At state \(0\), the conditional mean of the next state is \(0\cdot1+1\cdot0+2\cdot0=0\). At state \(1\), it is \(0\cdot\tfrac12+1\cdot0+2\cdot\tfrac12=1\). At state \(2\), it is \(0\cdot0+1\cdot0+2\cdot1=2\). Therefore the conditional mean equals the current state at every possible state, so \(\mathbb{E}[X_{n+1}\mid\mathcal{F}_n]=X_n\) almost surely for every \(n\).
The process is adapted to its natural filtration. Its values are always in the finite set \(\{0,1,2\}\), so every \(X_n\) is integrable. These checks establish all the requirements for a martingale. As a further check at time \(0\), \(X_0=1\); the chain moves to \(0\) or \(2\) with probability \(\tfrac12\) each, giving \(\mathbb{E}[X_1]=0\cdot\tfrac12+2\cdot\tfrac12=1\), consistent with the conditional calculation.
Worked Example: A Submartingale with Strict Conditional Increase
Let \(S=\{0,1\}\), start at \(X_0=0\), and take \(P=\begin{bmatrix}\tfrac12&\tfrac12\\0&1\end{bmatrix}\). The row sums are \(\tfrac12+\tfrac12=1\) and \(0+1=1\). The initial distribution is \(\mu_0=(1,0)\), which sums to \(1\), and again use the natural filtration.
From state \(0\), the next state is \(0\) or \(1\), each with probability \(\tfrac12\), so its conditional mean is \(0\cdot\tfrac12+1\cdot\tfrac12=\tfrac12\), which is at least the current value \(0\). From state \(1\), the next state is \(1\) with probability \(1\), so its conditional mean is \(1\), equal to the current value. The conditional comparison is therefore at least the current state at both states. The process is adapted and bounded by \(1\), hence integrable at every time; it is a submartingale.
It is not a martingale: at time \(0\), \(X_0=0\) with probability \(1\), whereas \(\mathbb{E}[X_1\mid\mathcal{F}_0]=\tfrac12\) with probability \(1\). The inequality is strict at that time. Its unconditional expectation confirms the increase: \(\mathbb{E}[X_0]=0\) and \(\mathbb{E}[X_1]=0\cdot\tfrac12+1\cdot\tfrac12=\tfrac12\).
Worked Example: A Supermartingale with Strict Conditional Decrease
Let \(S=\{0,1\}\), start at \(X_0=1\), and take \(P=\begin{bmatrix}1&0\\\tfrac12&\tfrac12\end{bmatrix}\). The row sums are \(1+0=1\) and \(\tfrac12+\tfrac12=1\). The initial distribution is \(\mu_0=(0,1)\), which sums to \(1\), and use the natural filtration.
At state \(0\), the next state is \(0\) with probability \(1\), so its conditional mean is \(0\), equal to the current state. At state \(1\), the next state is \(0\) or \(1\), each with probability \(\tfrac12\). Its conditional mean is \(0\cdot\tfrac12+1\cdot\tfrac12=\tfrac12\), which is at most the current value \(1\). Thus the conditional comparison is at most the current state at both states. The process is adapted and bounded, so it is an integrable supermartingale.
It is not a martingale: at time \(0\), the current value is \(1\) with probability \(1\), while the conditional mean of the next value is \(\tfrac12\). As an unconditional check, \(\mathbb{E}[X_0]=1\) and \(\mathbb{E}[X_1]=0\cdot\tfrac12+1\cdot\tfrac12=\tfrac12\), a decrease.
What the Conditional Inequalities Imply
Proof. Suppose first that \((X_n)\) is a submartingale. Its defining inequality is \(\mathbb{E}[X_{n+1}\mid\mathcal{F}_n]\ge X_n\) almost surely. Both sides are integrable: conditional expectation preserves integrability, and \(X_n\) is integrable by assumption. Taking expectations and using \(\mathbb{E}[\mathbb{E}[X_{n+1}\mid\mathcal{F}_n]]=\mathbb{E}[X_{n+1}]\) gives \(\mathbb{E}[X_{n+1}]\ge\mathbb{E}[X_n]\). For a supermartingale, the defining inequality has the reverse direction, so the same expectation identity gives \(\mathbb{E}[X_{n+1}]\le\mathbb{E}[X_n]\).
If the process is both, then for each \(n\) its conditional expectation is at least \(X_n\) and at most \(X_n\) almost surely. Hence it equals \(X_n\) almost surely, which is the martingale condition. Finally, suppose it is a submartingale and its expectations are equal at consecutive times. The random variable \(Z=\mathbb{E}[X_{n+1}\mid\mathcal{F}_n]-X_n\) is integrable and nonnegative almost surely. Its expectation is \(\mathbb{E}[Z]=\mathbb{E}[X_{n+1}]-\mathbb{E}[X_n]=0\). A nonnegative integrable random variable with expectation zero is zero almost surely: if it were positive with positive probability, then for some positive integer \(k\), the event \(\{Z\ge 1/k\}\) would have positive probability, forcing \(\mathbb{E}[Z]\ge\mathbb{P}(Z\ge1/k)/k>0\). Thus \(Z=0\) almost surely, and the conditional expectation equals \(X_n\). This holds for every \(n\), proving the martingale condition. \(\square\)
The theorem gives useful checks, but unconditional expectations alone do not establish any of the three definitions. They average over the information in \(\mathcal{F}_n\). A conditional inequality must hold given that information, not merely after averaging across it.
A Common Mistake: Constant Means Do Not Guarantee a Martingale
Worked Example: An Oscillating Chain with Constant Unconditional Mean
Let \(S=\{0,1\}\), use \(P=\begin{bmatrix}0&1\\1&0\end{bmatrix}\), and set \(\mu_0=(\tfrac12,\tfrac12)\). The row sums are \(0+1=1\) and \(1+0=1\), and the initial probabilities sum to \(\tfrac12+\tfrac12=1\). Use the natural filtration.
A tempting but wrong calculation is: “\(\mathbb{E}[X_n]=\tfrac12\) at every time, so this is a martingale.” The unconditional calculation is correct. If \(\mu_n=(\tfrac12,\tfrac12)\), then \(\mu_{n+1}=\mu_nP=(\tfrac12\cdot0+\tfrac12\cdot1,\ \tfrac12\cdot1+\tfrac12\cdot0)=(\tfrac12,\tfrac12)\). Since \(X_n\) takes values \(0\) and \(1\) with these probabilities, \(\mathbb{E}[X_n]=0\cdot\tfrac12+1\cdot\tfrac12=\tfrac12\) at every time.
The correction is to check the conditional comparison. From state \(0\), the next state is \(1\) with probability \(1\), so \(\mathbb{E}[X_{n+1}\mid\mathcal{F}_n]=1>0=X_n\). From state \(1\), the next state is \(0\) with probability \(1\), so \(\mathbb{E}[X_{n+1}\mid\mathcal{F}_n]=0<1=X_n\). Both states have positive probability at every time, so neither conditional inequality direction holds almost surely. The process is neither a submartingale nor a supermartingale, and in particular it is not a martingale. It is adapted and bounded; the failure is specifically in the conditional comparison.
This example highlights the distinction between a process property and a statement about its averaged values. For a submartingale, expected values cannot decrease; for a supermartingale, they cannot increase. The converse is not generally true, because the conditional comparisons can fail in opposite directions at different states and cancel in the unconditional average.
When checking a proposed process, proceed in order: identify the filtration, verify adaptation and integrability at every time, and then compare \(\mathbb{E}[X_{n+1}\mid\mathcal{F}_n]\) with \(X_n\) almost surely. For finite-state chains, compute this conditional mean for each state that can occur. Equality gives a martingale; a consistent inequality gives the corresponding submartingale or supermartingale. A comparison that changes direction across possible states does not qualify as either.
Check Your Understanding
For each question, distinguish the conditional comparison from any statement about ordinary expectations.
- State the adaptation, integrability, and conditional-expectation requirements for a supermartingale.
- For a chain on \(\{0,1\}\) with \(P=\begin{bmatrix}\tfrac14&\tfrac34\\0&1\end{bmatrix}\), compute the conditional mean of the next state at each current state. Which process type does \((X_n)\) satisfy?
- Why is integrability automatic for a process taking values in a fixed finite subset of the real numbers?
- If a process is both a submartingale and a supermartingale with respect to the same filtration, what conditional equality follows?
- Can constant unconditional expectations prove that a process is a martingale? Explain using the conditional comparison.