Covariance Structure Of A Random Walk

Why Two Random-Walk Observations Are Related

A random walk accumulates increments over time. Consequently, two observations from the same walk can contain some of the same increments. Their covariance is determined by those shared increments: increments that appear in only one observation contribute to its variance, but not to the covariance between the pair.

This tutorial derives that relationship and shows how to apply it when the observations are at different times, when the walk has a deterministic drift, and when its starting value is random. The main calculation depends on stating the increment assumptions clearly.

Definition: For this tutorial, a random walk with deterministic starting value $x_0$ and drift $c$ is the process $$ X_t=x_0+ct+\sum_{j=1}^{t}\varepsilon_j,\qquad t=0,1,2,\ldots, $$ where the increments $\varepsilon_j$ have mean zero, common variance $\sigma^2$, and are mutually uncorrelated. Thus, $E[\varepsilon_j]=0$, $\operatorname{Var}(\varepsilon_j)=\sigma^2$, and $\operatorname{Cov}(\varepsilon_j,\varepsilon_k)=0$ for $j\ne k$. Independence is sufficient for the uncorrelated-increment condition, but is stronger than needed for the covariance result. The drift $c$ and starting value $x_0$ are deterministic.

The deterministic terms $x_0+ct$ determine the mean of $X_t$, but do not directly contribute to covariance. For two observations $X_s$ and $X_t$, the key question is therefore how many increments occur in both sums. If $s\le t$, the increments $\varepsilon_1,\ldots,\varepsilon_s$ are shared; the increments $\varepsilon_{s+1},\ldots,\varepsilon_t$ occur only in $X_t$.

Deriving the Covariance Formula

To derive the result, first subtract the mean of each observation. Under the stated assumptions, $E[X_t]=x_0+ct$, so the centered observation is the sum of its increments. Suppose first that $s\le t$. Expanding the covariance gives a double sum of covariances between increments. Every term involving two different increments is zero, while each shared increment contributes its variance $\sigma^2$.

Center each observation.
Since the starting value and drift are deterministic, $X_s-E[X_s]=\sum_{j=1}^{s}\varepsilon_j$ and $X_t-E[X_t]=\sum_{k=1}^{t}\varepsilon_k$.

Expand the covariance.
Use the covariance of two sums as the sum of the covariances of every pair of increments.

Keep only matching increments.
Mutual uncorrelatedness makes the terms with $j\ne k$ zero. There are $s$ matching pairs when $s\le t$, and each matching pair contributes $\sigma^2$.

\begin{aligned} \operatorname{Cov}(X_s,X_t) &=\operatorname{Cov}\left(\sum_{j=1}^{s}\varepsilon_j,\sum_{k=1}^{t}\varepsilon_k\right)\\ &=\sum_{j=1}^{s}\sum_{k=1}^{t}\operatorname{Cov}(\varepsilon_j,\varepsilon_k)\\ &=\sum_{j=1}^{s}\sigma^2\\ &=s\sigma^2,\qquad s\le t. \end{aligned}

Covariance is symmetric, so the same reasoning applies whichever time index is earlier. The number of shared increments is the smaller of the two time indices.

Theorem: For a random walk with deterministic starting value, constant drift, and mutually uncorrelated increments of common variance $\sigma^2$, $$ \operatorname{Cov}(X_s,X_t)=\sigma^2\min(s,t),\qquad s,t\ge 0. $$ The formula includes $s=0$ or $t=0$: $X_0=x_0$ is fixed, so its covariance with any observation is zero. In particular, setting $s=t$ gives $\operatorname{Var}(X_t)=t\sigma^2$.

The formula requires the stated increment conditions. If the increments have different variances but remain mutually uncorrelated, the covariance is the sum of the variances of the shared increments, not simply the number of shared increments times one common variance. If increments are correlated, cross-covariance terms may also contribute.

Worked Covariance Calculations

Worked Example: Two Observations With a Fixed Start

A hypothetical index follows a random walk with $X_0=100$, no drift ($c=0$), and independent increments $\varepsilon_j\sim N(0,4)$. Find $\operatorname{Cov}(X_3,X_7)$, and check the result using the variances of the two observations.

Method and assumptions. The starting value is fixed, and each increment has mean $0$ and variance $4$. The first observation contains increments $\varepsilon_1,\varepsilon_2,\varepsilon_3$; the second contains those same three increments plus $\varepsilon_4,\ldots,\varepsilon_7$.

Calculate the covariance. Each of the three shared increments contributes $4$. The four increments that occur only in $X_7$ have zero covariance with $X_3$, by independence.

\operatorname{Cov}(X_3,X_7) =4+4+4 =12.

Check using the variances. There are three increments in $X_3$, so $\operatorname{Var}(X_3)=3(4)=12$. There are seven in $X_7$, so $\operatorname{Var}(X_7)=7(4)=28$. The covariance $12$ is no greater than $\sqrt{12\cdot28}=\sqrt{336}$, as required by the covariance bound. It also equals the variance of the earlier observation, consistent with all of $X_3$'s random increments being shared with $X_7$.

Common mistake avoided. Using $7(4)=28$ as the covariance confuses the later observation’s variance with the shared contribution. Only three increments are common, so the covariance is $12$.

Worked Example: Drift Does Not Change Covariance

A hypothetical account balance is modeled by a random walk with $X_0=50$, drift $c=0.5$ per time step, and independent increments $\varepsilon_j\sim N(0,9)$. Calculate $\operatorname{Cov}(X_2,X_5)$ and verify it against the two variances.

Method and assumptions. The increment mean is $0$, each increment variance is $9$, and the starting value and drift are deterministic. The means are $E[X_2]=50+2(0.5)=51$ and $E[X_5]=50+5(0.5)=52.5$. Centering removes these deterministic means.

The centered observations are $X_2-E[X_2]=\varepsilon_1+\varepsilon_2$ and $X_5-E[X_5]=\varepsilon_1+\varepsilon_2+\varepsilon_3+\varepsilon_4+\varepsilon_5$. Exactly two increments are shared, each with variance $9$.

\operatorname{Cov}(X_2,X_5) =\operatorname{Cov}(\varepsilon_1+\varepsilon_2,\varepsilon_1+\varepsilon_2+\varepsilon_3+\varepsilon_4+\varepsilon_5) =9+9=18.

Check. The variances are $\operatorname{Var}(X_2)=2(9)=18$ and $\operatorname{Var}(X_5)=5(9)=45$. The covariance bound is satisfied because $18\le\sqrt{18\cdot45}=\sqrt{810}$. The formula also gives $9\min(2,5)=18$.

Common mistake avoided. The drift changes the means, but it is not a random component. Adding $0.5$ to each increment’s variance or multiplying the covariance by the drift would be incorrect.

Worked Example: A Random Starting Value

Suppose a hypothetical index begins at a random level $U\sim N(40,5)$, where the second parameter is the variance. It has drift $c=0.25$ and independent increments $\varepsilon_j\sim N(0,2)$, independent of $U$. Find $\operatorname{Cov}(X_4,X_6)$, using $X_t=U+ct+\sum_{j=1}^{t}\varepsilon_j$.

Method and assumptions. Unlike a fixed starting value, $U$ contributes random variation to both observations. Its variance is $5$, and because it is independent of all increments, its covariance with each increment is zero. Four increments are shared by $X_4$ and $X_6$.

The means are $E[X_4]=40+0.25(4)=41$ and $E[X_6]=40+0.25(6)=41.5$. After centering, the random parts are $U-40+\sum_{j=1}^{4}\varepsilon_j$ and $U-40+\sum_{j=1}^{6}\varepsilon_j$. The shared starting value contributes $5$, and the four shared increments contribute $4(2)=8$.

\operatorname{Cov}(X_4,X_6)=5+4(2)=13.

Check. The variances are $\operatorname{Var}(X_4)=5+4(2)=13$ and $\operatorname{Var}(X_6)=5+6(2)=17$. The covariance bound holds: $13\le\sqrt{13\cdot17}=\sqrt{221}$. The answer includes the starting-value variance because the same random $U$ appears in both observations.

Common mistake avoided. Applying $2\min(4,6)=8$ alone would omit the shared random starting value. That shortcut applies only when the starting value is fixed, or when its variance is otherwise excluded from the model.

Interpreting the Structure and Avoiding Traps

For a fixed starting value, the covariance between two observations depends on the earlier time index and the increment variance. It does not depend on the gap between the observations. For example, $X_2$ shares two increments with $X_5$, and it also shares two increments with $X_{20}$; under the same model, both covariances equal $2\sigma^2$. The later observation has more increments overall, but those additional increments do not appear in the earlier observation.

This interpretation is useful because it gives a direct way to check a calculation: list the increments in each observation, identify their intersection, and add the variances of the common increments. For equal-variance, mutually uncorrelated increments, that sum reduces to $\sigma^2$ times the number of shared increments. With a random initial value independent of the increments, add its variance as a further shared contribution.

Key takeaway: In a random walk with fixed starting value and equal-variance, mutually uncorrelated increments, two observations share exactly the increments up to the earlier time. Therefore, $\operatorname{Cov}(X_s,X_t)=\sigma^2\min(s,t)$. A deterministic drift changes neither the shared increments nor their covariance.

Check Your Understanding

For each question, identify the shared random components before calculating.

A random walk has a fixed starting value, zero drift, and independent increments with distribution $N(0,3)$. Find $\operatorname{Cov}(X_2,X_5)$. (Answer: $6$)
A random walk has $X_0=12$, drift $c=-0.2$, and mutually independent increments with distribution $N(0,2.5)$. Find $\operatorname{Cov}(X_4,X_4)$. (Answer: $10$)
A fixed-start random walk has independent increments with distribution $N(0,0.8)$. Find $\operatorname{Cov}(X_1,X_6)$, and state how many increments are shared. (Answer: $0.8$; one shared increment)
A random walk starts at $U\sim N(7,4)$, has zero drift, and has independent increments with distribution $N(0,3)$, independent of $U$. Find $\operatorname{Cov}(X_2,X_5)$. (Answer: $10$)
A fixed-start random walk has mutually uncorrelated increments, each with variance $1.5$, and deterministic drift $c=2$. Find $\operatorname{Cov}(X_3,X_8)$, then explain why the drift does not enter the answer. (Answer: $4.5$; the drift is deterministic and disappears when the observations are centered.)

What You'll Learn