Cyclic Three-State Chains and Complex Eigenvalues

A Three-State Cycle

In a continuous-time Markov chain, a transition probability can vary with time even when the transition rates are constant. For a chain that moves clockwise around three states at the same rate, the probability of being at a particular state oscillates as the chain cycles, while the oscillation’s amplitude decays. Complex eigenvalues describe this behavior compactly; combining conjugate terms turns the result into real-valued probabilities.

Work on a probability space $(\Omega,\mathcal{F},\mathbb{P})$, and let $(X_t)_{t\ge0}$ be a continuous-time Markov chain with state space $S=\{1,2,3\}$. From state $1$ the chain jumps to $2$, from $2$ to $3$, and from $3$ to $1$, each at rate $\lambda$, where $\lambda>0$. The generator and the time-$t$ transition matrix are

Q=\begin{bmatrix} -\lambda&\lambda&0\\ 0&-\lambda&\lambda\\ \lambda&0&-\lambda \end{bmatrix}, \qquad P(t)=e^{tQ}.

Each row of $Q$ sums to $-\lambda+\lambda+0=0$, so the generator has the required zero row sums. The diagonal entries are nonpositive, and the off-diagonal entries are nonnegative. We use row distributions: if the initial distribution is the row vector $\mu$, the distribution at time $t$ is $\mu P(t)$. No conditioning is needed below; if conditioning on the chain’s past is used, its natural filtration is $\mathcal{F}_t=\sigma(X_s:0\le s\le t)$.

Definition: For $i,j\in S$, $p_{ij}(t)=\mathbb{P}(X_t=j\mid X_0=i)$ is the probability of being at $j$ at time $t$ when the chain starts at $i$. The matrix $P(t)=(p_{ij}(t))$ is the transition matrix at time $t$. The cyclic displacement from $i$ to $j$ is the unique $d\in\{0,1,2\}$ congruent to $j-i$ modulo $3$.

From the Generator to a Real Formula

Let $C$ denote the clockwise shift matrix. Then $Q=\lambda(C-I)$, where $I$ is the identity matrix, and $C^3=I$. The complex cube roots of unity are $1,\omega,\omega^2$, where $\omega=e^{2\pi i/3}=-\tfrac12+i\tfrac{\sqrt3}{2}$. Since these are the distinct roots of the characteristic polynomial $x^3-1$ of $C$, the eigenvalues of $Q$ are $\lambda(1-1)=0$, $\lambda(\omega-1)=-\tfrac32\lambda+i\tfrac{\sqrt3}{2}\lambda$, and its complex conjugate. The nonzero pair explains the damped oscillation.

Theorem: For $\lambda>0$, $t\ge0$, and $d\in\{0,1,2\}$, set $a=e^{-3\lambda t/2}$ and $\theta=\sqrt3\lambda t/2$. The probability of a cyclic displacement $d$ is $$ p_d(t)=\frac13\left[1+2a\cos\left(\theta-\frac{2\pi d}{3}\right)\right]. $$ Consequently, $$ P(t)= \begin{bmatrix} p_0(t)&p_1(t)&p_2(t)\\ p_2(t)&p_0(t)&p_1(t)\\ p_1(t)&p_2(t)&p_0(t) \end{bmatrix}. $$

Proof. Put $z=\lambda t$. Since $Q=\lambda(C-I)$ and $C$ commutes with $I$, the matrix exponential power series gives $P(t)=e^{-z}\sum_{n=0}^{\infty}z^nC^n/n!$. The entry from $i$ to $j$ in $C^n$ is $1$ exactly when $n$ is congruent to $d=j-i$ modulo $3$, and is $0$ otherwise. For any integer $n$, the indicator of that congruence is $\tfrac13\sum_{m=0}^2\omega^{m(n-d)}$: if $n-d$ is divisible by $3$, the sum is $3$; otherwise it is $1+\omega+\omega^2=0$. Substituting this identity into the series yields $$ p_d(t)=\frac13\sum_{m=0}^2\omega^{-md}e^{z(\omega^m-1)}. $$ The $m=0$ term is $1$. The $m=1$ term is $a e^{i(\theta-2\pi d/3)}$, and the $m=2$ term is its conjugate. Their sum is $2a\cos(\theta-2\pi d/3)$, proving the formula. The series expression $e^{-z}\sum_{n\equiv d\ (3)}z^n/n!$ also shows that every $p_d(t)$ is nonnegative. In each row of the displayed matrix, the three displacements occur once each, and the indicator identity (or the exponential series at $C$) gives their sum as $e^{-z}e^z=1$. The entries therefore are transition probabilities and the rows sum to $1$. $\square$

The formula separates the constant term $1/3$ from the oscillating terms. The factor $a$ shrinks with time, while the cosine phase changes with time. The displacement convention matters: $d=1$ means one clockwise move, not one counterclockwise move.

Worked Example: A Transition Matrix at a Specified Phase

Take $\lambda=1$ and $t=4\pi/(3\sqrt3)$. Then $\lambda t=4\pi/(3\sqrt3)$, so $\theta=\sqrt3\lambda t/2=2\pi/3$ and $a=e^{-3\lambda t/2}=e^{-2\pi/\sqrt3}$. Substitution into the formula gives

\begin{aligned} p_0(t)&=\tfrac13[1+2a\cos(2\pi/3)]=\tfrac{1-a}{3},\\ p_1(t)&=\tfrac13[1+2a\cos(0)]=\tfrac{1+2a}{3},\\ p_2(t)&=\tfrac13[1+2a\cos(-2\pi/3)]=\tfrac{1-a}{3}. \end{aligned}

Here $\cos(2\pi/3)=\cos(-2\pi/3)=-1/2$ and $\cos(0)=1$. Because $2\pi/\sqrt3>0$, $0<a<1$, so these entries are positive. The generator has rows $( -1,1,0)$, $(0,-1,1)$, and $(1,0,-1)$; each sums to $0$. The transition matrix is

P(t)=\frac13 \begin{bmatrix} 1-a&1+2a&1-a\\ 1-a&1-a&1+2a\\ 1+2a&1-a&1-a \end{bmatrix}.

Check: Each row sums to $\big((1-a)+(1+2a)+(1-a)\big)/3=3/3=1$. The entry from $1$ to $2$, the clockwise neighbor, is $(1+2a)/3$, while the entry from $1$ to $3$ is $(1-a)/3$. The clockwise direction is therefore reflected in the computed probabilities.

Worked Example: Probabilities at a Different Phase

Take $\lambda=1$ and $t=\pi/\sqrt3$. Then $\theta=\sqrt3\lambda t/2=\pi/2$ and $a=e^{-3\lambda t/2}=e^{-\pi/2}$. The three cosine values, in displacement order, are $\cos(\pi/2)=0$, $\cos(-\pi/6)=\sqrt3/2$, and $\cos(-5\pi/6)=-\sqrt3/2$. Thus

p_0(t)=\tfrac13,\qquad p_1(t)=\frac{1+\sqrt3e^{-\pi/2}}{3},\qquad p_2(t)=\frac{1-\sqrt3e^{-\pi/2}}{3}.

These values are nonnegative: $e^\pi>1+\pi>4>3$, so $e^{\pi/2}>\sqrt3$ and $\sqrt3e^{-\pi/2}<1$. The generator is again the displayed cycle generator with $\lambda=1$, whose row sums are each $-1+1+0=0$. Starting from state $1$, the distribution at this time is $(p_0(t),p_1(t),p_2(t))$.

Check: The row sum is $\tfrac13+(1+\sqrt3e^{-\pi/2})/3+(1-\sqrt3e^{-\pi/2})/3=3/3=1$. The difference between the clockwise and counterclockwise neighbor probabilities is $p_1(t)-p_2(t)=2\sqrt3e^{-\pi/2}/3>0$, consistent with the chain’s clockwise orientation.

Worked Example: Evolving a Nonuniform Initial Distribution

Let $\lambda=1$, $t=4\pi/(3\sqrt3)$, and $\mu=(1/2,1/3,1/6)$. The rates are the same as in the first example, so every generator row sums to $0$. At this time, define $a=e^{-2\pi/\sqrt3}$; the transition matrix entries are $p_0=p_2=x=(1-a)/3$ and $p_1=y=(1+2a)/3$. Multiplication by the row distribution gives

\begin{aligned} (\mu P(t))_1&=\tfrac12x+\tfrac13p_2+\tfrac16p_1 =\tfrac56x+\tfrac16y =\frac{5(1-a)+(1+2a)}{18} =\frac{2-a}{6},\\ (\mu P(t))_2&=\tfrac12p_1+\tfrac13x+\tfrac16p_2 =\tfrac12y+\tfrac12x =\frac{(1+2a)+(1-a)}{6} =\frac{2+a}{6},\\ (\mu P(t))_3&=\tfrac12p_2+\tfrac13p_1+\tfrac16x =\tfrac23x+\tfrac13y =\frac{2(1-a)+(1+2a)}{9} =\frac13. \end{aligned}

Since $0<a<1$, each resulting component is positive. This calculation uses the row-vector convention: the probability of ending in state $j$ is the sum over starting states $i$ of $\mu_i p_{ij}(t)$.

Check: The components sum to $(2-a)/6+(2+a)/6+1/3=4/6+2/6=1$. At $t=0$, the general formula gives $a=1$, $\theta=0$, and $(p_0,p_1,p_2)=(1,0,0)$, so $P(0)=I$ and the initial distribution is unchanged, as required.

Long-Time Behavior and a Sign Pitfall

Corollary: If $\lambda>0$, then for every $i,j\in S$, $p_{ij}(t)$ tends to $1/3$ as $t$ tends to infinity. Consequently, for any initial row distribution $\mu$, $\mu P(t)$ tends to $(1/3,1/3,1/3)$.

Proof. Since $\lambda>0$, $a=e^{-3\lambda t/2}$ tends to $0$ as $t$ tends to infinity. The cosine factor is bounded between $-1$ and $1$, so $2a\cos(\theta-2\pi d/3)$ tends to $0$. The formula gives $p_d(t)\to1/3$ for each displacement, hence every entry of $P(t)$ tends to $1/3$. For each $j$, $(\mu P(t))_j=\sum_{i=1}^3\mu_i p_{ij}(t)\to\sum_{i=1}^3\mu_i/3=1/3$, because $\sum_i\mu_i=1$. $\square$

The strict condition $\lambda>0$ is essential. If $\lambda=0$, then $Q$ is the zero matrix and $P(t)=I$ for every $t$. For example, starting from state $1$ gives distribution $(1,0,0)$ at every time, not the uniform distribution. The limit is therefore a consequence of positive transition rates around the cycle, not of the formula’s constant term alone.

Common mistake: reversing the phase sign. Replacing $\theta-2\pi d/3$ with $\theta+2\pi d/3$ swaps the two nonzero displacements. For the clockwise neighbor $d=1$, the incorrect expression gives $\widetilde p_1(t)=\tfrac13[1+2a\cos(\theta+2\pi/3)]$. At $t=0$, $a=1$ and $\theta=0$, so $\widetilde p_1(0)=\tfrac13[1+2\cos(2\pi/3)]=0$, which alone does not detect the error. Differentiate at $t=0$: $a'(0)=-3\lambda/2$ and $\theta'(0)=\sqrt3\lambda/2$, giving $\widetilde p_1'(0)=0$. But the generator specifies $q_{12}=\lambda>0$, so the small-time expansion requires $p_{12}(t)=\lambda t+o(t)$. The incorrect sign assigns the clockwise rate to the other neighbor. Keeping the displacement convention $d=j-i$ and the phase $\theta-2\pi d/3$ avoids this reversal.

Check Your Understanding

Use the displacement convention and real-valued formula developed above. When checking a transition matrix, include its row sums.

For $\lambda=2$ and $t=\pi/(2\sqrt3)$, calculate $\theta$ and $a$, then write $p_0(t),p_1(t),p_2(t)$.
Starting from state $2$, which entry of $P(t)$ gives the probability of being at state $1$, and what is its cyclic displacement?
Explain why the complex-conjugate eigenvalue terms combine to give a real probability.
What fails in the long-time conclusion if $\lambda=0$? Give an initial distribution for which the limit is not uniform.
For $t=0$, substitute into the cosine formula and verify all three displacement probabilities.

Cyclic Three-State Chains and Complex Eigenvalues

What You'll Learn