Exceptional Parameter Cases in Two-State Chains

Why These Parameter Values Need Separate Attention

For a two-state chain, the transition matrix can be written using the switching probabilities \(a\) and \(b\). A compact formula for its powers involves the sum \(a+b\) in a denominator and the quantity \(1-a-b\) raised to the \(n\)th power. Three values of that sum have especially simple behavior: zero, one, and two. The first makes the chain stay where it is, the second makes the two rows of the matrix identical, and the third forces the chain to alternate deterministically.

Let \((X_n)_{n\ge0}\) be a discrete-time Markov chain on the state space \(S=\{0,1\}\), defined on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). At each step, the chain switches from state \(0\) to state \(1\) with probability \(a\), and from state \(1\) to state \(0\) with probability \(b\), where \(0\le a,b\le1\). Thus its row-stochastic transition matrix is

Each row sums to one: \((1-a)+a=1\) and \(b+(1-b)=1\). Write \(P^{(n)}=P^n\), with \(P^{(0)}=I\), and let \(p_{ij}^{(n)}=\mathbb{P}(X_n=j\mid X_0=i)\) when the conditioning event has positive probability. The Markov transition probabilities define these entries for either starting state. Set \(s=a+b\). Because \(a,b\) are probabilities, \(0\le s\le2\).

A Recursion That Reveals the Cases

The Chapman–Kolmogorov equations give a short recurrence for the probability of being in the other state after \(n\) steps. Define \(u_n=p_{01}^{(n)}\) and \(v_n=p_{10}^{(n)}\). Since each row of \(P^{(n)}\) sums to one, \(p_{00}^{(n)}=1-u_n\) and \(p_{11}^{(n)}=1-v_n\). Conditioning on the state at time \(n\) gives

and, in the same way,

These recurrences apply for every \(a,b\in[0,1]\), including the exceptional values. They are useful here because they avoid dividing by \(s\), which would not be allowed when \(s=0\).

Proof. If \(s=0\), the restrictions \(a,b\ge0\) and \(a+b=0\) force \(a=b=0\). The matrix is therefore \(I\), whose every nonnegative integer power is \(I\). Equivalently, the recurrences give \(u_{n+1}=u_n\) and \(v_{n+1}=v_n\), with both initial values zero. If \(s=1\), then \(b=1-a\), so the first row of \(P\) is \((b,a)\), and the second row \((b,1-b)\) is also \((b,a)\). Multiplying either row \((b,a)\) by \(P\) gives \((b^2+ab,ab+a^2)=(b,a)\), because \(a+b=1\). Thus \(P^2=P\); repeated multiplication gives \(P^{(n)}=P\) for every \(n\ge1\), while \(P^{(0)}=I\) by definition. The recurrences show the same result: for \(n\ge0\), \(u_{n+1}=a\) and \(v_{n+1}=b\). Finally, if \(s=2\), the bounds \(a,b\le1\) force \(a=b=1\), so \(P=\begin{bmatrix}0&1\\1&0\end{bmatrix}\) and \(P^2=I\). Pairing factors in the product \(P^n\) gives \(I\) for even \(n\) and \(P\) for odd \(n\). In the recurrences, \(u_{n+1}=1-u_n\) and \(v_{n+1}=1-v_n\), starting from zero, so both probabilities alternate between zero and one. These matrix powers give all four entries stated in the theorem. \(\square\)

The case \(s=1\) is not a claim that the chain is in the same state after every positive number of steps. Rather, each row of \(P^{(n)}\), for \(n\ge1\), is the same probability distribution \((b,a)\). The starting state no longer affects the distribution after one step. In the case \(s=2\), by contrast, the state at every time is determined by the starting state and the parity of the number of steps.

Worked Examples

How the Exceptional Cases Fit the General Formula

For \(s>0\), the two-state \(n\)-step formula has the factor \(1-s\). In the notation used here, for \(n\ge1\) it is

When \(s=1\), the denominator is \(1\) and \((1-s)^n=0\) for \(n\ge1\); the formula reduces to rows \((b,a)\). When \(s=2\), \(a=b=1\), and \((1-s)^n=(-1)^n\); for even \(n\) the matrix is \(I\), while for odd \(n\) it is \(P\). When \(s=0\), this expression is not defined because it divides by \(s\). That case must instead be evaluated from the original transition matrix or the recurrences. Also, the formula displayed here is for \(n\ge1\); at \(n=0\), use \(P^{(0)}=I\).

Common mistake. Substituting \(s=0\) directly into the general formula and treating \(b/s\) as a valid probability is an invalid calculation: \(b/s=0/0\) in this case. The correction is to use \(a=b=0\), which gives \(P=I\) and therefore \(P^{(n)}=I\) for all \(n\ge0\). A second related error is to say that \(P^{(n)}=P\) for every \(n\) when \(s=1\). This misses \(n=0\): \(P^{(0)}=I\), and the identical-row result begins at \(n=1\).

These cases are useful checks when computing powers of a two-state matrix. The identity case tests whether zero switching has been handled without division by zero. The identical-row case tests whether the initial state’s influence disappears after one step. The alternating case tests whether parity has been preserved. In each case, checking \(n=0\), one positive step, and the relevant matrix product helps prevent a formula from being applied outside its stated conditions.