Two-State Stationary Distributions

The General Two-State Chain

For a chain with only two states, stationarity reduces to one balance equation and a normalization condition. Solving this small system carefully is useful beyond the two-state case: it shows how probability flowing out of one state must be balanced by probability flowing in, and it exposes boundary cases that can be hidden by formulas involving division.

Let \((X_n)_{n\ge0}\) be a discrete-time Markov chain on the probability space \((\Omega,\mathcal{F},\mathbb{P})\), with state space \(S=\{1,2\}\) and transition matrix \(P=(p_{ij})\). Use the parameters \(a=p_{12}\) and \(b=p_{21}\). Since each row of a transition matrix sums to \(1\), the matrix has the form

Here \(a\) is the probability of moving from state \(1\) to state \(2\), and \(b\) is the probability of moving from state \(2\) to state \(1\). A distribution is a row vector \(\mu=(\mu_1,\mu_2)\) with nonnegative entries summing to \(1\); its one-step update is \(\mu P\).

Write a candidate probability vector as \(\pi=(x,1-x)\), where \(0\le x\le1\). The stationarity equations are

These two equations express the same constraint. The first coordinate after a transition consists of the mass that stays in state \(1\), plus the mass that enters state \(1\) from state \(2\). Rearranging the first equation gives \(ax=b(1-x)\). This is the balance of probability flow between the states.

Proof. Let \(\pi=(x,1-x)\) be any stationary distribution. Its first coordinate must satisfy \(x=x(1-a)+(1-x)b\). Subtracting \(x(1-a)\) from both sides gives \(ax=b(1-x)\), or \((a+b)x=b\). If \(a+b>0\), division gives the only possible value \(x=b/(a+b)\), so \(1-x=a/(a+b)\). Both entries are nonnegative because \(a,b\ge0\), and their sum is \((a+b)/(a+b)=1\). To check stationarity, the first coordinate of \(\pi P\) is \[ \frac{b}{a+b}(1-a)+\frac{a}{a+b}b =\frac{b-ab+ab}{a+b} =\frac{b}{a+b}. \] The entries of \(\pi P\) sum to \(1\), because each row of \(P\) sums to \(1\); hence its second coordinate is \(a/(a+b)\). Thus the candidate is stationary and is the only possibility. If \(a+b=0\), nonnegativity implies \(a=b=0\), so \(P\) is the identity matrix and \(\mu P=\mu\) for every probability vector \(\mu\). This proves both cases. \(\square\)

The theorem gives an exhaustive answer, including the endpoints \(a=0\) or \(b=0\). A useful equivalent formulation is that a probability vector is stationary exactly when the flow from state \(1\) to state \(2\) equals the flow from state \(2\) to state \(1\).

Proof. If \(\pi P=\pi\), its first coordinate satisfies \(\pi_1=\pi_1(1-a)+\pi_2b\). Subtracting \(\pi_1(1-a)\) gives \(\pi_1a=\pi_2b\). Conversely, suppose \(\pi_1+\pi_2=1\) and \(\pi_1a=\pi_2b\). The first coordinate of \(\pi P\) is \(\pi_1(1-a)+\pi_2b=\pi_1(1-a)+\pi_1a=\pi_1\). The two coordinates of \(\pi P\) sum to \(\pi_1+\pi_2=1\), so its second coordinate is \(1-\pi_1=\pi_2\). Therefore \(\pi P=\pi\). \(\square\)

Worked Cases: Unique and Nonunique Stationarity

First check the rows of the matrix, then solve the equation and substitute the resulting vector back into \(\pi P=\pi\). This order is especially important at parameter values where a transition probability is zero.

Periodicity Does Not Remove Stationarity

A stationary distribution need not describe the limit of the distribution at time \(n\). A simple way to see the distinction is the irreducible two-state chain that always switches states. For this matrix, \(a=b=1\); it has a unique stationary distribution, but a chain started in state \(1\) alternates deterministically between the states.

Proof. When \(a=b=1\), every transition switches states. Starting from either state, a return is possible after \(n\) steps exactly when \(n\) is even, so the set of possible positive return times is \(\{2,4,6,\ldots\}\), whose greatest common divisor is \(2\). Now suppose \(a,b>0\) and they are not both \(1\). If \(a<1\), then \(p_{11}=1-a>0\), so state \(1\) has a return in one step and its period is \(1\). If \(a=1\), then \(b<1\) and \(b>0\). From state \(1\), a two-step return has probability \(1\cdot b=b>0\), and a three-step return along \(1\to2\to2\to1\) has probability \(1\cdot(1-b)\cdot b>0\). The greatest common divisor of \(2\) and \(3\) is \(1\), so state \(1\) has period \(1\). In either case, irreducibility means the two states communicate, and their return periods agree; thus the chain has period \(1\). \(\square\)

A Common Mistake and a Practical Summary

A frequent error is to write only the mass that stays in state \(1\) when computing the next probability of being in state \(1\), omitting the incoming mass from state \(2\). In the first worked matrix, this incorrect calculation would be \(\pi_1=\tfrac23\pi_1\), which forces \(\pi_1=0\); normalization would then suggest the wrong vector \((0,1)\). Substitution exposes the error: \((0,1)P=(\tfrac12,\tfrac12)\), not \((0,1)\). The correct first-coordinate equation includes both sources of mass, \(\pi_1=\tfrac23\pi_1+\tfrac12\pi_2\), or equivalently \(\pi_1a=\pi_2b\).

For any two-state matrix, the parameter test \(a+b>0\) settles uniqueness without needing to assume irreducibility or aperiodicity. If both transition probabilities are positive, the stationary distribution puts positive mass on both states. If exactly one is zero, the unique stationary distribution is concentrated on the absorbing state. If both are zero, each state is absorbing and every initial distribution is stationary. Periodicity affects the behavior of distributions over time, not the stationarity equation itself.