What a Stationary Distribution Must Satisfy
Stationary-distribution calculations often fail for reasons that are easy to miss: the vector is multiplied on the wrong side, its entries are not normalized, or an invariant measure on an infinite state space is treated as a probability distribution. A separate issue is uniqueness. A valid stationary distribution need not be the only one, especially when the chain is reducible. The safest practice is to check the defining equation, the total mass, and the assumptions behind any uniqueness claim.
Throughout, let \((X_n)_{n\ge0}\) be a discrete-time Markov chain on a finite or countable state space \(S\), defined on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), with row-stochastic transition matrix \(P=(p_{ij})\). Thus \(p_{ij}=\mathbb{P}(X_{n+1}=j\mid X_n=i)\), and each row sums to \(1\). Distributions are row vectors: if the initial distribution is \(\mu_0\), then \(\mu_n=\mu_0P^n\).
In a countable state space, the sum \(\sum_i\nu_i\) may be infinite. An invariant measure with finite, strictly positive total mass can be normalized to a stationary distribution. An infinite measure cannot: dividing by its total mass is not an operation that produces a probability distribution.
Wrong Side of the Transition Matrix
With the row-vector convention, the stationarity equations are \(\pi_j=\sum_i\pi_i p_{ij}\) for each destination state \(j\), or equivalently \(\pi P=\pi\). The equation \(P\pi^{\mathsf T}=\pi^{\mathsf T}\) is not the same calculation. It treats the row-stochastic matrix as though it were column-stochastic.
Worked Example: A Column-Vector Calculation Gives the Wrong Answer
Take \(S=\{1,2\}\) and \(P=\begin{bmatrix}\tfrac34&\tfrac14\\[2pt]\tfrac12&\tfrac12\end{bmatrix}\). The first row sums to \(\tfrac34+\tfrac14=1\), and the second sums to \(\tfrac12+\tfrac12=1\). Find the stationary distribution using the row-vector convention.
Write \(\pi=(a,b)\). The first coordinate of \(\pi P=\pi\) gives \(a=\tfrac34a+\tfrac12b\). Rearranging gives \(\tfrac14a=\tfrac12b\), hence \(a=2b\). The normalization condition \(a+b=1\) then gives \(3b=1\), so \(b=\tfrac13\) and \(a=\tfrac23\).
Check: Multiplication using the stated convention gives \((\tfrac23,\tfrac13)P=(\tfrac12+\tfrac16,\tfrac16+\tfrac16)=(\tfrac23,\tfrac13)\). The entries are nonnegative and sum to \(1\).
The wrong calculation: If one instead solves \(P(a,b)^{\mathsf T}=(a,b)^{\mathsf T}\), the first equation is \(\tfrac34a+\tfrac14b=a\), which gives \(a=b\). Normalizing then gives the apparent answer \((\tfrac12,\tfrac12)\). But the row-vector check gives \((\tfrac12,\tfrac12)P=(\tfrac58,\tfrac38)\ne(\tfrac12,\tfrac12)\), so this is not stationary.
Invariance Is Not Normalization
The equation \(\nu P=\nu\) is homogeneous: if \(\nu\) is invariant, then \(c\nu\) is invariant for every \(c\ge0\). Therefore, solving only the stationarity equations determines relative weights, not necessarily probabilities. The normalization condition must also be imposed whenever the total mass is finite and positive.
Proof. Since \(Z\) is finite and positive, each \(\pi_i=\nu_i/Z\) is defined and nonnegative, and \(\sum_i\pi_i=(1/Z)\sum_i\nu_i=1\). For each \(j\), nonnegativity permits the sum defining the \(j\)th coordinate, and invariance gives \(\sum_i\pi_i p_{ij}=(1/Z)\sum_i\nu_i p_{ij}=\nu_j/Z=\pi_j\). Thus \(\pi P=\pi\) and \(\pi\) is a stationary distribution. \(\square\)
Worked Example: Relative Weights Need a Normalizing Constant
For the two-state matrix in the previous example, the invariance equation gave the ratio \(a:b=2:1\). If we stop at \((2,1)\), it is an invariant measure, not a stationary distribution: its total mass is \(2+1=3\).
Divide both entries by the total mass \(3\). This gives \(\pi=(\tfrac23,\tfrac13)\). The invariance check is \(\pi P=(\tfrac23\cdot\tfrac34+\tfrac13\cdot\tfrac12,\ \tfrac23\cdot\tfrac14+\tfrac13\cdot\tfrac12)=(\tfrac12+\tfrac16,\tfrac16+\tfrac16)=(\tfrac23,\tfrac13)\), and its total mass is \(\tfrac23+\tfrac13=1\).
Common mistake: Calling \((2,1)\) a stationary distribution because it satisfies the invariance equation. The correction is to distinguish the invariant measure from its normalized version and check both invariance and unit total mass.
One Finite-State Equation Is Redundant
In a finite state space, the stationary equations are not all independent. This does not mean that one may omit normalization or ignore the remaining equations. It means that once all but one coordinate equation hold for a normalized candidate, the last coordinate equation follows from the row sums of \(P\).
Proof. Because \(P\) is row-stochastic, \(\sum_j p_{ij}=1\) for each \(i\). Therefore \(\sum_j(xP)_j=\sum_j\sum_i x_i p_{ij}=\sum_i x_i\sum_jp_{ij}=\sum_i x_i\). This equals \(\sum_j x_j\), so \(\sum_j((xP)_j-x_j)=0\). If all but one of these differences are zero, the remaining difference must also be zero because their sum is zero. \(\square\)
Worked Example: Solving with a Redundant Coordinate Equation
Consider \(S=\{1,2,3\}\) and \(P=\begin{bmatrix}\tfrac12&\tfrac12&0\\0&\tfrac12&\tfrac12\\\tfrac12&0&\tfrac12\end{bmatrix}\). The row sums are respectively \(\tfrac12+\tfrac12+0=1\), \(0+\tfrac12+\tfrac12=1\), and \(\tfrac12+0+\tfrac12=1\). Let \(\pi=(a,b,c)\).
The first two coordinate equations of \(\pi P=\pi\) are \(a=\tfrac12a+\tfrac12c\) and \(b=\tfrac12a+\tfrac12b\). The first gives \(a=c\); the second gives \(b=a\). Thus \(a=b=c\). Normalization gives \(a+b+c=3a=1\), so \(\pi=(\tfrac13,\tfrac13,\tfrac13)\).
The third equation is \(c=\tfrac12b+\tfrac12c\), which also holds for these values. It was redundant: once the first two coordinates agree, the coordinate-sum result proves the third agrees. As a direct check, \((\tfrac13,\tfrac13,\tfrac13)P=(\tfrac16+\tfrac16,\tfrac16+\tfrac16,\tfrac16+\tfrac16)=(\tfrac13,\tfrac13,\tfrac13)\).
Common mistake: Treating the redundant equation as independent and concluding that the system is inconsistent if arithmetic errors make it appear not to hold. The correct response is to substitute the proposed vector into the full equation; a genuine solution must satisfy every coordinate, even though one equation is algebraically implied by the others.
An Infinite Invariant Measure May Not Normalize
On a countable state space, an invariant measure can have infinite total mass. The existence of such a measure alone does not establish the existence of a stationary distribution. The two-sided simple random walk on the integers gives a direct counterexample.
Worked Example: Counting Measure on the Integers
Let \(S=\mathbb{Z}\), and from each state \(i\) move to \(i-1\) or \(i+1\), each with probability \(\tfrac12\). Each transition row has exactly these two nonzero entries, and their sum is \(\tfrac12+\tfrac12=1\). The counting measure \(\nu_i=1\) is invariant because for every \(j\), \(\sum_i\nu_i p_{ij}=p_{j-1,j}+p_{j+1,j}=\tfrac12+\tfrac12=1=\nu_j\).
But its total mass is \(\sum_{i\in\mathbb{Z}}1=\infty\), so dividing by that total cannot produce a probability distribution. In fact, no stationary distribution exists for this chain. If \(\pi\) were one, its stationarity equations would give \(\pi_j=\tfrac12\pi_{j-1}+\tfrac12\pi_{j+1}\), or \(\pi_{j+1}-\pi_j=\pi_j-\pi_{j-1}\) for every integer \(j\). Thus the successive differences are all one constant \(d\), and \(\pi_j=\pi_0+jd\) for all integers \(j\). If \(d>0\), these values are negative for sufficiently negative \(j\); if \(d<0\), they are negative for sufficiently positive \(j\). Nonnegativity therefore forces \(d=0\), making \(\pi_j=\pi_0\) for every \(j\). A nonnegative constant sequence on \(\mathbb{Z}\) has finite sum only if \(\pi_0=0\), but then its sum is \(0\), not \(1\). This contradiction rules out a stationary distribution.
Check: The invariant measure has value \(1\) at every state, while a probability distribution would need total mass exactly \(1\). Invariance is satisfied; normalization is not available.
Stationarity Does Not Guarantee Uniqueness
A stationary distribution is unique under appropriate conditions, including irreducibility and positive recurrence for a countable-state chain. Uniqueness should not be assumed for an arbitrary chain. In a reducible chain, distinct closed classes can support different stationary distributions, and their mixtures can also be stationary.
Worked Example: A Chain with Many Stationary Distributions
Let \(S=\{1,2\}\) and \(P=\begin{bmatrix}1&0\\0&1\end{bmatrix}\). Each row sums to \(1\). Both states are absorbing. For any \(a\in[0,1]\), let \(\pi=(a,1-a)\). Then \(\pi P=(a\cdot1+(1-a)\cdot0,\ a\cdot0+(1-a)\cdot1)=(a,1-a)=\pi\), and the entries are nonnegative and sum to \(a+(1-a)=1\). Thus every such \(\pi\) is stationary.
For example, \((1,0)\) and \((0,1)\) are distinct stationary distributions, as is \((\tfrac12,\tfrac12)\). Therefore this chain does not have a unique stationary distribution. The failure is not a normalization error: each listed vector is correctly normalized. The issue is reducibility.
When using a uniqueness result, check its hypotheses rather than inferring uniqueness from the fact that one solution has been found. The earlier results on irreducible positive recurrent chains give a standard uniqueness theorem under those assumptions. They do not imply uniqueness for a reducible chain such as the two absorbing states above. Also distinguish uniqueness from convergence: a stationary distribution solves \(\pi P=\pi\); convergence of \(\mu_0P^n\) to it is a separate question and can require additional conditions.
A Practical Error Check
Before accepting a stationary-distribution calculation, check the following items in order:
- Convention: Confirm that \(P\) is row-stochastic and that the candidate is a row vector satisfying \(\pi P=\pi\).
- Normalization: Check \(\pi_i\ge0\) for every state and \(\sum_i\pi_i=1\). If the calculation first gives invariant weights, divide by their finite, positive total.
- All coordinates: Substitute into every coordinate equation. In a finite chain one equation is redundant, but the final candidate must still satisfy it.
- State-space size: On a countable space, check that the proposed weights have a finite total before normalizing them.
- Uniqueness: Verify the hypotheses of the uniqueness result. If the chain is reducible, look for multiple closed classes and test whether different mixtures are stationary.
These checks diagnose different errors. A vector can satisfy invariance but fail normalization; a normalized vector can use the wrong multiplication convention; and a valid stationary distribution can coexist with other stationary distributions. Keeping the defining conditions separate makes each failure visible.
Check Your Understanding
For each question, state which condition is being checked: invariance, normalization, finite total mass, or uniqueness.
- For a row-stochastic matrix \(P\), what equation must a stationary row vector satisfy, and why is a right eigenvector calculation not interchangeable with it?
- An invariant vector on a finite state space is \((3,1,2)\). Normalize it and state the two checks the resulting vector must pass.
- Why does the sum of the coordinate differences between \(xP\) and \(x\) equal zero for a finite row-stochastic matrix?
- For the simple random walk on \(\mathbb{Z}\), why can the invariant counting measure not be normalized? What additional argument rules out a different stationary probability distribution?
- For the identity transition matrix on two states, give two distinct stationary distributions and explain what this shows about an unqualified uniqueness claim.