From Invariance to State-by-State Balance
A stationary distribution is unchanged by one transition of a Markov chain. That invariance is a vector equation, but its meaning becomes more concrete when written one state at a time: the stationary probability at a state equals the total stationary probability flowing into it. The individual terms record both where the chain starts and the chance that it moves to the chosen destination.
Let \((\Omega,\mathcal{F},\mathbb{P})\) be a probability space, and let \((X_n)_{n\ge 0}\) be a discrete-time Markov chain on a finite or countable state space \(S\), with transition matrix \(P=(p_{ij})\). Here \(p_{ij}=\mathbb{P}(X_{n+1}=j\mid X_n=i)\), using a transition kernel defined for every \(i\in S\). Each row is a probability distribution: \(p_{ij}\ge 0\) and \(\sum_{j\in S}p_{ij}=1\). Use the natural filtration \(\mathcal{F}_n=\sigma(X_0,\ldots,X_n)\). A probability distribution is written as a row vector \(\pi=(\pi_i)_{i\in S}\), with \(\pi_i\ge 0\) and \(\sum_i\pi_i=1\).
For a countable state space, matrix multiplication in this definition is understood coordinate by coordinate: the mass at destination \(j\) after one step is \(\sum_{i\in S}\pi_i p_{ij}\). The sum is well-defined because its terms are nonnegative and its value is at most \(\sum_i\pi_i=1\).
Proof. Assume first that \(\pi P=\pi\). By the definition of row-vector matrix multiplication, the \(j\)th coordinate of \(\pi P\) is \(\sum_{i\in S}\pi_i p_{ij}\). Equality of the \(j\)th coordinates of \(\pi P\) and \(\pi\) therefore gives \(\sum_i\pi_i p_{ij}=\pi_j\) for every \(j\). Conversely, assume those coordinate equalities hold for every \(j\). They state that every coordinate of \(\pi P\) equals the corresponding coordinate of \(\pi\), so \(\pi P=\pi\). The given \(\pi\) is a probability distribution, so this equality is exactly stationarity. \(\square\)
Reading the Terms as Probability Flow
Suppose the chain starts with distribution \(\pi\). At time \(0\), the probability of being at \(i\) is \(\pi_i\). Given state \(i\), the probability of moving to \(j\) in one step is \(p_{ij}\). Thus the probability of starting at \(i\) and moving to \(j\) is \(\pi_i p_{ij}\). These disjoint possibilities over all starting states \(i\) account for every way to be at \(j\) one step later. Their sum is the probability of being at \(j\) after the transition.
The balance equation is not a claim that the flow from each individual state \(i\) into \(j\) equals \(\pi_j\). It is the sum over all origins. It includes the possibility of staying at \(j\), whose contribution is \(\pi_jp_{jj}\), as well as moves from other states. A self-transition is part of the incoming flow, not an extra term to add after the sum.
Worked Example: Checking Balance in a Two-State Chain
Consider \(S=\{1,2\}\) and $$ P=\begin{bmatrix} \tfrac34&\tfrac14\\ \tfrac13&\tfrac23 \end{bmatrix}. $$ The first row sums to \(\tfrac34+\tfrac14=1\), and the second row sums to \(\tfrac13+\tfrac23=1\). Check the proposed distribution \(\pi=(\tfrac47,\tfrac37)\) using the coordinate balance equations.
For destination state \(1\), the incoming contributions are from states \(1\) and \(2\): $$ \sum_{i\in S}\pi_i p_{i1} =\tfrac47\cdot\tfrac34+\tfrac37\cdot\tfrac13 =\tfrac37+\tfrac17 =\tfrac47 =\pi_1. $$ For destination state \(2\), they are $$ \sum_{i\in S}\pi_i p_{i2} =\tfrac47\cdot\tfrac14+\tfrac37\cdot\tfrac23 =\tfrac17+\tfrac27 =\tfrac37 =\pi_2. $$ The entries of \(\pi\) are nonnegative and sum to \(\tfrac47+\tfrac37=1\). Both coordinate equations hold, so the theorem shows that \(\pi\) is stationary.
Check by row-vector multiplication: $$ \pi P =\left(\tfrac47\cdot\tfrac34+\tfrac37\cdot\tfrac13,\ \tfrac47\cdot\tfrac14+\tfrac37\cdot\tfrac23\right) =\left(\tfrac47,\tfrac37\right) =\pi. $$ This independently confirms the coordinate calculation.
The equation for state \(j\) is sometimes called the global balance equation for \(j\): contributions from all states are combined. The word “global” here refers to the sum over possible origins, not to a condition that each pair of states balances separately. In general, stationarity does not require \(\pi_i p_{ij}=\pi_j p_{ji}\) for every pair \(i,j\); those pairwise equalities are stronger than the coordinate equations.
Worked Example: A Cycle with Three States
Let \(S=\{1,2,3\}\), and suppose the chain moves deterministically from \(1\) to \(2\), from \(2\) to \(3\), and from \(3\) to \(1\): $$ P=\begin{bmatrix} 0&1&0\\ 0&0&1\\ 1&0&0 \end{bmatrix}. $$ Each row sums to \(1\): the row sums are \(0+1+0=1\), \(0+0+1=1\), and \(1+0+0=1\). Test \(\pi=(\tfrac13,\tfrac13,\tfrac13)\).
For destination \(1\), only state \(3\) sends positive transition probability to \(1\), so $$ \sum_i\pi_i p_{i1} =\tfrac13\cdot0+\tfrac13\cdot0+\tfrac13\cdot1 =\tfrac13. $$ For destination \(2\), $$ \sum_i\pi_i p_{i2} =\tfrac13\cdot1+\tfrac13\cdot0+\tfrac13\cdot0 =\tfrac13. $$ For destination \(3\), $$ \sum_i\pi_i p_{i3} =\tfrac13\cdot0+\tfrac13\cdot1+\tfrac13\cdot0 =\tfrac13. $$ Each result equals the corresponding coordinate of \(\pi\); also \(\tfrac13+\tfrac13+\tfrac13=1\). Thus \(\pi\) is stationary.
Check: Multiplying gives \(\pi P=(\tfrac13,\tfrac13,\tfrac13)\): the first coordinate comes from state \(3\), the second from state \(1\), and the third from state \(2\). The chain moves at every step, but this distribution is unchanged because its mass is evenly distributed among the three states.
Balance Across a Set of States
State-by-state equations also imply a useful aggregate interpretation. For a subset \(A\subseteq S\), write \(\pi(A)=\sum_{j\in A}\pi_j\), and write \(P(i,A)=\sum_{j\in A}p_{ij}\) for the chance of entering \(A\) from \(i\) in one step. Under stationarity, the total flow from \(A\) to its complement \(A^c=S\setminus A\) equals the total flow from \(A^c\) into \(A\).
Proof. Sum the coordinate equations \(\pi_j=\sum_i\pi_i p_{ij}\) over \(j\in A\). All terms are nonnegative, so the order of the countable sums can be interchanged. This gives $$ \pi(A)=\sum_{i\in S}\pi_i P(i,A) =\sum_{i\in A}\pi_i P(i,A)+\sum_{i\in A^c}\pi_i P(i,A). $$ For each \(i\), row stochasticity gives \(P(i,A)+P(i,A^c)=1\). Therefore $$ \pi(A)=\sum_{i\in A}\pi_i-\sum_{i\in A}\pi_iP(i,A^c) +\sum_{i\in A^c}\pi_iP(i,A). $$ Subtracting \(\sum_{i\in A}\pi_i\) from both sides yields the stated equality. The sums are finite in value: each is bounded by \(\sum_i\pi_i=1\). If \(A\) is empty or \(A=S\), both crossing flows are zero, so the equality also covers these cases. \(\square\)
Worked Example: Checking Flow Across a Boundary
Consider \(S=\{1,2,3\}\) with $$ P=\begin{bmatrix} \tfrac12&\tfrac12&0\\ \tfrac14&\tfrac12&\tfrac14\\ 0&\tfrac13&\tfrac23 \end{bmatrix}, \qquad \pi=\left(\tfrac29,\tfrac49,\tfrac13\right). $$ The rows sum to \(\tfrac12+\tfrac12+0=1\), \(\tfrac14+\tfrac12+\tfrac14=1\), and \(0+\tfrac13+\tfrac23=1\). The proposed masses are nonnegative and sum to \(\tfrac29+\tfrac49+\tfrac39=1\).
First check all three state equations. For destination \(1\), $$ \tfrac29\cdot\tfrac12+\tfrac49\cdot\tfrac14+\tfrac13\cdot0 =\tfrac19+\tfrac19 =\tfrac29. $$ For destination \(2\), $$ \tfrac29\cdot\tfrac12+\tfrac49\cdot\tfrac12+\tfrac13\cdot\tfrac13 =\tfrac19+\tfrac29+\tfrac19 =\tfrac49. $$ For destination \(3\), $$ \tfrac29\cdot0+\tfrac49\cdot\tfrac14+\tfrac13\cdot\tfrac23 =\tfrac19+\tfrac29 =\tfrac13. $$ Thus \(\pi\) is stationary.
Now take \(A=\{1,2\}\), so \(A^c=\{3\}\). The only flow from \(A\) to \(A^c\) is the transition from \(2\) to \(3\), and its stationary mass is $$ \sum_{i\in A}\pi_iP(i,A^c) =\tfrac29\cdot0+\tfrac49\cdot\tfrac14 =\tfrac19. $$ The flow from \(A^c\) into \(A\) is $$ \sum_{i\in A^c}\pi_iP(i,A) =\tfrac13\left(0+\tfrac13\right) =\tfrac19. $$ The two directions agree, as the proposition requires. Notice that this conclusion compares total crossing flows; it does not say that each transition in one direction has a matching reverse transition.
Common Mistakes and What the Equations Do Not Say
Common mistake: using the wrong matrix index. In the two-state example, the incoming mass at state \(1\) is obtained from column \(1\), so its equation is \(\pi_1=\pi_1p_{11}+\pi_2p_{21}\). A tempting but incorrect calculation uses \(p_{12}\) and \(p_{22}\), the entries in column \(2\), and calls them flow into state \(1\). For the displayed candidate, that gives $$ \tfrac47\cdot\tfrac14+\tfrac37\cdot\tfrac23 =\tfrac17+\tfrac27 =\tfrac37, $$ which is not \(\pi_1=\tfrac47\). The correction is to fix the destination \(j\) and sum down column \(j\): the correct calculation is \(\tfrac47\cdot\tfrac34+\tfrac37\cdot\tfrac13=\tfrac47\). With row-stochastic matrices, the first index is the origin and the second index is the destination.
Another mistake is to treat balance as a statement that nothing changes along a sample path. Stationarity concerns the distribution at each time, not whether \(X_{n+1}=X_n\). In the three-state cycle, the chain changes state with probability \(1\) at every step, yet the uniform distribution is stationary. Nor do the balance equations alone assert that a stationary probability distribution exists or is unique. They say exactly how to verify invariance for a proposed probability distribution; existence and uniqueness require separate arguments.
Check Your Understanding
For each question, identify the destination before summing incoming contributions.
- For a row-stochastic matrix, which index of \(p_{ij}\) denotes the destination state?
- Write the coordinate balance equation for state \(j\) in a finite state space.
- Why must the term \(\pi_jp_{jj}\) be included in the balance equation for state \(j\)?
- For the three-state deterministic cycle, which origin state contributes positive flow into state \(2\), and how much flow does it contribute under the uniform distribution?
- For the subset proposition, what are the two directions of flow being compared?
- Does verifying \(\pi P=\pi\) for one probability distribution prove that it is the unique stationary distribution? Explain what the verification establishes.