Stationary Distributions and Invariant Measures

Stationarity as a Fixed-Point Equation

A stationary distribution describes a probability law that is unchanged by one transition of a Markov chain. On a finite state space, it is a probability vector solving a finite system of equations. On a countably infinite state space, there is a related but broader concept: a nonnegative invariant measure may be unchanged by transitions even though its total mass is infinite, so it cannot be normalized into a probability distribution.

Let \((X_n)_{n\ge 0}\) be a discrete-time Markov chain on a finite or countable state space \(S\), defined on a probability space \((\Omega,\mathcal F,\mathbb P)\). Write \(\mathcal F_n=\sigma(X_0,\ldots,X_n)\). The transition matrix \(P=(p_{ij})_{i,j\in S}\) is part of the chain's specification: its entries are nonnegative, \(\sum_{j\in S}p_{ij}=1\) for every \(i\), and the Markov property is

\mathbb P(X_{n+1}=j\mid\mathcal F_n)=p_{X_nj} \quad\text{almost surely, for every }n\ge0,\ j\in S.

This formulation specifies transition probabilities even at states that the chain, under its chosen initial law, visits with probability zero. A conditional probability given the event \(X_n=i\) determines \(p_{ij}\) when that event has positive probability, but need not determine it when the event has probability zero. The specified transition matrix supplies the values in those cases as well.

Definition: A stationary distribution for \(P\) is a row vector \(\pi=(\pi_i)_{i\in S}\) such that \(\pi_i\ge0\), \(\sum_{i\in S}\pi_i=1\), and \(\pi P=\pi\). Componentwise, the fixed-point equation is \(\pi_j=\sum_{i\in S}\pi_i p_{ij}\) for every \(j\in S\).

The row-vector convention matters: distributions are rows, and one transition sends a row distribution \(\mu\) to \(\mu P\). For countable \(S\), each sum defining \((\pi P)_j\) is a sum of nonnegative terms. The equation says that the probability mass arriving at \(j\), summed over all possible starting states \(i\), equals the mass \(\pi_j\) assigned to \(j\).

Definition: An invariant measure for \(P\) is a family \(\nu=(\nu_i)_{i\in S}\) with finite nonnegative entries such that \(\nu_j=\sum_{i\in S}\nu_i p_{ij}\) for every \(j\in S\). Its total mass \(\sum_{i\in S}\nu_i\) may be finite or infinite. A nonzero invariant measure has at least one positive entry.

Thus every stationary distribution is an invariant measure of total mass one. An invariant measure of infinite total mass is not a stationary distribution: it does not assign probabilities summing to one. For a countable state space it can still be useful because its entries satisfy the same balance equation. Neither the word “stationary” nor the fixed-point equation alone asserts that a chain started from an arbitrary initial law converges to this distribution.

Two Basic Facts About Invariant Measures

Proposition: If \(\nu\) is an invariant measure with total mass \(M\) satisfying \(0<M<\infty\), then \(\pi_i=\nu_i/M\) is a stationary distribution. Conversely, every stationary distribution is an invariant measure with total mass one.

Proof. Since \(M\) is finite and positive, each \(\pi_i=\nu_i/M\) is nonnegative and \(\sum_i\pi_i=(1/M)\sum_i\nu_i=1\). For each \(j\), invariance of \(\nu\) gives

(\pi P)_j=\sum_{i\in S}\frac{\nu_i}{M}p_{ij} =\frac{1}{M}\sum_{i\in S}\nu_i p_{ij} =\frac{\nu_j}{M} =\pi_j.

So \(\pi\) is a stationary distribution. Conversely, if \(\pi\) is a stationary distribution, its entries are finite and nonnegative, and its equation \(\pi P=\pi\) is exactly the invariant-measure equation. Its total mass is one by definition. \(\square\)

Proposition: If \(\nu\) and \(\eta\) are invariant measures and \(a,b\ge0\) are finite constants, then \(a\nu+b\eta\) is an invariant measure, provided each entry of \(a\nu+b\eta\) is finite.

Proof. The new entries are nonnegative, and the stated condition makes them finite. For each \(j\), nonnegativity permits adding the sums term by term, so

\sum_{i\in S}(a\nu_i+b\eta_i)p_{ij} =a\sum_{i\in S}\nu_i p_{ij}+b\sum_{i\in S}\eta_i p_{ij} =a\nu_j+b\eta_j.

This is the invariant-measure equation for \(a\nu+b\eta\). The zero measure is included by taking \(a=b=0\); it is invariant but is not a probability distribution. \(\square\)

The first proposition explains exactly when normalization works. A finite, positive total mass can be divided out without changing the fixed-point equation. Infinite mass cannot be divided by its total to produce probabilities, and zero total mass gives the zero measure rather than a probability law.

Worked Example: A Finite Chain with an Explicit Stationary Distribution

Let \(S=\{1,2,3\}\) and specify the transition matrix with every row equal to \(q=(\tfrac12,\tfrac13,\tfrac16)\):

P= \begin{bmatrix} \tfrac12&\tfrac13&\tfrac16\\ \tfrac12&\tfrac13&\tfrac16\\ \tfrac12&\tfrac13&\tfrac16 \end{bmatrix}.

Each row sums to \(\tfrac12+\tfrac13+\tfrac16=\tfrac36+\tfrac26+\tfrac16=1\), so the specified matrix is row-stochastic. To check that \(q\) is stationary, multiply it as a row vector by \(P\). The first coordinate is \(\tfrac12\cdot\tfrac12+\tfrac13\cdot\tfrac12+\tfrac16\cdot\tfrac12=\tfrac14+\tfrac16+\tfrac1{12}=\tfrac12\). The second is \(\tfrac12\cdot\tfrac13+\tfrac13\cdot\tfrac13+\tfrac16\cdot\tfrac13=\tfrac16+\tfrac19+\tfrac1{18}=\tfrac13\). The third is \(\tfrac12\cdot\tfrac16+\tfrac13\cdot\tfrac16+\tfrac16\cdot\tfrac16=\tfrac1{12}+\tfrac1{18}+\tfrac1{36}=\tfrac16\). Hence \(qP=q\). Its entries sum to one, so it is a stationary distribution.

For a further check, the invariant measure \(\nu=(3,2,1)=6q\) has total mass \(3+2+1=6\). Direct multiplication gives first coordinate \((3+2+1)\tfrac12=3\), second coordinate \((3+2+1)\tfrac13=2\), and third coordinate \((3+2+1)\tfrac16=1\). Thus \(\nu P=\nu\), and normalizing by six returns \(q\).

Invariant Measures on Countable State Spaces

On an infinite state space, the coordinate equations still make sense, but a nonnegative solution need not have finite total mass. The following examples show both why such measures are useful and why they must not be confused with probability distributions.

Worked Example: A Reflecting Walk with Infinite Invariant Mass

Let \(S=\{0,1,2,\ldots\}\). At state \(0\), the chain moves to \(1\) with probability one. At each \(i\ge1\), it moves to \(i-1\) and \(i+1\), each with probability \(\tfrac12\). These rules specify the kernel at every state. The row at \(0\) sums to \(1\), and every other row sums to \(\tfrac12+\tfrac12=1\).

Define \(\nu_0=\tfrac12\) and \(\nu_i=1\) for every \(i\ge1\). At state \(0\), the incoming mass is \(\nu_1p_{10}=1\cdot\tfrac12=\tfrac12=\nu_0\). At state \(1\), it is \(\nu_0p_{01}+\nu_2p_{21}=\tfrac12\cdot1+1\cdot\tfrac12=1=\nu_1\). For \(j\ge2\), the incoming mass is \(\nu_{j-1}p_{j-1,j}+\nu_{j+1}p_{j+1,j}=1\cdot\tfrac12+1\cdot\tfrac12=1=\nu_j\). Thus every coordinate satisfies the invariant equation.

But \(\sum_{i\ge0}\nu_i=\tfrac12+\sum_{i\ge1}1=\infty\). Therefore \(\nu\) is an invariant measure, not a stationary distribution. In fact, this chain has no stationary distribution. If \(\pi\) were one, its equation at state \(0\) would give \(\pi_0=\tfrac12\pi_1\), so \(\pi_1=2\pi_0\). At state \(1\), \(\pi_1=\pi_0+\tfrac12\pi_2\), which then implies \(\pi_2=2\pi_0=\pi_1\). For \(j\ge2\), the equation is \(\pi_j=\tfrac12\pi_{j-1}+\tfrac12\pi_{j+1}\), or \(\pi_{j+1}-\pi_j=\pi_j-\pi_{j-1}\). Since \(\pi_2-\pi_1=0\), induction gives \(\pi_j=2\pi_0\) for every \(j\ge1\). If \(\pi_0>0\), these entries have infinite sum; if \(\pi_0=0\), every entry is zero. Neither possibility has total mass one.

Worked Example: A Shift on the Integers

Let \(S=\mathbb Z\) and let the chain move from each \(i\) to \(i+1\) with probability one. Each transition row has exactly one nonzero entry, equal to one, so every row sums to one. Set \(\nu_i=1\) for every integer \(i\). For a fixed \(j\), the only state that can move into \(j\) is \(j-1\); consequently \(\sum_{i\in\mathbb Z}\nu_i p_{ij}=\nu_{j-1}\cdot1=1=\nu_j\). This is an invariant measure, and its total mass \(\sum_{i\in\mathbb Z}1\) is infinite.

There is no stationary distribution for this chain. If \(\pi\) were one, the equation at each \(j\) would be \(\pi_j=\pi_{j-1}\). Repeated application shows that all coordinates equal one constant \(c\ge0\). If \(c>0\), summing over the infinitely many integers gives infinite total mass; if \(c=0\), the total mass is zero. Neither case sums to one. This example makes the distinction especially direct: the invariant equation can hold at every state while no probability distribution satisfies it.

Checking the Equation and Avoiding a Direction Error

For a candidate stationary distribution on a finite state space, check two separate requirements: the entries must be nonnegative and sum to one, and the row-vector product must satisfy \(\pi P=\pi\). For an invariant measure, replace the normalization requirement by nonnegative finite entries and check the same coordinate equations. On a countable space, verify each fixed coordinate using its incoming transitions, and state explicitly whether the total mass is finite.

A common mistake is to multiply a row-stochastic matrix by a distribution as if the distribution were a column vector. For example, take

P= \begin{bmatrix} \tfrac12&\tfrac12\\ \tfrac14&\tfrac34 \end{bmatrix}, \qquad \pi=\begin{bmatrix}\tfrac13&\tfrac23\end{bmatrix}.

The rows sum to \(\tfrac12+\tfrac12=1\) and \(\tfrac14+\tfrac34=1\). The correct row-vector calculation gives \(\pi P=(\tfrac16+\tfrac16,\ \tfrac16+\tfrac12)=(\tfrac13,\tfrac23)=\pi\), so \(\pi\) is stationary. The incorrect column-vector calculation would use \(P(\tfrac13,\tfrac23)^\mathsf{T}\), whose first coordinate is \(\tfrac16+\tfrac13=\tfrac12\) and second is \(\tfrac1{12}+\tfrac12=\tfrac7{12}\). That different product is not the stationarity test under the row-vector convention.

Stationarity is a fixed-point property, not a claim about convergence from other initial distributions. A stationary distribution may exist even if a chain started elsewhere does not approach it, and an infinite invariant measure is not itself a probability law. Keeping the normalization and vector orientation explicit prevents these distinct ideas from being conflated.

Check Your Understanding

For each question, state whether the object is a stationary distribution, an invariant measure, both, or neither, and justify your answer from the defining equations.

If an invariant measure has total mass \(5\), what normalization produces a stationary distribution, and why does the invariant equation remain valid?
For the shift chain on \(\mathbb Z\), verify the incoming-mass equation at \(j=-2\) for the counting measure \(\nu_i=1\).
Can the zero measure be invariant? Can it be a stationary distribution? Explain using the total-mass condition.
For a countable chain, what does the coordinate equation \(\nu_j=\sum_i\nu_i p_{ij}\) say about mass entering state \(j\)?
Why does computing \(P\pi^\mathsf{T}\) test the wrong equation when distributions are row vectors?

Stationary Distributions and Invariant Measures

What You'll Learn