Stationary Marginals from an Invariant Initial Law

Invariance Persists at Every Time

A transition matrix describes how a chain’s distribution changes in one step. If a probability distribution is unchanged by that step, it is natural to ask whether it stays unchanged after many steps. The answer follows by repeatedly applying the same transition rule: an invariant initial law is the marginal distribution at every time. This statement concerns a chain started from that law; it does not say that every initial distribution converges to it.

Let \((X_n)_{n\ge0}\) be a time-homogeneous Markov chain on a finite or countable state space \(S\), defined on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). Let \(\mathcal{F}_n=\sigma(X_0,\ldots,X_n)\) be its natural filtration. Its transition matrix is \(P=(p_{ij})\), where \(p_{ij}=\mathbb{P}(X_{n+1}=j\mid X_n=i)\); each row is a probability distribution. Write the law of \(X_n\) as the row vector \(\mu_n\), so that \(\mu_n(i)=\mathbb{P}(X_n=i)\) and the one-step distribution rule is \(\mu_{n+1}=\mu_nP\).

The definition says that one application of the transition rule leaves \(\pi\) unchanged. The result below shows that this one-step property is enough for all times. It does not require irreducibility, aperiodicity, or finiteness of the state space; the transition probabilities must be defined for every state, and \(P\) must be the same at every time.

Proof. We use induction on \(n\). At \(n=0\), \(\mu_0=\pi\) by assumption. Suppose \(\mu_n=\pi\). The Markov property gives \(\mathbb{P}(X_{n+1}=j)=\sum_{i\in S}\mathbb{P}(X_n=i)p_{ij}\) for each \(j\in S\). Thus \(\mu_{n+1}(j)=\sum_{i\in S}\pi_i p_{ij}=(\pi P)_j=\pi_j\). The sum is well-defined because its terms are nonnegative and bounded in total by \(\sum_i\pi_i=1\). Hence \(\mu_{n+1}=\pi\), completing the induction. \(\square\)

Equivalently, repeated application of the distribution rule gives \(\mu_n=\mu_0P^n\). If \(\mu_0=\pi\), then \(\mu_n=\pi P^n=\pi\): this follows by induction from \(\pi P=\pi\), since \(\pi P^{n+1}=(\pi P^n)P=\pi P=\pi\) whenever \(\pi P^n=\pi\). The one-step proof makes explicit how each marginal is obtained; the matrix-power form is often convenient for calculations.

Finite-State Calculations

In each example, distributions are row vectors and states are listed in the same order in the matrix and vector. Before using a transition matrix, check its row sums. To verify an invariant law, compute every coordinate of \(\pi P\) and check that the proposed probabilities sum to one. Once verified, the theorem gives all later marginals without requiring a fresh multiplication at each time.

Consequences and Limits of the Result

Stationary marginals make one-time probability calculations straightforward. If \(A\subseteq S\), then for a chain started from \(\pi\), \(\mathbb{P}(X_n\in A)=\sum_{i\in A}\pi_i\) for every \(n\). More generally, for any bounded function \(f:S\to\mathbb{R}\), the expectation of \(f(X_n)\) does not depend on \(n\).

Proof. Since \(\mu_n=\pi\), the definition of expectation for a countable-valued random variable gives \(\mathbb{E}[f(X_n)]=\sum_{i\in S}f(i)\mathbb{P}(X_n=i)=\sum_{i\in S}f(i)\pi_i\). The sum is absolutely convergent: if \(|f(i)|\le C\), then \(\sum_i|f(i)|\pi_i\le C\sum_i\pi_i=C\). For an event \(A\), take \(f=\mathbf{1}_A\), obtaining \(\mathbb{P}(X_n\in A)=\sum_{i\in A}\pi_i\). \(\square\)

This result is useful whenever a question concerns the state of a chain at a specified time and the chain is initialized from an invariant law. It lets us answer those questions using \(\pi\) alone, without computing \(P^n\). It also identifies the scope of the conclusion: it gives the marginal law at each time, not a claim that the realized state is constant, nor a claim that a chain started from an arbitrary law approaches \(\pi\). The three-state cycle illustrates why the latter claim would require additional hypotheses.

Common pitfall: treating a row distribution as a column vector. With the two-state matrix above, the convention is \(\mu_{n+1}=\mu_nP\), so \(\pi P=(\tfrac23,\tfrac13)\). If one incorrectly computes \(P\begin{bmatrix}\tfrac23\\\tfrac13\end{bmatrix}\), the result is \(\begin{bmatrix}\tfrac7{12}\\\tfrac12\end{bmatrix}\): its first coordinate is \(\tfrac34\cdot\tfrac23+\tfrac14\cdot\tfrac13=\tfrac7{12}\), and its second is \(\tfrac12\cdot\tfrac23+\tfrac12\cdot\tfrac13=\tfrac12\). That calculation does not test invariance under the row-vector convention. The correction is to multiply the row vector on the left, as in the verified calculation in the example.